hacking_religion_textbook/hacking_religion/chapter_1.qmd

# The 2021 UK Census

## Your first project: building a pie chart

Let's start by importing some data into R. Because R is what is called an object-oriented programming language, we'll always take our information and give it a home inside a named object. There are many different kinds of objects, which you can specify, but usually R will assign a type that seems to fit best.

[If you'd like to explore this all in a bit more depth, you can find a very helpful summary in R for Data Science, chapter 8, ["data import"](https://r4ds.hadley.nz/data-import#reading-data-from-a-file).]{.aside}

In the example below, we're going to read in data from a comma separated value file ("csv") which has rows of information on separate lines in a text file with each column separated by a comma. This is one of the standard plain text file formats. R has a function you can use to import this efficiently called "read.csv". Each line of code in R usually starts with the object, and then follows with instructions on what we're going to put inside it, where that comes from, and how to format it:


```{r}
#| include: true
#| label: fig-polar
setwd("/Users/kidwellj/gits/hacking_religion_textbook/hacking_religion")
library(here) # much better way to manage working paths in R across multiple instances
library(tidyverse)
here::i_am("chapter_1.qmd")
uk_census_2021_religion <- read.csv(here("example_data", "census2021-ts030-rgn.csv")) 
```

### Examining data:

What's in the table? You can take a quick look at either the top of the data frame, or the bottom using one of the following commands:

```{r .column-page}
head(uk_census_2021_religion)
```

This is actually a fairly ugly table, so I'll use an R tool called kable to give you prettier tables in the future, like this:

```{r}
knitr::kable(head(uk_census_2021_religion))
```

You can see how I've nested the previous command inside the `kable` command. For reference, in some cases when you're working with really complex scripts with many different libraries and functions, they may end up with functions that have the same name. You can specify the library where the function is meant to come from by preceding it with :: as we've done `knitr::` above. The same kind of output can be gotten using `tail`:

```{r}
knitr::kable(tail(uk_census_2021_religion))
```

### Parsing and Exploring your data

The first thing you're going to want to do is to take a smaller subset of a large data set, either by filtering out certain columns or rows. Now let's say we want to just work with the data from the West Midlands, and we'd like to omit some of the columns. We can choose a specific range of columns using `select`, like this:

You can use the `filter` command to do this. To give an example, `filter` can pick a single row in the following way:


```{r}
uk_census_2021_religion_wmids <- uk_census_2021_religion %>% 
  filter(geography=="West Midlands")  
```

Now we'll use select in a different way to narrow our data to specific columns that are needed (no totals!).

[Some readers will want to pause here and check out Hadley Wickham's "R For Data Science" book, in the section, ["Data visualisation"](https://r4ds.hadley.nz/data-visualize#introduction) to get a fuller explanation of how to explore your data.]{.aside}

In keeping with my goal to demonstrate data science through examples, we're going to move on to producing some snappy looking charts for this data.


## Making your first chart

We've got a nice lean set of data, so now it's time to visualise this. We'll start by making a pie chart:

```{r}
uk_census_2021_religion_wmids <- uk_census_2021_religion_wmids %>% select(no_religion:no_response)
uk_census_2021_religion_wmids <- gather(uk_census_2021_religion_wmids)
```


There are two basic ways to do visualisations in R. You can work with basic functions in R, often called "base R" or you can work with an alternative library called ggplot:

#### Base R

```{r}
df <- uk_census_2021_religion_wmids[order(uk_census_2021_religion_wmids$value,decreasing = TRUE),]
barplot(height=df$value, names=df$key)
``` 


#### GGPlot

```{r}
ggplot(uk_census_2021_religion_wmids, aes(x = key, y = value)) +   # <1>
  geom_bar(stat = "identity")                   # <1>
ggplot(uk_census_2021_religion_wmids, aes(x= reorder(key,-value),value)) + geom_bar(stat ="identity") # <2>
```

1. First we'll plot the data using `ggplot` and then...
2. We'll re-order the column by size.

Let's assume we're working with a data set that doesn't include a "totals" column and that we might want to get sums for each column. This is pretty easy to do in R:

```{r}
uk_census_2021_religion_totals <- uk_census_2021_religion %>% select(no_religion:no_response) # <1>
uk_census_2021_religion_totals <- uk_census_2021_religion_totals %>%
   summarise(across(everything(), ~ sum(., na.rm = TRUE)))  # <2>
uk_census_2021_religion_totals <- gather(uk_census_2021_religion_totals) # <3>
ggplot(uk_census_2021_religion_totals, aes(x= reorder(key,-value),value)) + geom_bar(stat ="identity") # <4>
```

1. First, remove the column with region names and the totals for the regions as we want just integer data. 
2. Second calculate the totals. In this example we use the tidyverse library `dplyr()`, but you can also do this using base R with `colsums()` like this: `uk_census_2021_religion_totals <- colSums(uk_census_2021_religion_totals, na.rm = TRUE)`. The downside with base R is that you'll also need to convert the result into a dataframe for `ggplot` like this: `uk_census_2021_religion_totals <- as.data.frame(uk_census_2021_religion_totals)`
3. In order to visualise this data using ggplot, we need to shift this data from wide to long format. This is a quick job using gather()
4. Now plot it out and have a look!

You might have noticed that these two dataframes give us somewhat different results. But with data science, it's much more interesting to compare these two side-by-side in a visualisation. We can join these two dataframes and plot the bars side by side using `bind()` - which can be done by columns with cbind() and rows using rbind():

```{r}
uk_census_2021_religion_merged <- rbind(uk_census_2021_religion_totals, uk_census_2021_religion_wmids)
```

Do you notice there's going to be a problem here? How can we tell one set from the other? We need to add in something idenfiable first! This isn't too hard to do as we can simply create a new column for each with identifiable information before we bind them:

```{r}
uk_census_2021_religion_totals$dataset <- c("totals")
uk_census_2021_religion_wmids$dataset <- c("wmids")
uk_census_2021_religion_merged <- rbind(uk_census_2021_religion_totals, uk_census_2021_religion_wmids)
```

Now we're ready to plot out our data as a grouped barplot:

```{r}
ggplot(uk_census_2021_religion_merged, aes(fill=dataset, x= reorder(key,-value), value)) + geom_bar(position="dodge", stat ="identity")
```
If you're looking closely, you will notice that I've added two elements to our previous ggplot. I've asked ggplot to fill in the columns with reference to the `dataset` column we've just created. Then I've also asked ggplot to alter the `position="dodge"` which places bars side by side rather than stacked on top of one another. You can give it a try without this instruction to see how this works. We will use stacked bars in a later chapter, so remember this feature.

If you inspect our chart, you can see that we're getting closer, but it's not really that helpful to compare the totals. What we need to do is get percentages that can be compared side by side. This is easy to do using another `dplyr` feature `mutate`:

```{r}
uk_census_2021_religion_totals <- uk_census_2021_religion_totals %>% 
  dplyr::mutate(perc = scales::percent(value / sum(value), accuracy = 0.1, trim = FALSE)) # <3>
uk_census_2021_religion_wmids <- uk_census_2021_religion_wmids %>% 
  dplyr::mutate(perc = scales::percent(value / sum(value), accuracy = 0.1, trim = FALSE)) # <3>
uk_census_2021_religion_merged <- rbind(uk_census_2021_religion_totals, uk_census_2021_religion_wmids)
ggplot(uk_census_2021_religion_merged, aes(fill=dataset, x=key, y=perc)) + geom_bar(position="dodge", stat ="identity")
```
Now you can see a very rough comparison

Add time series data for 2001 and 2011 census, change to grouped bar plot:

https://r-graphics.org/recipe-bar-graph-grouped-bar#discussion-8

<!--
Reference on callout box syntax here: https://quarto.org/docs/authoring/callouts.html
-->






# References {.unnumbered}

::: {#refs}
:::