Updating ur and simd sections

mapping_draft.Rmd

@@ -307,9 +307,6 @@ ggplot() +
           legend.text = element_text(size = 8),
           legend.position = c(0.9, 0.25))
 
-# Save plot to PDF
-ggsave("admin1_choropleth_ecs.pdf")
-
 ```
 
 ```{r plot_admin_ecs_normed_choropleth, fig.width=4, fig.show="hold", fig.cap="Figure 2"}
@@ -388,10 +385,12 @@ admin.df <- data.frame(admin_lev1)
 # Need to flatten admin_lev1 based on all the count columns and generate using ggplot
 
 admin_gathered <- gather(data.frame(admin_lev1), key="group_type", value="number", ecs_count, transition_count, dtas_count, permaculture_count)
+
+ggplot(admin_gathered, aes(fill=group_type, y=number, x=name)) + geom_bar(position="dodge", stat="identity") + coord_flip() + labs(title = "Figure 4", subtitle="Comparison of Groups by Admin Region", fill = "Groups")
 ```
 
+
 ```{r create_choropleth_others, fig.width=2.65, fig.show="hold", fig.cap="Figure 4}
-ggplot(admin_gathered, aes(fill=group_type, y=number, x=name)) + geom_bar(position="dodge", stat="identity") + coord_flip() + labs(title = "Figure 4", subtitle="Comparison of Groups by Admin Region", fill = "Groups")
 
 # TODO: add bubbleplot with dots on map, as here: https://www.r-graph-gallery.com/330-bubble-map-with-ggplot2/ or here: https://www.r-graph-gallery.com/177-map-with-proportional-symbols/
 
@@ -482,12 +481,10 @@ A wide variety of historians and sociologists of religion have noted the regiona
 So why provide this kind of data (i.e. at the level of individual churches) when more granular data (i.e. at the level of individuals persons) is available in the form of the census and related parallel publications such as the 2008 Scottish Environmental Attitudes survey? We believe that mapping places of worship provides a useful intermediate level of analysis and may complement our more atomised understanding of EA which has been assessed at the level of individual persons to date. Because representation within some administrative areas of Scotland, can lead to a small number of data points, we have kept analysis to a National level and have not provided more specific administrative-area level calculations.
 
 ```{r create_ecs_denomination_table}
-# TODO: Need to find a prettier way to do this:
+knitr::kable(ecs@data[["denomination"]], caption = 'ECS by denomination')
-table(ecs@data[["denomination"]])
 # TODO: Add dataframe with overall church numbers for the UK from "ScottishChurches" dataset by JK
 ```
 
-![][Figure5]
 
 As one might expect, there is a strong representation of the Church of Scotland, almost 74% of eco-congregations, with this number remaining the same when we only count awarded sites. We can confirm, on the basis of this analysis that ECS has a disproportional representation by Church of Scotland churches. At the 2002 church census count, it only represented 40.20% of Scottish churches (1666 of 4144 total churches). Similarly, on the 2011 Scottish census, only 32.44% of persons claimed to be members of the Church of Scotland. We can adjust this representation to 60%, if one excludes the 2,445,204 persons (46% of the total on the census) who reported either "no religion" or adherence to a religious tradition not currently represented among the eco-congregation sites. There is a slight over-representation by the United Reformed church, though this seems considerably more dramatic when one takes into account the fact that this is a trebling or more of their overall share of Scottish churches. The URC makes up only sightly more than 1% of church buildings in Scotland and a tiny 0.04% of respondents to the 2011 census. The Scottish Episcopal church hovers right around a proportional representation within ECS. More concerning are the significant underrepresentation by Roman Catholic churches, Baptists, the Free Church of Scotland, and other independent churches. 
 
@@ -497,9 +494,10 @@ While Roman Catholic churches make up just over 10% of the church buildings in S
 
 ```{r ur8fold}
 # read in relevant polygons for UR8fold scale
+if (file.exists("data/SG_UrbanRural_2016") == FALSE) {
 download.file("http://sedsh127.sedsh.gov.uk/Atom_data/ScotGov/ZippedShapefiles/SG_UrbanRural_2016.zip", 
               destfile = "data/SG_UrbanRural_2016.zip")
-unzip("data/SG_UrbanRural_2016.zip", exdir = "data")
+unzip("data/SG_UrbanRural_2016.zip", exdir = "data")}
 urbanrural <- readOGR("./data", "SG_UrbanRural_2016") 
 
 # TODO: worth considering uploading data to zenodo for long-term reproducibility as ScotGov shuffles this stuff around periodically breaking URLs
@@ -525,7 +523,7 @@ urbanrural$permaculture_percent<- prop.table(urbanrural$permaculture_count)
 urbanrural_df<-data.frame(urbanrural)
 ```
 
-Rather than bifurcate congregations into an urban/rural dichotomy, for this study we used the Scottish Government's eight-point remoteness scale to categorise eco-congregations along a spectrum of highly populated to remote areas. This 8-fold scale (calculated biennially) offers a more nuanced measurement that combines measurements of remoteness and population along the following lines:
+Rather than bifurcate congregations into an urban/rural dichotomy, for this study we used the Scottish Government's six-point remoteness scale to categorise eco-congregations along a spectrum of highly populated to remote areas. This 8-fold scale (calculated biennially) offers a more nuanced measurement that combines measurements of remoteness and population along the following lines:
 
 1. Large Urban Areas - Settlements of over 125,000 people.
 2. Other Urban Areas - Settlements of 10,000 to 125,000 people. 
@@ -540,10 +538,52 @@ The key question which this analysis seeks to answer is whether ECS, or the othe
 
 Of all the groups surveyed in this study, Eco-Congregation Scotland is the most heavily concentrated in large urban areas (33.53%), exceeding by almost 50% the rate for all places of worship (22.96% in large urban areas). Transition is a much more modest 20% and development trusts a bit lower at 15%. It is interesting to note that the rate of ECS concentration in these large urban areas matches the level of overall population distribution (34.5%). On the other end of the scale, Eco-Congregation Scotland is the least concentrated in remote rural areas (with 3.93% on level 7 and 5.44% on level 8 on the urban-rural scale), though again, they correlate roughly to the general population distribution (3.2% and 2.9% respectively). Places of worship outpace both the population of Scotland and the footprint of Eco-Congregation Scotland, with 14.98% in very remote rural areas, but this is exceeded by transition at 16.47% and both by Scottish community development trusts at 32.14%. So while Eco-Congregation Scotland correlates roughly with Scottish population distribution across the urban-rural scale, it has a considerably more urban profile than either of the other two groups surveyed.
 
+```{r create_ur_barplot}
+# comvert admin back to dataframe for analysis
+urbanrural.df <- data.frame(urbanrural)
+
+# Need to flatten urbanrural based on all the count columns and generate using ggplot
+
+urbanrural_gathered <- gather(data.frame(urbanrural), key="group_type", value="number", ecs_count, transition_count, dtas_count, permaculture_count)
+
+# TODO: change to ur category column
+ggplot(admin_gathered, aes(fill=group_type, y=number, x=name)) + geom_bar(position="dodge", stat="identity") + coord_flip() + labs(title = "Figure 7", subtitle="Comparison of Groups by UrbanRural category", fill = "Groups")
+```
+
 ```{r create_urbanrural_ecs_chart_choropleth}
-# Create bar chart
+
-# TODO: use ggplot here to generate stacked bar chart showing representation across 8 urban/rural bands from urbanrural_df
+# Prepare urbanrural for tidyr and reinsert dropped columns
-# TODO: generate map based on urbanrural polygons, shade based on 8-fold categories, draw transparent dots onto polygons
+# TODO: change to relevant urbanrural column names
+
+names(admin_lev1)[names(admin_lev1) == "newcode"] <- "id"
+urbanrural@data$id <- as.integer(rownames(admin_lev1@data))
+urbanrural@data$id <- urbanrural@data$id-1
+urbanrural_fortified <- tidy(urbanrural)
+urbanrural_fortified <- join(urbanrural_fortified,urbanrural@data, by="id")
+
+ggplot() + 
+  geom_polygon(aes(x = long, y = lat, group = group, 
+                   fill = cut_number(urbanrural$ecs_count, 5)), 
+               data = admin_lev1_fortified, 
+               colour = 'black', 
+               alpha = .7, 
+               size = .3) + 
+  viridis::scale_fill_viridis(discrete = TRUE) + 
+  labs(x = NULL, y = NULL, fill = "Groups",
+       title = "Figure 1", 
+       subtitle="Eco-Congregation Scotland concentrations in Urban Rural 8-fold classifications",
+       caption = paste("Jeremy H. Kidwell :: jeremykidwell.info",
+                       "Data: UK Data Service (OGL) & Jeremy H. Kidwell",
+                       "You may redistribute this graphic under the terms of the CC-by-SA 4.0 license.",
+                       sep = "\n")) + 
+  theme_void() + 
+  theme(text = element_text(family = "Arial Narrow", size = 8),
+          plot.title = element_text(size = 12, face = "bold"),
+          plot.margin = unit(c(0, 0.25, 0.0, 0.25), "in"),
+          panel.border = element_rect(fill = NA, colour = "#cccccc"),
+          legend.text = element_text(size = 8),
+          legend.position = c(0.9, 0.25))
+          
 
 pdf(file="figures/urbanrural_choropleth_ecs.pdf", width=4, height=4)
 bins <- unique(quantile(urbanrural$ecs_count, seq(0,1,length.out=30)))
@@ -559,16 +599,16 @@ title(main="Figure 5", sub="Eco-Congregation Scotland\nconcentrations in Urban R
 dev.off()
 ```
 
-![][Figure6 map of ECS concentration by UR Areas Urban - Rural via Brown to Green]
-
 
 # Wealth, Employment, and Literacy
 
 ```{r simd}
 # read in relevant polygons, Scottish Index of Multiple deprivation
+if (file.exists("data/c_dz_11.shp") == FALSE) {
 download.file("http://simd.scot/2016/data/simd2016_withgeog.zip", 
               destfile = "data/simd2016_withgeog.zip")
 unzip("data/simd2016_withgeog.zip", exdir = "data", junkpaths = TRUE)
+}
 simd_shapes <- readOGR("./data", "sc_dz_11") 
 simd_indicators <- read.csv("./data/simd2016_withinds.csv")
 simd <- merge(x=simd_shapes, y=simd_indicators, by="Data_Zone")
@@ -637,6 +677,7 @@ dtas_null<-dtas[!complete.cases(dtas),]
 coordinates(dtas) <- c("X", "Y")
 proj4string(dtas) <- proj4string(admin_lev1)
 
+# TODO: change names to match simd2016 conventions
 # STAGE 3a, calculate sums based on SIMD12R columns and generate new integer sets with quintile count data
 simd_rownames = c("Quintile 1","Quintile 2","Quintile 3","Quintile 4","Quintile 5")
 simdr12_ecs = c((sum(ecs_clean$SIMDR12<1301)), (sum(ecs_clean$SIMDR12 > 1300 & ecs_clean$SIMDR12 < 2602)), (sum(ecs_clean$SIMDR12 > 2601 & ecs_clean$SIMDR12 < 3903)), (sum(ecs_clean$SIMDR12 > 3902 & ecs_clean$SIMDR12 < 5204)), (sum(ecs_clean$SIMDR12 > 5203 & ecs_clean$SIMDR12 < 6505)))
@@ -698,15 +739,26 @@ estr12_dtas_percent<- prop.table(estr12_dtas)
 simd = data.frame(simdr12_ecs, simdr12_ecs_percent, incr12_ecs, incr12_ecs_percent, empr12_ecs, empr12_ecs_percent, her12_ecs, her12_ecs_percent, estr12_ecs, estr12_ecs_percent, simdr12_transition, simdr12_transition_percent, incr12_transition, incr12_transition_percent, empr12_transition, empr12_transition_percent, her12_transition, her12_transition_percent, estr12_transition, estr12_transition_percent, simdr12_permaculture, simdr12_permaculture_percent, incr12_permaculture, incr12_permaculture_percent, empr12_permaculture, empr12_permaculture_percent, her12_permaculture, her12_permaculture_percent, estr12_permaculture, estr12_permaculture_percent, simdr12_dtas, simdr12_dtas_percent, incr12_dtas, incr12_dtas_percent, empr12_dtas, empr12_dtas_percent, her12_dtas, her12_dtas_percent, estr12_dtas, estr12_dtas_percent)
 write.csv(simd, "derivedData/simd.csv", row.names=FALSE)
 
-
 simd_percents_only = data.frame(simd_rownames, simdr12_ecs_percent, incr12_ecs_percent, empr12_ecs_percent, her12_ecs_percent, estr12_ecs_percent, simdr12_transition_percent, incr12_transition_percent, empr12_transition_percent, her12_transition_percent, estr12_transition_percent, simdr12_permaculture_percent, incr12_permaculture_percent, empr12_permaculture_percent, her12_permaculture_percent, estr12_permaculture_percent, simdr12_dtas_percent, incr12_dtas_percent, empr12_dtas_percent, her12_dtas_percent, estr12_dtas_percent)
 write.csv(simd_percents_only, "derivedData/simd_percents_only.csv", row.names=FALSE)
+```
 
-
+```{r create_simd_barplot}
 # STAGE 5, generate cool charts
 
+# comvert admin back to dataframe for analysis
+urbanrural.df <- data.frame(urbanrural)
+
+# Need to flatten urbanrural based on all the count columns and generate using ggplot
+
+urbanrural_gathered <- gather(data.frame(urbanrural), key="group_type", value="number", ecs_count, transition_count, dtas_count, permaculture_count)
+
+# TODO: change to ur category column
+ggplot(admin_gathered, aes(fill=group_type, y=number, x=name)) + geom_bar(position="dodge", stat="identity") + coord_flip() + labs(title = "Figure 7", subtitle="Comparison of Groups by UrbanRural category", fill = "Groups")
+
 # clustered bar charts
 # convert to long format
+
 library(reshape2)
 simd_percents_only_long <- melt(simd_percents_only, id.vars = "simd_rownames", 
               measure.vars = grep("^12", names(simd_percents_only), value = TRUE))
@@ -714,7 +766,7 @@ simd_percents_only_long <- melt(simd_percents_only, id.vars = "simd_rownames",
 qplot(data=simd_percents_only_long , geom="bar", fill=(factor(simd_rownames)))
 
 # jitterplot option, from Teutonico 2015, p. 63
-
+https://ggplot2.tidyverse.org/reference/geom_jitter.html
 ```
 
 Another crucial point of assessment relates to the relation of Eco-Congregation communities to the Scottish Index of Multiple Deprivation. This instrument aggregates a large variety of factors which can lead to deprivation including crime rates, employment levels, access to services (implicating remoteness), and literacy. By assessing ECS, Transition, and dtas against the deprivation scale, we can assess whether eco-congregations fall within particular demographics and also whether the fully aggregated SIMD measurement provides a useful point of comparison for our purposes. The SIMD essentially divides Scotland into 6407 geographic zones and then ranks them based on their relative deprivation. This data set can be split into any number of groups, but for our purposes we have settled on Quintiles, splitting the SIMD data set at every 1302 entries. We then measured where each transition group, ECS, and dtas fell within these zones and calculated how they fell into these five quintiles, from more to least deprived.
@@ -723,6 +775,7 @@ The first, and most compelling finding is that, in general Eco-Congregation Scot
 
 We can find divergence between transition communities and eco-congregation when we split out SIMD domains. In the lowest quartile, measuring exclusively for the income domain, ECS is more represented (11%) - roughly the same as DTAS (12%), and transition is less (6%) represented. In general (as shown on the chart in Appendix D), these trends hold when representation of our groups are measured within other non-remoteness domains of the SIMD. Our basic conclusion is that transition towns are least likely to operate within the lowest quartile of SIMD and DTASs are most likely, with ECS somewhere in the middle. Given the general disparity against the presence of places of worship, it seems fair to suggest that this might be an area for improvement, perhaps even worth developing a special programme which might target areas in SIMD quartile 1 for eco-congregation outreach. This might be considered particularly in light of the starkest underrepresentation of ECS and transition within the SIMD domain of education, skills, and training.
 
+# Proximity to "Wilderness"
 
 # Appendix A