Starting some coding

Let’s first call our packages. I am using the package packman to help me manage my libraries.

pacman::p_load(tidyverse, gapminder, kableExtra, tidyr, ggthemes, patchwork, broom)

This is a tidy data:

library(gapminder)
gapminder

## # A tibble: 1,704 x 6
##    country     continent  year lifeExp      pop gdpPercap
##    <fct>       <fct>     <int>   <dbl>    <int>     <dbl>
##  1 Afghanistan Asia       1952    28.8  8425333      779.
##  2 Afghanistan Asia       1957    30.3  9240934      821.
##  3 Afghanistan Asia       1962    32.0 10267083      853.
##  4 Afghanistan Asia       1967    34.0 11537966      836.
##  5 Afghanistan Asia       1972    36.1 13079460      740.
##  6 Afghanistan Asia       1977    38.4 14880372      786.
##  7 Afghanistan Asia       1982    39.9 12881816      978.
##  8 Afghanistan Asia       1987    40.8 13867957      852.
##  9 Afghanistan Asia       1992    41.7 16317921      649.
## 10 Afghanistan Asia       1997    41.8 22227415      635.
## # … with 1,694 more rows

There are three interrelated rules which make a dataset tidy.

• Each variable must have its own column.
• Each observation must have its own row.
• Each value must have its own cell.

Why is it important to have a tidy dataset? A more general answer is that to map a variable to a plot, you need this variables to be defined as an single column. A more practical answer is that all the tidyverse packages (dplyr, purrr, tidyr, ggplot…) work with tidy data.

Getting our own data

Throughout this workshop, we will work with the pooling data for the last Presidential Election in the US aggregated by The Economist. Let’s download the data:

data <- read_csv("https://docs.google.com/spreadsheets/d/e/2PACX-1vQ56fySJKLL18Lipu1_i3ID9JE06voJEz2EXm6JW4Vh11zmndyTwejMavuNntzIWLY0RyhA1UsVEen0/pub?gid=0&single=true&output=csv")

Pivoting to make the data tidy (Pivot Longer)

Our main variables of interest are in the wide format (Trump and Biden’s Polls Results). This is wide dataset because we are spreading the variable vote choice is spread among two columns. To make it a long data set, we will use the function pivot_longer from the tidyr package.

data_long <- data %>% 
  pivot_longer(cols=c(biden,trump), 
               names_to="Vote_Choice", 
               values_to="Vote") %>%
  mutate(end.date=as.Date(str_replace_all(end.date, "/", "-"), format = "%m-%d-%Y"))

We have three inputs in pivot_longer:

• cols: the variables you want to convert from wide to long.
• names_to: new variable for the columns names from the wide data.
• values_to: new variable for the values from the wide data.

And now:

data_long

## # A tibble: 3,228 x 17
##    state pollster        sponsor start.date end.date   entry.date.time..et.
##    <chr> <chr>           <chr>   <chr>      <date>     <chr>               
##  1 MT    Change Research <NA>    10/29/2020 2020-11-02 11/2/2020 23:42:00  
##  2 MT    Change Research <NA>    10/29/2020 2020-11-02 11/2/2020 23:42:00  
##  3 ME    Change Research <NA>    10/29/2020 2020-11-02 11/2/2020 23:42:00  
##  4 ME    Change Research <NA>    10/29/2020 2020-11-02 11/2/2020 23:42:00  
##  5 IA    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
##  6 IA    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
##  7 WI    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
##  8 WI    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
##  9 PA    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
## 10 PA    Change Research <NA>    10/29/2020 2020-11-01 11/2/2020 23:42:00  
## # … with 3,218 more rows, and 11 more variables: number.of.observations <dbl>,
## #   population <chr>, mode <chr>, biden_margin <dbl>, other <dbl>,
## #   undecided <dbl>, url <chr>, include <lgl>, notes <chr>, Vote_Choice <chr>,
## #   Vote <dbl>

With this tidy dataset, we can start our visualization.

Let’s now separate the data for our later use

national_polls <- data_long %>% filter(state=="--")
state_polls <- data_long %>% filter(state!="--")
swing_states <- c("PA", "FL", "GA", "AZ", "IO", "OH", "TX", "MI", "WI")

Mapping is how you connect your data and variables with the visual representations of a graph. We will do this in three steps.

1. The Data Step: Tell ggplot what your data is.

2. The Mapping Step: Tell ggplot what variables -> visuals you want to see.

3. The Geom Step: Tell ggplot how you want to see

Toy Example

Let’s plot all the polls over time using a scatter plot

ggplot(data=national_polls, # the data step
       aes(x=end.date, y=Vote)) + # the map step
geom_point()

Bit of a mess. But we will make it prettier.

What do you see?

• How many variables?

• How many different Aesthetics?

• What else?

Some Aesthethics Examples:

ggplot(data=national_polls, # the data step
       aes(x=end.date, y=Vote, 
           color=Vote_Choice)) + # the map step
geom_point()

Shape

ggplot(data=national_polls, # the data step
       aes(x=end.date, y=Vote,
           color=Vote_Choice,
           shape=Vote_Choice)) + # the map step
geom_point() 

Alpha

ggplot(data=national_polls, # the data step
       aes(x=end.date, y=Vote,
           color=Vote_Choice,
           shape=Vote_Choice, 
           alpha=end.date)) + # the map step
geom_point() 

Linetype

ggplot(data=national_polls, # the data step
       aes(x=end.date, y=Vote,
           color=Vote_Choice,
           alpha=end.date, 
           linetype=Vote_Choice)) + # the map step
geom_smooth()

Infinite possibilities to show variations on your dataset and maps your variables to visuals. We will go over some examples of more complete visualizations. Now let’s see some examples of geoms you might consider.

Some Geom Examples

ggplot(data=national_polls,
       aes(x=end.date, y=Vote, color=Vote_Choice, fill=Vote_Choice)) + 
geom_point(alpha=.2) +
geom_smooth()

ggplot(data=national_polls,
       aes(x=end.date)) +
geom_density(fill="steelblue")

# Another one using the facet_wrap
ggplot(data=state_polls %>% filter(state%in%swing_states)) +
  geom_density(aes(x=end.date, fill=state), alpha=.3) +
  facet_wrap(~state, ncol=3)

ggplot(data=data_long,
       aes(x=end.date, fill=mode)) +
geom_bar() +
scale_fill_brewer(palette = "Set3")

# Another example
ggplot(state_polls %>% filter(state%in%swing_states),
       aes(x=Vote,y=fct_rev(state), fill=Vote_Choice)) +
  geom_boxplot() +
  scale_fill_manual(values=c("biden"="blue","trump"="red")) 

Part 4: Adjust scales, labels, titles, and more.

After you are set on the mapping and geoms, the next step is to adjust the scale of your the graph. These functions are usually on the form: scale_aesthethic_type. These functions usually change how the aesthethics are presented. For exemple:

• scale_x_log10: To convert the numeric axis to the log scale
• scale_y_reverse: To reverse the scale
• scale_fill_manual: To create your own discrete set of fill.
• scale_colour_brewer(): Change the Pallet of Colours

From now on, I will present more complete graphs, that would be similar to one I would add in a research paper.

p <- ggplot(data=national_polls,
       aes(x=end.date, y=Vote, color=Vote_Choice, shape=Vote_Choice, fill=Vote_Choice)) + 
geom_point(alpha=.2) +
geom_smooth() +
scale_shape_manual(values =c(21, 23)) +
scale_fill_manual(values=c("red", "blue")) +
scale_color_manual(values=c("red", "blue"), , labels=c("Biden", "Trump"), 
                  name= "Vote Choice") +
scale_x_date(date_breaks = "1 month", date_labels = "%b %d") +
guides(fill=FALSE, shape=FALSE) +
labs(x = "End of the Poll", y = "Results",
         title = "Polls US Presidential Election",
         subtitle = "",
         caption = "Source: The Economist")  
p

A single model

Let’s start with a simple and silly model. We will estimate how Biden results improved over time.

# Separate the data
biden <- national_polls %>%
      filter(Vote_Choice=="biden") %>%
      mutate(first_day=min(end.date, na.rm=TRUE), 
             days=as.numeric(end.date-first_day))


# simple linear model

lm_time <- lm(Vote~ days, data=biden)

With this model, most of us learned to extract quantities using functions likes coef, predict, summary, among others. A better approach is to use the functions from the broom package. We have three options:

• tidy: to extract the model main parameters
• augment: to extract observation-level statistics (predictions)
• glance: to extract model-level statistics.

Extract Quantities

# a data frame
results <- tidy(lm_time)
results

## # A tibble: 2 x 5
##   term        estimate std.error statistic  p.value
##   <chr>          <dbl>     <dbl>     <dbl>    <dbl>
## 1 (Intercept)  46.7      0.318       147.  0       
## 2 days          0.0164   0.00156      10.5 2.06e-23

You can plot these quantities directly. The graph for you paper is basically ready.

# a data frame
results <- tidy(lm_time)
results %>% 
  filter(term!="(Intercept)") %>%
  mutate(up=estimate + 1.96*std.error,
         lb=estimate - 1.96*std.error)  %>%
  ggplot(data=., aes(x=term,y=estimate, ymin=lb, ymax=up)) +
  geom_pointrange(shape=21, fill="blue", color="black", alpha=.8) +
  geom_hline(yintercept = 0, linetype="dashed", color="gray") +
  coord_flip() +
  theme_minimal() 

Predicted Values with Augment

augment(lm_time)

## # A tibble: 495 x 8
##     Vote  days .fitted .resid .std.resid    .hat .sigma   .cooksd
##    <dbl> <dbl>   <dbl>  <dbl>      <dbl>   <dbl>  <dbl>     <dbl>
##  1    52   305    51.7  0.339      0.126 0.00653   2.71 0.0000519
##  2    52   304    51.6  0.355      0.132 0.00645   2.71 0.0000564
##  3    51   304    51.6 -0.645     -0.239 0.00645   2.71 0.000185 
##  4    52   306    51.7  0.323      0.120 0.00661   2.71 0.0000476
##  5    48   304    51.6 -3.64      -1.35  0.00645   2.70 0.00593  
##  6    50   305    51.7 -1.66      -0.616 0.00653   2.71 0.00125  
##  7    53   305    51.7  1.34       0.497 0.00653   2.71 0.000810 
##  8    53   306    51.7  1.32       0.491 0.00661   2.71 0.000800 
##  9    53   305    51.7  1.34       0.497 0.00653   2.71 0.000810 
## 10    50   306    51.7 -1.68      -0.622 0.00661   2.71 0.00129  
## # … with 485 more rows

# Plot
augment(lm_time, se_fit = TRUE) %>% 
  mutate(lb=.fitted - 1.96*.se.fit, 
         ub=.fitted + 1.96*.se.fit) %>%
ggplot(data=.) +
  geom_ribbon(aes(y=.fitted, ymin=lb, ymax=ub, x=days), alpha=.2) +
  geom_line(aes(y=.fitted, x=days), color="blue") +
  geom_point(aes(y = Vote, x=days), alpha=.2) 

Risk Perceptions and Partisanship

The plan is to estimate a model of partisanship on three different outcomes. Let’s see how we can do this with the tools we learned so far.

load("CV_data.Rdata")
library(tidyverse)
library(tidyr)

d_pivot <- d %>% 
            pivot_longer(cols=c(covid_job, covid_health, covid_government), 
                         names_to="covid", 
                         values_to="covid_values") 

data_nested <- d_pivot %>%
                group_by(covid) %>%
                nest() %>%
                mutate(model=map(data, ~ lm(as.numeric(covid_values) ~ 
                                              runoff_haddad + 
                                              runoff_bolsonaro + 
                                 income + gender + work + 
                                 as.numeric(education) + age , data=.x)),
                      res=map(model,tidy)) %>%
                unnest(res) %>%
                mutate(lb=estimate - 1.96*std.error, 
                       up= estimate + 1.96*std.error)

to_plot <- data_nested %>% 
              filter(str_detect(term, "runoff")) %>%
              mutate(labels_iv=fct_recode(term, "Haddad Voters"="runoff_haddadOn", 
                                                "Bolsonaro Voters"="runoff_bolsonaroOn")) %>%
              mutate(outcome= ifelse(covid=="covid_job", 
                         "How likely is it that you \n could lose your job? ",
                         ifelse(covid=="covid_health", 
                                "How likely will your health \n be affected by COVID-19?", 
                                "Has the government response \n been appropriate ?"))) 

Graph

#pick my colors
pal <- RColorBrewer::brewer.pal(9, "Spectral")

#graph
ggplot(to_plot, aes(y=estimate, x=labels_iv, 
                      ymin=up, ymax=lb, color=labels_iv)) +
  geom_pointrange(shape=21, fill="white", size=2) +
  labs(x="", y="Point Estimates", 
       title = "\nPartisanship, Risk Perceptions and Government Responses to Covid in Brazil", 
       subtitle = "Regression Estimates with Controls by Income, Gender, Age, Education, and Occupation.", 
       caption ="Note: Baseline are Independent Voters") +
  geom_hline(yintercept = 0, linetype="dashed", color="darkred") + 
  scale_color_manual(values=c("Bolsonaro Voters"=pal[9], "Haddad Voters"=pal[1]),
                     name="Who would you vote for?") +
  facet_wrap(~outcome)  + 
  theme_bw() +
  theme(strip.text = element_text(size=7),
        axis.text.x = element_blank())