Simple Linear Regression

# Simple Linear Regression
### Dr. Maria Tackett
### 01.16.19

---

## Announcements

- DVS Intro to R Workshop - Jan 22 at 2p
     - [https://duke.libcal.com/event/4799048](https://duke.libcal.com/event/4799048)

- No lecture Monday - Martin Luther King, Jr. Holiday

---

## Agenda

- Simple Linear Regression 
    - Estimating & interpreting coefficients
    - Residuals and model assumptions
    - `$R^2$`

- Git & GitHub (time permitting)

---

## Packages and Data

```r
library(tidyverse)
library(broom)
library(modelr)
library(fivethirtyeight)
```

```r
movie_scores <- fandango %>%
 rename(tomatometer = rottentomatoes, audience = rottentomatoes_user)
```

---

## rottentomatoes.com

Can ratings from movie critics be used to predict the ratings from movie goers?

---

## Critic vs. Audience Ratings

- To answer this question, we will analyze the critic and audience scores from rottentomatoes.com.  
    - The data was first used in the article [Be Suspicious of Online Movie Ratings, Especially Fandango's](https://fivethirtyeight.com/features/fandango-movies-ratings/).

- Variables: 
    - `tomatometer`: Rotten Tomatoes Tomatometer score for the film (0 - 100)
    - `audience`: Rotten Tomatoes user score for the film (0 - 100)

---

## `glimpse` of the data

```r
glimpse(movie_scores)
```

```
## Observations: 146
## Variables: 23
## $ film <chr> "Avengers: Age of Ultron", "Cinderell…
## $ year <dbl> 2015, 2015, 2015, 2015, 2015, 2015, 2…
## $ tomatometer <int> 74, 85, 80, 18, 14, 63, 42, 86, 99, 8…
## $ audience <int> 86, 80, 90, 84, 28, 62, 53, 64, 82, 8…
## $ metacritic <int> 66, 67, 64, 22, 29, 50, 53, 81, 81, 8…
## $ metacritic_user <dbl> 7.1, 7.5, 8.1, 4.7, 3.4, 6.8, 7.6, 6.…
## $ imdb <dbl> 7.8, 7.1, 7.8, 5.4, 5.1, 7.2, 6.9, 6.…
## $ fandango_stars <dbl> 5.0, 5.0, 5.0, 5.0, 3.5, 4.5, 4.0, 4.…
## $ fandango_ratingvalue <dbl> 4.5, 4.5, 4.5, 4.5, 3.0, 4.0, 3.5, 3.…
## $ rt_norm <dbl> 3.70, 4.25, 4.00, 0.90, 0.70, 3.15, 2…
## $ rt_user_norm <dbl> 4.30, 4.00, 4.50, 4.20, 1.40, 3.10, 2…
## $ metacritic_norm <dbl> 3.30, 3.35, 3.20, 1.10, 1.45, 2.50, 2…
## $ metacritic_user_nom <dbl> 3.55, 3.75, 4.05, 2.35, 1.70, 3.40, 3…
## $ imdb_norm <dbl> 3.90, 3.55, 3.90, 2.70, 2.55, 3.60, 3…
## $ rt_norm_round <dbl> 3.5, 4.5, 4.0, 1.0, 0.5, 3.0, 2.0, 4.…
## $ rt_user_norm_round <dbl> 4.5, 4.0, 4.5, 4.0, 1.5, 3.0, 2.5, 3.…
## $ metacritic_norm_round <dbl> 3.5, 3.5, 3.0, 1.0, 1.5, 2.5, 2.5, 4.…
## $ metacritic_user_norm_round <dbl> 3.5, 4.0, 4.0, 2.5, 1.5, 3.5, 4.0, 3.…
## $ imdb_norm_round <dbl> 4.0, 3.5, 4.0, 2.5, 2.5, 3.5, 3.5, 3.…
## $ metacritic_user_vote_count <int> 1330, 249, 627, 31, 88, 34, 17, 124, …
## $ imdb_user_vote_count <int> 271107, 65709, 103660, 3136, 19560, 3…
## $ fandango_votes <int> 14846, 12640, 12055, 1793, 1021, 397,…
## $ fandango_difference <dbl> 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.…
```

---

```r
ggplot(data=movie_scores,mapping=aes(x=tomatometer,y=audience)) + 
  geom_point() + 
  labs(title="Audience Score vs. Tomatometer") + 
  theme_gray()
```

---
.small[

```r
ggplot(data=movie_scores,mapping=aes(x=tomatometer,y=audience)) + 
  geom_point() + 
  geom_smooth(method="lm",se=FALSE) +
  labs(title="Audience Score vs. Tomatometer") + 
  theme_gray()
```

<img src="02-slr_files/figure-html/unnamed-chunk-5-1.png" style="display: block; margin: auto;" />
]

---

## Terminology

- `audience` is the response variable `$(Y)$`
 - variable whose variation we want to understand / variable we wish to predict

- `tomatometer` is the predictor variable `$(X)$`
 - variable used to account for variation in the outcome
 
---

## Least-Squares Regression

- There is some true relationship between `$X$` and `$Y$` that exists in the population

`$$Y = f(X) + \epsilon$$`

- If `$f$` is approximated by a linear function, then we can write the relationship as

`$$Y = \beta_0 + \beta_1 X + \epsilon$$`

- We'll estimate the slope and intercept of this linear function using *least-squares regrssion*

- We'll use statistical inference to determine if the relationship we observe in the data is significant or if it's due to random chance (we'll discuss this next class)

---

## Regression Model

`$$Y|X \sim N(\beta_0 + \beta_1 X,\sigma^2)$$`

- `$\sigma$`: the standard deviation of `$Y$` as a function of `$X$`

- **Assumption:** `$\sigma$` is equal for all values of `$X$`

---

## Regression Model

- For a single observation `$(x_i, y_i)$`

`$$\color{blue}{y_i = \beta_0 + \beta_1 x_i + \epsilon_i \hspace{10mm} \epsilon_i \sim N(0,\sigma^2)}$$`

- We want to use the `$n$` observations `$(x_1,y_1), \ldots, (x_n, y_n)$` to estimate `$\beta_0$` and `$\beta_1$`. We will use *least-squares regression* estimates.

---

## Residuals

The .vocab[residual] is the difference between the observed and predicted response.

---

## Residual Sum of Squares

- The residual for the `$i^{th}$` observation is

`$$e_i = y_i - \hat{y}_i =  y_i - (\hat{\beta}_0 + \hat{\beta}_1 x_i)$$`

- The *residual sum of squares* is

`$$RSS = e^2_1 + e^2_2 + \dots + e^2_n$$`

- The .vocab[least-squares regression] approach chooses coefficients `$\hat{\beta}_0$` and `$\hat{\beta}_1$` to minimize RSS.

---

## Estimating Coefficients

- .vocab3[Slope:]

`$$\hat\beta_1 = \frac{\sum\limits_{i=1}^n(x_i-\bar{x})(y_i - \bar{y})}{\sum\limits_{i=1}^n(x_i-\bar{x})^2} = r\frac{s_y}{s_x}$$`
such that `$r$` is the correlation between `$x$` And `$y$`.

- .vocab3[Intercept:] `$\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}$`

---

## Least-Squares Model

```r
model <- lm(audience ~ tomatometer, data=movie_scores)
tidy(model)
```

```
## # A tibble: 2 x 5
## term estimate std.error statistic p.value
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) 32.3 2.34 13.8 4.03e-28
## 2 tomatometer 0.519 0.0345 15.0 2.70e-31
```

`$$\hat{\text{audience}} =  32.316 + 0.519 \times \text{tomatometer}$$`

---

## Interpreting Slope & Intercept

- Slope: Increase in the mean response for every one unit increase in the explanatory variable

- Intercept: Mean response when the explanatory variable equals 0

- The regression equation for the Rotten Tomatoes data is
`$$\hat{\text{audience}} =  32.316 + 0.519 \times \text{tomatometer}$$`

---

## Nonsensical Intercept

- Sometimes it doesn't make sense to interpret the intercept
  + Ex. The explanatory variable is not close to 0
  + The intercept is negative even though the response variable should always be positive
  
- The intercept helps the line fit the data as closely as possible

- It is fine to have a nonsensical intercept if it helps the model give better overall predictions

---

### Should You Interpret the Intercept?

- Example 1: 
 - **Explanatory**: number of home runs in a baseball game
 - **Response**: attendance at the next baseball game

- Example 2:
    - **Explanatory**: height of a person
    - **Response**: weight of a person 
    
---

## Model Assumptions

---

## Assumptions for Regression

1. Linearity: The plot of the mean value for `$y$` against `$x$` falls on a straight line

2. Constant Variance: The regression variance is the same for all values of `$x$`

3. Normality: For a given `$x$`, the distribution of `$y$` around its mean is Normal

4. Independence: All observations are independent

---

## Checking Assumptions

We can use plots of the residuals to check the assumptions for regression.

1. Scatterplot of `$Y$` vs. `$X$` (linearity). 
    - Check this before fitting the regression model.

2. Plot of residuals vs. predictor variable (constant variance, linearity)

3. Histogram and Normal QQ-Plot of residuals (Normality)

---

## Residuals vs. Predictor

- When all the assumptions are true, the values of the residuals reflect random (chance) error

- We can look at a plot of the residuals vs. the predictor variable

- There should be no distinguishable pattern in the residuals plot, i.e. the residuals should be randomly scattered

- A non-random pattern suggests assumptions might be violated

---

## Plots of Residuals

---

```r
movie_scores <- movie_scores %>% mutate(residuals=resid(model))
```

```r
ggplot(data=movie_scores,mapping=aes(x=tomatometer, y=residuals)) + 
  geom_point() + 
  geom_hline(yintercept=0,color="red")+
  labs(title="Residuals vs. Tomatometer") +
  theme_gray()
```

---

## Checking Normality

- Examine the distribution of the residuals to determine if the Normality assumption is satisfied

- Plot the residuals in a histogram and a Normal QQ plot to visualize their distribution and assess Normality

- Most inference methods for regression are robust to some departures from Normality

---

## Normal QQ-Plot

---

```r
ggplot(data=movie_scores,mapping=aes(x=residuals)) + 
  geom_histogram() + 
  labs(title="Distribution of Residuals") +
  theme_gray()
```

---

```r
ggplot(data=movie_scores,mapping=aes(sample=residuals)) + 
  stat_qq() + 
  stat_qq_line() +
  labs(title="Normal QQ Plot of Residuals") +
  theme_gray()
```

---

## Checking Independence

- Often, we can conclude that the independence assumption is sufficiently met based on a description of the data and how it was collected.

- Two common violations of the independence assumption: 
 - Serial Effect: If the data were collected over time, the residuals should be plotted in time order to determine if there is serial correlation

- Cluster Effect: You can plot the residuals vs. a group identifier or use different markers (colors/shapes) in the residual plot to determine if there is a cluster effect.

---

### Example: Birth rate vs. Per Capita Income

- Pew Research did a <a href="http://www.pewsocialtrends.org/2011/10/12/in-a-down-economy-fewer-births/" target="_blank">study in 2011</a> looking at how the economy's effect on birthrate in the United States. 
- We will look at data for Virginia and Washington D.C. years 2000 - 2009
- Birth rate: Births per 100,000 women ages 15-44
- Per Capita Income: average income per person
---

```r
ggplot(data=pew_data, mapping = aes(x=percapitaincome,y=birthrate)) + 
  geom_point() + 
  geom_smooth(method="lm",se=FALSE) + 
  labs(title="Birth rate vs. Per Capita Income", 
       x="Per Capita Income ($)", y="Birth Rate") + 
  theme_gray()
```

<img src="02-slr_files/figure-html/unnamed-chunk-16-1.png" style="display: block; margin: auto;" />

---
## Birthrate vs. Per Capita Income

```r
pew_model <- lm(birthrate ~ percapitaincome,data=pew_data)
tidy(pew_model)
```

```
## # A tibble: 2 x 5
## term estimate std.error statistic p.value
## <chr> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) -18.2 15.3 -1.19 0.250 
## 2 percapitaincome 0.00190 0.000372 5.12 0.0000709
```
 
`$$\hat{\text{Birth Rate}} = -18.2 + 0.0019 \times \text{ Per Capita Income}$$`
---
### Residuals vs. Explanatory Variable

```r
pew_data <- pew_data %>% 
 mutate(residuals = resid(pew_model))
```
<img src="02-slr_files/figure-html/unnamed-chunk-19-1.png" style="display: block; margin: auto;" />

---
### Residuals: Cluster Effect 
<img src="02-slr_files/figure-html/unnamed-chunk-20-1.png" style="display: block; margin: auto;" />
---
class: regular 
### Residuals: Serial Effect
<img src="02-slr_files/figure-html/unnamed-chunk-21-1.png" style="display: block; margin: auto;" />
---
class: middle, center
## Assessing Model Fit
---
## `$R^2$`
We can use the coefficient of determination, `$R^2$`, to measure how well the model fits the data 
- `$R^2$` is the proportion of variation in `$Y$` that is explained by the regression line (reported as percentage)
- It is difficult to determine what's a "good" value of `$R^2$`. It depends on the context of the data.
---
## Calculating `$R^2$`
.instructions[
`$$R^2 = \frac{\text{TSS} - \text{RSS}}{\text{TSS}} = 1 - \frac{\text{RSS}}{\text{TSS}}$$`
]
- Total Sum of Squares: Total variation in the `$Y$`'s before fitting the regression 
`$$\text{TSS}= \sum\limits_{i=1}^{n}(y_i - \bar{y})^2 = (n-1)s^2_y$$`

- Residual Sum of Squares (RSS): Total variation in the `$Y$`'s around the regression line (sum of squared residuals)
`$$\text{RSS} = \sum\limits_{i=1}^{n}[y_i - (\hat{\beta}_0 + \hat{\beta}_1x_i)]^2$$`

---

## Rotten Tomatoes Data

```r
rsquare(model,movie_scores)
```

```
## [1] 0.6106479
```

The Tomatometer score explains about 61.06% of the variation in audience scores on rottentomatoes.com.

---

## Recap

- Foundation of least-squares regression

- Estimating & interpreting coefficients

- Checking model assumptions

- `$R^2$`

---

## Git and GitHub

---

## Version control

- We will use GitHub as a platform for assignments and collaboration

- But it's much more than that...

- It's actually designed for version control

---

## Versioning

---

## Versioning

with human readable messages

---

### Why do we need version control?

---

## Git and GitHub tips

- **Git** is a version control system -- like “Track Changes” features from Microsoft Word.

- **GitHub** is the home for your Git-based projects on the internet (like DropBox but much better).

- There are a lot of Git commands and very few people know them all. 99% of the time you will use git to add, commit, push, and pull.

---

## Git and GitHub tips

- We will be doing git things and interfacing with GitHub through RStudio
    - If you Google for help you might come across methods for doing these things in the command line -- skip that and move on to the next resource unless you feel comfortable trying it out.

- There is a great resource for working with git and R: [happygitwithr.com](http://happygitwithr.com/).
    - Some of the content in there is beyond the scope of this course, but it's a good place to look for help.

- You will start using git and GitHub in Lab 01 on Friday.

---

## Before next class

- Reading 02 posted online - due Wed Jan 23
    - Review inference
    
- If you have not already done so, 
    - Accept the invite to join `STA210-Sp19` organization on GitHub
    - complete "Getting to know you" survey on Sakai 
    - complete Pretest ([test info & access code](https://www2.stat.duke.edu/courses/Spring19/sta210.001/slides/lab-slides/00-lab-slides.html#8))