Getting Started

A brief outline of getting started is shown below. See the Lab 01 Instructions for more details about the steps.

  • Go to the sta210-sp20 organization on GitHub (https://github.com/sta210-sp20). Click on the repo with the prefix hw-05. It contains the starter documents you need to complete the lab.
  • Clone the repo in RStudio Cloud

Tips

Here are some tips as you complete HW 04:

  • Periodically knit your document and commit changes (the more often the better, for example, once per each new task)
  • Push all your changes back to your GitHub repo. The Git pane in RStudio should be empty after you push. You can also check your assignment repo on github.com to make sure it has updated.
  • Once you have completed your work, you will submit your assignment in Gradescope. You are welcome to resubmit multiple times until the assignment deadline. We will grade the most recent version of the .pdf file in Gradescope.

Packages

Fill in the packages you need to complete the assignment in your R Markdown document.

library(tidyverse)
library(knitr)
library(broom)
library(nnet)

# Include other libraries as needed

The Data

For this assignment, we will analyze data from the eye witness identification experiment in Carlson and Carlson (2014). In this experiment, participants were asked to watch a video of a mock crime (from the first person perspective), spend a few minutes completing a random task, and then identify the perpetrator of the mock crime from a line up shown on the screen. Every lineup in this analysis included the true perpetrator from the video. After viewing the line-up , each participant could make one of the following decisions (id):

  • correct: correctly identified the true perpetrator
  • foil: incorrectly identified the “foil”, i.e. a person who looks very similar to the perpetrator
  • reject: incorrectly concluded the true perpetrator is not in the lineup

The main objective of the analysis is to understand how different conditions of the mock crime and suspect lineup affect the decision made by the participant. We will consider the following conditions to describe the decisions:

  • lineup: How potential suspects are shown to the participants
    • Simultaneous Lineup: Participants were shown photos of all 6 potential suspects at the same time and were required to make a single decision (identify someone from the lineup or reject the lineup).
    • Sequential 5 Lineup: Photos of the 6 suspects were shown one at a time. The participant was required to make a decision (choose or don’t choose) as each photo was shown. Once a decision was made, participants were not allowed to reexamine a photo. If the participant made an identification, the remaining photos were not shown. In each of these lineups the true perpetrator was always the 5th photo in the lineup.
  • weapon: Whether or not a weapon was present in the video of the mock crime.
  • feature: Whether or not the perpetrator had a distinctive marking on his face. In this experiment, the distinctive feature was a large “N” sticker on one cheek. (The letter “N” was chosen to represent the first author’s alma mater - University of Nebraska.)


The data may be found in eyewitness.csv in the data folder.

Questions

  1. Let’s begin by doing the exploratory data analysis. The univariate plots for each of the predictor variables and the response variable are shown below.

Complete the exploratory data analysis by creating the plots and/or summary statistics to examine the relationship between the response variable (id) and each of the explanatory variables (lineup, weapon, and feature).

  1. Using the plots/tables from Question 1:

    • What is one thing you learn about the data from the univariate plots?
    • Based on the bivariate plots, do any of the predictors appear to have a significant effect on the id? Briefly explain.

  2. Briefly explain why you should use a multinomial logistic regression model to predict id using lineup, weapon and feature.

  3. Fit the multinomial logistic model that only includes main effects. Display the model output.

    • What is the baseline category for the response variable?
    • Interpret the intercepts for each part of the model in terms of the odds.
    • Interpret the coefficients of lineup for each part of the model in terms of the odds.

  4. You want to consider all possible interaction effects for the model. Use the appropriate test to determine if there is at least one significant interaction effect.

    • Based on your test, is there evidence of any significant interaction effects?

Regardless of your answer to Question 5, use the model that includes the interaction terms for the remainder of the assignment.

  1. According to the model,

    • If there was no weapon but the perpetrator had a distinctive feature in the mock crime, how do the log-odds of reject vs. a correct ID change when there is a simultaneous lineup vs. a sequential lineup?
    • If there was no weapon but the perpetrator had a distinctive feature in the mock crime, how do the odds of reject vs. a correct ID change when there is a simultaneous lineup vs. a sequential lineup?
    • Which group of participants (i.e. which set of experimental conditions) is described by the intercept?

  2. The plots of the residuals versus the predicted probabilities, and the average residuals across categories of each predictor variable are shown below.

    Use these plots and tables along with other relevant information to assess the assumptions for the model. Are the assumptions satisifed? Include an assessment about each assumption and a brief explanation about your conclusion.

## # A tibble: 2 x 4
##   lineup         correct.avg foil.avg    reject.avg
##   <chr>                <dbl>    <dbl>         <dbl>
## 1 Sequential 5 -0.0000000518 -7.27e-9  0.0000000591
## 2 Simultaneous  0.000000118  -5.38e-8 -0.0000000641
## # A tibble: 2 x 4
##   weapon   correct.avg      foil.avg    reject.avg
##   <chr>          <dbl>         <dbl>         <dbl>
## 1 no      0.000000111  -0.0000000722 -0.0000000384
## 2 yes    -0.0000000451  0.0000000114  0.0000000338
## # A tibble: 2 x 4
##   feature   correct.avg      foil.avg    reject.avg
##   <chr>           <dbl>         <dbl>         <dbl>
## 1 no      0.0000000555  -0.0000000410 -0.0000000146
## 2 yes     0.00000000916 -0.0000000195  0.0000000104

  1. Use the model to predict the decision made by each participant. Make a table of the predicted vs.the actual decisions.

    • Briefly describe how the predicted decision is determined for each participant.
    • What is the misclassification rate?

Submitting Your Assignment

Once your work is finalized in your GitHub repo, you will submit it to Gradescope. Your assignment must be submitted on Gradescope by the deadline to be considered on time.

See Submitting the Assignment for more details on how to submit the assignment on Gradescope.

Grading

Total 50
Questions 1 - 8 45
Document neatly organized with clear headers 3
At least 3 informative commit messages 2