Today’s agenda

Data wrangling with dplyr

library(dplyr)

A little bit of text parsing with stringr
Working with NC bike crash data released by https://opendurham.nc.gov

Getting started with data in R

NC DOT Fatal Crashes in North Carolina

From https://opendurham.nc.gov

bike <- read.csv("https://stat.duke.edu/~mc301/data/nc_bike_crash.csv", 
                 sep = ";", stringsAsFactors = FALSE) %>%
  tbl_df()

View the names of variables via

names(bike)

##  [1] "FID"        "OBJECTID"   "AmbulanceR" "BikeAge_Gr" "Bike_Age"  
##  [6] "Bike_Alc_D" "Bike_Dir"   "Bike_Injur" "Bike_Pos"   "Bike_Race" 
## [11] "Bike_Sex"   "City"       "County"     "CrashAlcoh" "CrashDay"  
## [16] "Crash_Date" "Crash_Grp"  "Crash_Hour" "Crash_Loc"  "Crash_Mont"
## [21] "Crash_Time" "Crash_Type" "Crash_Ty_1" "Crash_Year" "Crsh_Sevri"
## [26] "Developmen" "DrvrAge_Gr" "Drvr_Age"   "Drvr_Alc_D" "Drvr_EstSp"
## [31] "Drvr_Injur" "Drvr_Race"  "Drvr_Sex"   "Drvr_VehTy" "ExcsSpdInd"
## [36] "Hit_Run"    "Light_Cond" "Locality"   "Num_Lanes"  "Num_Units" 
## [41] "Rd_Charact" "Rd_Class"   "Rd_Conditi" "Rd_Config"  "Rd_Defects"
## [46] "Rd_Feature" "Rd_Surface" "Region"     "Rural_Urba" "Speed_Limi"
## [51] "Traff_Cntr" "Weather"    "Workzone_I" "Location"

and see detailed descriptions at https://stat.duke.edu/~mc301/data/nc_bike_crash.html.

Aside: Strings (characters) vs factors

By default R will convert character vectors into factors when they are included in a data frame
Sometimes this is useful, sometimes it isn’t – either way it is important to know what type/class you are working with
This behavior can be changed using the stringsAsFactors = FALSE when loading a data drame

stringsAsFactorsHELLNO

Viewing your data

In the Environment, click on the name of the data frame to view it in the data viewer
Use the str() function to compactly display the internal structure of an R object

str(bike)

## Classes 'tbl_df', 'tbl' and 'data.frame':    5716 obs. of  54 variables:
##  $ FID       : int  18 29 33 35 49 53 56 60 63 66 ...
##  $ OBJECTID  : int  19 30 34 36 50 54 57 61 64 67 ...
##  $ AmbulanceR: chr  "No" "Yes" "No" "Yes" ...
##  $ BikeAge_Gr: chr  "" "50-59" "" "16-19" ...
##  $ Bike_Age  : int  6 51 10 17 6 52 18 40 6 7 ...
##  $ Bike_Alc_D: chr  "No" "No" "No" "No" ...
##  $ Bike_Dir  : chr  "Not Applicable" "With Traffic" "With Traffic" "" ...
##  $ Bike_Injur: chr  "C: Possible Injury" "C: Possible Injury" "Injury" "B: Evident Injury" ...
##  $ Bike_Pos  : chr  "Driveway / Alley" "Travel Lane" "Travel Lane" "Travel Lane" ...
##  $ Bike_Race : chr  "Black" "Black" "Black" "White" ...
##  $ Bike_Sex  : chr  "Female" "Male" "Male" "Male" ...
##  $ City      : chr  "Durham" "Greenville" "Farmville" "Charlotte" ...
##  $ County    : chr  "Durham" "Pitt" "Pitt" "Mecklenburg" ...
##  $ CrashAlcoh: chr  "No" "No" "No" "No" ...
##  $ CrashDay  : chr  "01-01-06" "01-01-02" "01-01-07" "01-01-05" ...
##  $ Crash_Date: chr  "2007-01-06" "2007-01-09" "2007-01-14" "2007-01-12" ...
##  $ Crash_Grp : chr  "Bicyclist Failed to Yield - Midblock" "Crossing Paths - Other Circumstances" "Bicyclist Failed to Yield - Sign-Controlled Intersection" "Loss of Control / Turning Error" ...
##  $ Crash_Hour: int  13 23 16 19 12 20 19 14 16 0 ...
##  $ Crash_Loc : chr  "Non-Intersection" "Intersection-Related" "Intersection" "Intersection" ...
##  $ Crash_Mont: chr  "" "" "" "" ...
##  $ Crash_Time: chr  "0001-01-01T08:21:58-04:56" "0001-01-01T18:12:58-04:56" "0001-01-01T11:48:58-04:56" "0001-01-01T14:59:58-04:56" ...
##  $ Crash_Type: chr  "Bicyclist Ride Out - Residential Driveway" "Crossing Paths - Intersection - Other /" "Bicyclist Ride Through - Sign-Controlled Intersection" "Motorist Lost Control - Other /" ...
##  $ Crash_Ty_1: int  353311 211180 111144 119139 112114 311231 119144 132180 112142 460910 ...
##  $ Crash_Year: int  2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...
##  $ Crsh_Sevri: chr  "C: Possible Injury" "C: Possible Injury" "O: No Injury" "B: Evident Injury" ...
##  $ Developmen: chr  "Residential" "Commercial" "Residential" "Residential" ...
##  $ DrvrAge_Gr: chr  "60-69" "30-39" "50-59" "30-39" ...
##  $ Drvr_Age  : int  66 34 52 33 NA 20 40 NA 17 51 ...
##  $ Drvr_Alc_D: chr  "No" "No" "No" "No" ...
##  $ Drvr_EstSp: chr  "11-15 mph" "0-5 mph" "21-25 mph" "46-50 mph" ...
##  $ Drvr_Injur: chr  "O: No Injury" "O: No Injury" "O: No Injury" "O: No Injury" ...
##  $ Drvr_Race : chr  "Black" "Black" "White" "White" ...
##  $ Drvr_Sex  : chr  "Male" "Male" "Female" "Female" ...
##  $ Drvr_VehTy: chr  "Pickup" "Passenger Car" "Passenger Car" "Sport Utility" ...
##  $ ExcsSpdInd: chr  "No" "No" "No" "No" ...
##  $ Hit_Run   : chr  "No" "No" "No" "No" ...
##  $ Light_Cond: chr  "Daylight" "Dark - Lighted Roadway" "Daylight" "Dark - Roadway Not Lighted" ...
##  $ Locality  : chr  "Mixed (30% To 70% Developed)" "Urban (>70% Developed)" "Mixed (30% To 70% Developed)" "Urban (>70% Developed)" ...
##  $ Num_Lanes : chr  "2 lanes" "5 lanes" "2 lanes" "4 lanes" ...
##  $ Num_Units : int  2 2 2 3 2 2 2 2 2 2 ...
##  $ Rd_Charact: chr  "Straight - Level" "Straight - Level" "Straight - Level" "Straight - Level" ...
##  $ Rd_Class  : chr  "Local Street" "Local Street" "Local Street" "NC Route" ...
##  $ Rd_Conditi: chr  "Dry" "Dry" "Dry" "Dry" ...
##  $ Rd_Config : chr  "Two-Way, Not Divided" "Two-Way, Divided, Unprotected Median" "Two-Way, Not Divided" "Two-Way, Divided, Unprotected Median" ...
##  $ Rd_Defects: chr  "None" "None" "None" "None" ...
##  $ Rd_Feature: chr  "No Special Feature" "Four-Way Intersection" "Four-Way Intersection" "Four-Way Intersection" ...
##  $ Rd_Surface: chr  "Smooth Asphalt" "Smooth Asphalt" "Smooth Asphalt" "Smooth Asphalt" ...
##  $ Region    : chr  "Piedmont" "Coastal" "Coastal" "Piedmont" ...
##  $ Rural_Urba: chr  "Urban" "Urban" "Rural" "Urban" ...
##  $ Speed_Limi: chr  "20 - 25  MPH" "40 - 45  MPH" "30 - 35  MPH" "40 - 45  MPH" ...
##  $ Traff_Cntr: chr  "No Control Present" "Stop And Go Signal" "Stop Sign" "Stop And Go Signal" ...
##  $ Weather   : chr  "Clear" "Clear" "Clear" "Cloudy" ...
##  $ Workzone_I: chr  "No" "No" "No" "No" ...
##  $ Location  : chr  "36.002743, -78.8785" "35.612984, -77.39265" "35.595676, -77.59074" "35.076767, -80.7728" ...

Data wrangling

Data wrangling with `dplyr`

The dplyr package is based on the concepts of functions as verbs that manipulate data frames:

filter(): pick rows matching criteria
select(): pick columns by name
rename(): rename specific columns
arrange(): reorder rows
mutate(): add new variables
transmute(): create new data frame with variables
sample_n() / sample_frac(): randomly sample rows
summarise(): reduce variables to values

`dplyr` rules

First argument is a data frame
Subsequent arguments say what to do with data frame
Always return a data frame
Avoid modify in place

Filter rows with `filter()`

Select a subset of rows in a data frame.
Easily filter for many conditions at once.

`filter()`

for crashes in Durham County

bike %>%
  filter(County == "Durham")

## Source: local data frame [253 x 54]
## 
##    FID OBJECTID AmbulanceR BikeAge_Gr Bike_Age Bike_Alc_D       Bike_Dir
## 1   18       19         No                   6         No Not Applicable
## 2   53       54        Yes      50-59       52         No   With Traffic
## 3   56       57        Yes      16-19       18         No               
## 4  209      210         No      16-19       16         No Facing Traffic
## 5  228      229        Yes      40-49       40         No   With Traffic
## 6  620      621        Yes      50-59       55         No   With Traffic
## 7  667      668        Yes      60-69       61         No Not Applicable
## 8  458      459        Yes      60-69       62         No   With Traffic
## 9  576      577         No      40-49       49         No   With Traffic
## 10 618      619         No      20-24       23         No   With Traffic
## .. ...      ...        ...        ...      ...        ...            ...
## Variables not shown: Bike_Injur (chr), Bike_Pos (chr), Bike_Race (chr),
##   Bike_Sex (chr), City (chr), County (chr), CrashAlcoh (chr), CrashDay
##   (chr), Crash_Date (chr), Crash_Grp (chr), Crash_Hour (int), Crash_Loc
##   (chr), Crash_Mont (chr), Crash_Time (chr), Crash_Type (chr), Crash_Ty_1
##   (int), Crash_Year (int), Crsh_Sevri (chr), Developmen (chr), DrvrAge_Gr
##   (chr), Drvr_Age (int), Drvr_Alc_D (chr), Drvr_EstSp (chr), Drvr_Injur
##   (chr), Drvr_Race (chr), Drvr_Sex (chr), Drvr_VehTy (chr), ExcsSpdInd
##   (chr), Hit_Run (chr), Light_Cond (chr), Locality (chr), Num_Lanes (chr),
##   Num_Units (int), Rd_Charact (chr), Rd_Class (chr), Rd_Conditi (chr),
##   Rd_Config (chr), Rd_Defects (chr), Rd_Feature (chr), Rd_Surface (chr),
##   Region (chr), Rural_Urba (chr), Speed_Limi (chr), Traff_Cntr (chr),
##   Weather (chr), Workzone_I (chr), Location (chr)

`filter()`

for crashes in Durham County where biker was < 10 yrs old

bike %>%
  filter(County == "Durham", Bike_Age < 10)

## Source: local data frame [20 x 54]
## 
##    FID OBJECTID AmbulanceR BikeAge_Gr Bike_Age Bike_Alc_D       Bike_Dir
## 1   18       19         No                   6         No Not Applicable
## 2   47       48         No     10-Jun        9         No Not Applicable
## 3  124      125        Yes     10-Jun        8         No   With Traffic
## 4  531      532        Yes     10-Jun        7         No   With Traffic
## 5  704      705        Yes     10-Jun        9         No Not Applicable
## 6   42       43         No     10-Jun        8         No   With Traffic
## 7  392      393        Yes        0-5        2         No Not Applicable
## 8  941      942         No     10-Jun        9         No   With Traffic
## 9  436      437        Yes     10-Jun        6         No Not Applicable
## 10 160      161        Yes     10-Jun        7         No   With Traffic
## 11 273      274        Yes     10-Jun        7         No Facing Traffic
## 12  78       79        Yes     10-Jun        7         No   With Traffic
## 13 422      423         No     10-Jun        9         No Not Applicable
## 14 570      571         No                   0    Missing Not Applicable
## 15 683      684        Yes     10-Jun        8         No Not Applicable
## 16  62       63        Yes     10-Jun        7         No   With Traffic
## 17 248      249         No        0-5        4         No Not Applicable
## 18 306      307        Yes     10-Jun        8         No   With Traffic
## 19 231      232        Yes     10-Jun        8         No   With Traffic
## 20 361      362        Yes     10-Jun        9         No   With Traffic
## Variables not shown: Bike_Injur (chr), Bike_Pos (chr), Bike_Race (chr),
##   Bike_Sex (chr), City (chr), County (chr), CrashAlcoh (chr), CrashDay
##   (chr), Crash_Date (chr), Crash_Grp (chr), Crash_Hour (int), Crash_Loc
##   (chr), Crash_Mont (chr), Crash_Time (chr), Crash_Type (chr), Crash_Ty_1
##   (int), Crash_Year (int), Crsh_Sevri (chr), Developmen (chr), DrvrAge_Gr
##   (chr), Drvr_Age (int), Drvr_Alc_D (chr), Drvr_EstSp (chr), Drvr_Injur
##   (chr), Drvr_Race (chr), Drvr_Sex (chr), Drvr_VehTy (chr), ExcsSpdInd
##   (chr), Hit_Run (chr), Light_Cond (chr), Locality (chr), Num_Lanes (chr),
##   Num_Units (int), Rd_Charact (chr), Rd_Class (chr), Rd_Conditi (chr),
##   Rd_Config (chr), Rd_Defects (chr), Rd_Feature (chr), Rd_Surface (chr),
##   Region (chr), Rural_Urba (chr), Speed_Limi (chr), Traff_Cntr (chr),
##   Weather (chr), Workzone_I (chr), Location (chr)

Commonly used logical operators in R

operator	definition
`<`	less than
`<=`	less than or equal to
`>`	greater than
`>=`	greater than or equal to
`==`	exactly equal to
`!=`	not equal to
`x \| y`	`x` OR `y`
`x & y`	`x` AND `y`

Commonly used logical operators in R

operator	definition
`is.na(x)`	test if `x` is `NA`
`!is.na(x)`	test if `x` is not `NA`
`x %in% y`	test if `x` is in `y`
`!(x %in% y)`	test if `x` is not in `y`
`!x`	not `x`

Aside: real data is messy!

What in the world does a BikeAge_gr of 10-Jun or 15-Nov mean?

bike %>%
  group_by(BikeAge_Gr) %>%
  summarise(crash_count = n())

## Source: local data frame [13 x 2]
## 
##    BikeAge_Gr crash_count
## 1                     112
## 2         0-5          60
## 3      10-Jun         421
## 4      15-Nov         747
## 5       16-19         605
## 6       20-24         680
## 7       25-29         430
## 8       30-39         658
## 9       40-49         920
## 10      50-59         739
## 11      60-69         274
## 12         70          12
## 13        70+          58

Careful data scientists clean up their data first!

We’re going to need to do some text parsing to clean up these data
- 10-Jun should be 6-10
- 15-Nov should be 11-15
New R package: stringr

Install and load: `stringr`

Install:

install.packages(stringr) # only have to do this once

Load:

library(stringr)

Package reference: Most R packages come with a vignette that describe in detail what each function does and how to use them, they’re incredibly useful resources (in addition to other worked out examples on the web) https://cran.r-project.org/web/packages/stringr/vignettes/stringr.html

Replace with `str_replace()` and add new variables with `mutate()`

Remember we want to do the following in the BikeAge_Gr variable
- 10-Jun should be 6-10
- 15-Nov should be 11-15

bike <- bike %>%
  mutate(BikeAge_Gr = str_replace(BikeAge_Gr, "10-Jun", "6-10")) %>%
  mutate(BikeAge_Gr = str_replace(BikeAge_Gr, "15-Nov", "11-15"))

Note that we’re overwriting existing data and columns, so be careful!
- But remember, it’s easy to revert if you make a mistake since we didn’t touch the raw data, we can always reload it and start over

Check before you move on

Always check your changes and confirm code did what you wanted it to do

bike %>%
  group_by(BikeAge_Gr) %>%
  summarise(count = n())

## Source: local data frame [13 x 2]
## 
##    BikeAge_Gr count
## 1               112
## 2         0-5    60
## 3       11-15   747
## 4       16-19   605
## 5       20-24   680
## 6       25-29   430
## 7       30-39   658
## 8       40-49   920
## 9       50-59   739
## 10       6-10   421
## 11      60-69   274
## 12         70    12
## 13        70+    58

`slice()` for certain row numbers

First five

bike %>%
  slice(1:5)

## Source: local data frame [5 x 54]
## 
##   FID OBJECTID AmbulanceR BikeAge_Gr Bike_Age Bike_Alc_D       Bike_Dir
## 1  18       19         No                   6         No Not Applicable
## 2  29       30        Yes      50-59       51         No   With Traffic
## 3  33       34         No                  10         No   With Traffic
## 4  35       36        Yes      16-19       17         No               
## 5  49       50         No                   6         No Facing Traffic
## Variables not shown: Bike_Injur (chr), Bike_Pos (chr), Bike_Race (chr),
##   Bike_Sex (chr), City (chr), County (chr), CrashAlcoh (chr), CrashDay
##   (chr), Crash_Date (chr), Crash_Grp (chr), Crash_Hour (int), Crash_Loc
##   (chr), Crash_Mont (chr), Crash_Time (chr), Crash_Type (chr), Crash_Ty_1
##   (int), Crash_Year (int), Crsh_Sevri (chr), Developmen (chr), DrvrAge_Gr
##   (chr), Drvr_Age (int), Drvr_Alc_D (chr), Drvr_EstSp (chr), Drvr_Injur
##   (chr), Drvr_Race (chr), Drvr_Sex (chr), Drvr_VehTy (chr), ExcsSpdInd
##   (chr), Hit_Run (chr), Light_Cond (chr), Locality (chr), Num_Lanes (chr),
##   Num_Units (int), Rd_Charact (chr), Rd_Class (chr), Rd_Conditi (chr),
##   Rd_Config (chr), Rd_Defects (chr), Rd_Feature (chr), Rd_Surface (chr),
##   Region (chr), Rural_Urba (chr), Speed_Limi (chr), Traff_Cntr (chr),
##   Weather (chr), Workzone_I (chr), Location (chr)

`slice()` for certain row numbers

Last five

last_row <- nrow(bike)
bike %>%
  slice((last_row-4):last_row)

## Source: local data frame [5 x 54]
## 
##   FID OBJECTID AmbulanceR BikeAge_Gr Bike_Age Bike_Alc_D       Bike_Dir
## 1 460      461        Yes       6-10        7         No Not Applicable
## 2 474      475        Yes      50-59       50         No   With Traffic
## 3 479      480        Yes      16-19       16         No Not Applicable
## 4 487      488         No      40-49       47        Yes   With Traffic
## 5 488      489        Yes      30-39       35         No Facing Traffic
## Variables not shown: Bike_Injur (chr), Bike_Pos (chr), Bike_Race (chr),
##   Bike_Sex (chr), City (chr), County (chr), CrashAlcoh (chr), CrashDay
##   (chr), Crash_Date (chr), Crash_Grp (chr), Crash_Hour (int), Crash_Loc
##   (chr), Crash_Mont (chr), Crash_Time (chr), Crash_Type (chr), Crash_Ty_1
##   (int), Crash_Year (int), Crsh_Sevri (chr), Developmen (chr), DrvrAge_Gr
##   (chr), Drvr_Age (int), Drvr_Alc_D (chr), Drvr_EstSp (chr), Drvr_Injur
##   (chr), Drvr_Race (chr), Drvr_Sex (chr), Drvr_VehTy (chr), ExcsSpdInd
##   (chr), Hit_Run (chr), Light_Cond (chr), Locality (chr), Num_Lanes (chr),
##   Num_Units (int), Rd_Charact (chr), Rd_Class (chr), Rd_Conditi (chr),
##   Rd_Config (chr), Rd_Defects (chr), Rd_Feature (chr), Rd_Surface (chr),
##   Region (chr), Rural_Urba (chr), Speed_Limi (chr), Traff_Cntr (chr),
##   Weather (chr), Workzone_I (chr), Location (chr)

`select()` to keep only the variables you mention

bike %>%
  select(Crash_Loc, Hit_Run) %>%
  table()

##                       Hit_Run
## Crash_Loc                No  Yes
##   .                       4    0
##   Intersection         2223  275
##   Intersection-Related  252   42
##   Location                3    7
##   Non-Intersection     2213  462
##   Non-Roadway           205   30

or `select()`to exclude variables

bike %>%
  select(-OBJECTID)

## Source: local data frame [5,716 x 53]
## 
##    FID AmbulanceR BikeAge_Gr Bike_Age Bike_Alc_D       Bike_Dir
## 1   18         No                   6         No Not Applicable
## 2   29        Yes      50-59       51         No   With Traffic
## 3   33         No                  10         No   With Traffic
## 4   35        Yes      16-19       17         No               
## 5   49         No                   6         No Facing Traffic
## 6   53        Yes      50-59       52         No   With Traffic
## 7   56        Yes      16-19       18         No               
## 8   60         No      40-49       40         No Facing Traffic
## 9   63        Yes       6-10        6         No Facing Traffic
## 10  66        Yes       6-10        7         No               
## .. ...        ...        ...      ...        ...            ...
## Variables not shown: Bike_Injur (chr), Bike_Pos (chr), Bike_Race (chr),
##   Bike_Sex (chr), City (chr), County (chr), CrashAlcoh (chr), CrashDay
##   (chr), Crash_Date (chr), Crash_Grp (chr), Crash_Hour (int), Crash_Loc
##   (chr), Crash_Mont (chr), Crash_Time (chr), Crash_Type (chr), Crash_Ty_1
##   (int), Crash_Year (int), Crsh_Sevri (chr), Developmen (chr), DrvrAge_Gr
##   (chr), Drvr_Age (int), Drvr_Alc_D (chr), Drvr_EstSp (chr), Drvr_Injur
##   (chr), Drvr_Race (chr), Drvr_Sex (chr), Drvr_VehTy (chr), ExcsSpdInd
##   (chr), Hit_Run (chr), Light_Cond (chr), Locality (chr), Num_Lanes (chr),
##   Num_Units (int), Rd_Charact (chr), Rd_Class (chr), Rd_Conditi (chr),
##   Rd_Config (chr), Rd_Defects (chr), Rd_Feature (chr), Rd_Surface (chr),
##   Region (chr), Rural_Urba (chr), Speed_Limi (chr), Traff_Cntr (chr),
##   Weather (chr), Workzone_I (chr), Location (chr)

`rename()` specific columns

Useful for correcting typos, and renaming to make variable names shorter and/or more informative

Original names:

names(bike)

##  [1] "FID"        "OBJECTID"   "AmbulanceR" "BikeAge_Gr" "Bike_Age"  
##  [6] "Bike_Alc_D" "Bike_Dir"   "Bike_Injur" "Bike_Pos"   "Bike_Race" 
## [11] "Bike_Sex"   "City"       "County"     "CrashAlcoh" "CrashDay"  
## [16] "Crash_Date" "Crash_Grp"  "Crash_Hour" "Crash_Loc"  "Crash_Mont"
## [21] "Crash_Time" "Crash_Type" "Crash_Ty_1" "Crash_Year" "Crsh_Sevri"
## [26] "Developmen" "DrvrAge_Gr" "Drvr_Age"   "Drvr_Alc_D" "Drvr_EstSp"
## [31] "Drvr_Injur" "Drvr_Race"  "Drvr_Sex"   "Drvr_VehTy" "ExcsSpdInd"
## [36] "Hit_Run"    "Light_Cond" "Locality"   "Num_Lanes"  "Num_Units" 
## [41] "Rd_Charact" "Rd_Class"   "Rd_Conditi" "Rd_Config"  "Rd_Defects"
## [46] "Rd_Feature" "Rd_Surface" "Region"     "Rural_Urba" "Speed_Limi"
## [51] "Traff_Cntr" "Weather"    "Workzone_I" "Location"

Rename Speed_Limi to Speed_Limit:

bike <- bike %>%
  rename(Speed_Limit = Speed_Limi)

Check before you move on

Always check your changes and confirm code did what you wanted it to do

names(bike)

##  [1] "FID"         "OBJECTID"    "AmbulanceR"  "BikeAge_Gr"  "Bike_Age"   
##  [6] "Bike_Alc_D"  "Bike_Dir"    "Bike_Injur"  "Bike_Pos"    "Bike_Race"  
## [11] "Bike_Sex"    "City"        "County"      "CrashAlcoh"  "CrashDay"   
## [16] "Crash_Date"  "Crash_Grp"   "Crash_Hour"  "Crash_Loc"   "Crash_Mont" 
## [21] "Crash_Time"  "Crash_Type"  "Crash_Ty_1"  "Crash_Year"  "Crsh_Sevri" 
## [26] "Developmen"  "DrvrAge_Gr"  "Drvr_Age"    "Drvr_Alc_D"  "Drvr_EstSp" 
## [31] "Drvr_Injur"  "Drvr_Race"   "Drvr_Sex"    "Drvr_VehTy"  "ExcsSpdInd" 
## [36] "Hit_Run"     "Light_Cond"  "Locality"    "Num_Lanes"   "Num_Units"  
## [41] "Rd_Charact"  "Rd_Class"    "Rd_Conditi"  "Rd_Config"   "Rd_Defects" 
## [46] "Rd_Feature"  "Rd_Surface"  "Region"      "Rural_Urba"  "Speed_Limit"
## [51] "Traff_Cntr"  "Weather"     "Workzone_I"  "Location"

`summarise()` in a new data frame

bike %>%
  group_by(BikeAge_Gr) %>%
  summarise(crash_count = n()) %>%
  arrange(crash_count)

## Source: local data frame [13 x 2]
## 
##    BikeAge_Gr crash_count
## 1          70          12
## 2         70+          58
## 3         0-5          60
## 4                     112
## 5       60-69         274
## 6        6-10         421
## 7       25-29         430
## 8       16-19         605
## 9       30-39         658
## 10      20-24         680
## 11      50-59         739
## 12      11-15         747
## 13      40-49         920

and `arrange()` to order rows

bike %>%
  group_by(BikeAge_Gr) %>%
  summarise(crash_count = n()) %>%
  arrange(desc(crash_count))

## Source: local data frame [13 x 2]
## 
##    BikeAge_Gr crash_count
## 1       40-49         920
## 2       11-15         747
## 3       50-59         739
## 4       20-24         680
## 5       30-39         658
## 6       16-19         605
## 7       25-29         430
## 8        6-10         421
## 9       60-69         274
## 10                    112
## 11        0-5          60
## 12        70+          58
## 13         70          12

Select rows with `sample_n()` or `sample_frac()`

sample_n(): randomly sample 5 observations

bike_n5 <- bike %>%
  sample_n(5, replace = FALSE)
dim(bike_n5)

## [1]  5 54

sample_frac(): randomly sample 20% of observations

bike_perc20 <-bike %>%
  sample_frac(0.2, replace = FALSE)
dim(bike_perc20)

## [1] 1143   54

Sta112FS
5. Data wrangling with dplyr

Dr. Çetinkaya-Rundel

September 10, 2015

Today’s agenda

Today’s agenda

Getting started with data in R

NC DOT Fatal Crashes in North Carolina

Aside: Strings (characters) vs factors

Viewing your data

Data wrangling

Data wrangling with `dplyr`

`dplyr` rules

Filter rows with `filter()`

`filter()`

`filter()`

Commonly used logical operators in R

Commonly used logical operators in R

Aside: real data is messy!

Careful data scientists clean up their data first!

Install and load: `stringr`

Replace with `str_replace()` and add new variables with `mutate()`

Check before you move on

`slice()` for certain row numbers

`slice()` for certain row numbers

`select()` to keep only the variables you mention

or `select()`to exclude variables

`rename()` specific columns

Check before you move on

`summarise()` in a new data frame

and `arrange()` to order rows

Select rows with `sample_n()` or `sample_frac()`

Sta112FS 5. Data wrangling with dplyr

Dr. Çetinkaya-Rundel

September 10, 2015

Today’s agenda

Today’s agenda

Getting started with data in R

NC DOT Fatal Crashes in North Carolina

Aside: Strings (characters) vs factors

Viewing your data

Data wrangling

Data wrangling with dplyr

dplyr rules

Filter rows with filter()

filter()

filter()

Commonly used logical operators in R

Commonly used logical operators in R

Aside: real data is messy!

Careful data scientists clean up their data first!

Install and load: stringr

Replace with str_replace() and add new variables with mutate()

Check before you move on

slice() for certain row numbers

slice() for certain row numbers

select() to keep only the variables you mention

or select()to exclude variables

rename() specific columns

Check before you move on

summarise() in a new data frame

and arrange() to order rows

Select rows with sample_n() or sample_frac()

Sta112FS
5. Data wrangling with dplyr

Data wrangling with `dplyr`

`dplyr` rules

Filter rows with `filter()`

`filter()`

`filter()`

Install and load: `stringr`

Replace with `str_replace()` and add new variables with `mutate()`

`slice()` for certain row numbers

`slice()` for certain row numbers

`select()` to keep only the variables you mention

or `select()`to exclude variables

`rename()` specific columns

`summarise()` in a new data frame

and `arrange()` to order rows

Select rows with `sample_n()` or `sample_frac()`