Homework 3

Due Thurs 28 Sept in class

Assignment:

For the most part, this assignment is homework problems out of the text. In addition, there's additional Splus computations that go with Exercise 18.

1. Go thru all the conceptual exercises in Chapter 1 - the answers are a bit further on in the section. This will give you an idea if you are understanding the material. We won't be grading any of these, so ask questions in class (or office hours) if you have them.
2. Chapter 1 Exercises: 16, 17, 18, 25, and 26.
3. Carry out a permutation test for the case study used in problem 18. The mini-lab below shows how you can do it for a contrived dataset based on the newspaper article shown in class.
   
   

   Permutation test in Splus

   • Here's an example of a permutation test on the soybean yield trial shown in class; the data are contrived to match the article. See Ch 14 for a real soybean trial and analysis. Enter the data into Splus:

     > ypray_c(20,15,19,14,17,19)
     > ycont_c(19,14,17,13,15,22)
     

   • The test statistic is the difference between the mean of the prayer treatment plots and the control plots:

     > diff0_mean(ypray) - mean(ycont)
     > diff0
     [1] 0.6666667
     

   • If there is no difference between the prayer treatment and the controls, then the assignment of plots to treatment groups is arbitrary. So we could shuffle the treatment labels that go with each yield and still expect to see a difference between means that is about as large as the one we observed.
     
     Put the two vectors into one long one called ally:

     > ally_c(ypray,ycont)
     > ally
      [1] 20 15 19 14 17 19 19 14 17 13 15 22
     

   • Now shuffle ally and hold the shuffled vector in yshuf

     > yshuf_sample(ally,size=12,replace=F)
     > yshuf
      [1] 19 19 15 20 14 15 17 14 19 13 17 22
     

   • If we label 1-6 as treatment and 7-12 as control, the difference between means of these arbitrary groups is

     > mean(yshuf[1:6]) - mean(yshuf[7:12])
      [1] 0
     

     so the observed difference was larger; let's repeat this random permuting of labels and calculating the mean difference 1000 times.
   • Make a vector to hold the test statistics for 1000 different random relabelings

     > diff_rep(NA,1000)
     

   • Now repeat the following for 1000 iterations:

     > for(i in 1:1000){
         yshuf_sample(ally,size=12,replace=F)
         diff[i]_mean(yshuf[1:6]) - mean(yshuf[7:12])
       }
     

     ...a quick hint here. Try it out with 10 or 1 iterations. Once you get that to work, then go for 1000.
   • We can make a histogram of these permutation statistics

     > hist(diff)
     

     and mark the value of the actual statistic

     > points(diff0,0,pch=16,cex=3)
     

   • The p-value is the chance of seeing a test statistic at least as large as what was observed (.667). This can be estimated by the proportion of permutation statistics that were as large or larger than the observed one

     > larger_ifelse(diff >= diff0,1,0)
     

     Now larger is a vector of 1's and 0's; 1 if the permutation test statistic was larger; 0 if not. The proportion of permutation statistics that are larger is the mean of this vector of 0's and 1's.

     > mean(larger)
     [1] 0.373
     

     *Note this is a "one sided" p-value since we expected the treatment (prayer) group to have a larger mean a priori. If we didn't have a preconception, a two-sided p-value would have been more appropriate. In this case, we look for the chance of seeing anything more extreme that what was observed.

     > extremer_ifelse(abs(diff) >= abs(diff0),1,0)
     > mean(extremer)
     

     *A second note: The test statistic here was the difference between means. Instead we could just as easily have taken the difference between medians, average ranks, variances, or whatever.

     > diff0_median(ypray) - median(ycont)
     > for(i in 1:1000){
         yshuf_sample(ally,size=12,replace=F)
         diff[i]_median(yshuf[1:6]) - median(yshuf[7:12])
       }
     

   • note that your value may not match mine since you'll generate a different set of 1000 random permutations. However, it should be close.
   • Now that you've mastered this, go ahead and carry out a permutation test for the first case study in Chapter 1. The data are available as CASE0101.ASC in the Sleuth datasets link of the course homepage. The first column is the score; the second is the group code: 1 for the Extrinsic group; 2 for the Intrinsic group. Be sure to give:
     1. The null and alternative hypotheses.
     2. The test statistic you used.
     3. The resulting p-value.
     4. Your conclusion (no more than a sentance or two).