Statistics 101
  Data Analysis and Statistical Inference

In-class problems on hypothesis tests
 




Conceptual questions on hypothesis testing

Decide whether the following statements are true or false.  Explain your reasoning.

Problems:

a)  A p-value of .08 is more evidence against the null hypothesis than a p-value of .04.

b)  If two independent studies are done on the same population with the purpose of testing the same hypotheses, the study with the larger sample size is more likely to have a smaller p-value than the study with the smaller sample size.  (Hint:  Consider if this is true in the case where the null hypothesis is true and in the case when the null hypothesis is false.)

c)  The statement, "the p-value is .003", is equivalent to the statement, "there is a 0.3% probability that the null hypothesis is true".

d)  Even though you rejected the null hypothesis,  it may still be true.

e)  Assuming the central limit theorem applies, hypothesis tests are valid.

f)  A researcher who tried to learn statistics without taking a formal course does a hypothesis test and gets a p-value of .024.   He says, "there is a 98.6% chance that the alternative hypothesis is false, so the null hypothesis is true."    What, if anything, is wrong with his statement?

g) You perform a hypothesis test using a sample size of four units, and you do not reject the null hypothesis.   Your research colleague says this statistical test provides conclusive evidence against the alternative hypothesis.   Do you agree or disagree with his conclusion?  Explain your reasoning in three or less sentences.

h)  You are the head of the Food and Drug Administration (F.D.A.), in charge of deciding whether new drugs are effective and should be allowed to be sold to people.   A pharmaceutical company trying to win approval for a new drug they manufacture claims that their drug is better than the standard drug at curing a certain disease.   The company bases this claim on a study in which they gave their drug to 1000 volunteers with the disease.   They compared these volunteers to a group of 1000 hospital patients who were treated with the standard drug and whose information is obtained from existing hospital records.  The company found a "statistically significant" difference between the percentage of volunteers who were cured and the percentage of the comparison group who were cured.   That is, they did a statistical hypothesis test and rejected the null hypothesis that the percentages are equal.  As director of the F.D.A., should you permit the new drug to be sold?  Explain your reasoning in three or less sentences.

i)  If you get a p-value of 0.13, it means there is a 13% chance that the sample average equals the population average.

j)  If you get a p-value of 0.13, it means there is a 13% chance that the sample average does not equal the population average.

k)  If you get a p-value of 0.13, it means there is an 87% chance that the sample average equals the population average.

l)  If you get a p-value of 0.13, it means there is an 87% chance that the sample average does not equal the population average.

m)  If you get a p-value of 0.13, it means that the null hypothesis is true in 13% of all samples.

n)  If you get a p-value of 0.13, it means that when the null hypothesis is true, a value of the test statistic as or more extreme than what was observed occurs in about 13% of all samples.