Calibration of P-values for Testing Precise Null Hypotheses
T. Sellke, M.J. Bayarri, and J. Berger
P-values are the most commonly used tool to measure evidence
against a hypothesis or hypothesized model. Unfortunately, they
are often incorrectly viewed as an error probability for rejection of the
hypothesis or, even worse, as the posterior probability that the hypothesis
is true. The fact that these interpretations can be completely misleading
when testing precise hypotheses is
first reviewed, through consideration of
two revealing simulations. Then two calibrations of a P-value are
developed, the first being interpretable as odds and the second as
either a (conditional) frequentist error probability or as the posterior
probability of the hypothesis.