P-values for Composite Null Models
M. J. Bayarri and James O. Berger
The problem of investigating compatibility of an assumed model with
the data is investigated, in the situation when the assumed model
has unknown parameters. The most frequently used measures of
compatibility are p-values, based on statistics T for
which large values are deemed to indicate
incompatibility of the data and the model.
When the null model has unknown parameters, p-values are not uniquely
defined. The proposals for computing a p-value in such a situation
include the plug-in and similar p-values on the
frequentist side, and the predictive and
posterior predictive p-values on the Bayesian side. We propose
two alternatives, the conditional predictive p-value and the
partial posterior predictive p-value, and indicate their
advantages from both Bayesian and frequentist perspectives.