A Unified Conditional Frequentist and Bayesian Test for Fixed and Sequential Simple Hypothesis Testing

James Berger, Larry Brown, and Robert Wolpert

Pre-experimental Frequentist error probabilities are arguably inadequate, as summaries of evidence from data, in many hypothesis-testing settings. The Conditional Frequentist responds to this by identifying certain subsets of the outcome space and reporting a conditional error probability, given the subset of the outcome space in which the observed data lie. Statistical methods consistent with the Likelihood Principle, including Bayesian methods, avoid the problem by a more extreme form of conditioning. In this paper we prove that the Conditional Frequentist's method can be made exactly equivalent to the Bayesian's in simple versus simple hypothesis testing: specifically, we find a conditioning strategy for which the Conditional Frequentist's reported conditional error probabilities are the same as the Bayesian's posterior probabilities of error. A Conditional Frequentist who uses such a strategy can exploit other features of the Bayesian approach --- for example, the validity of sequential hypothesis tests (including versions of the sequential probability ratio test, or SPRT) even if the stopping rule is incompletely specified. PDF File