Bayes' Factors

Abstract

Bayes' factors play a pivotal role in Bayesian hypothesis testing, representing the factor through which the prior odds between competing hypotheses are transformed to the posterior odds between the hypotheses. Mathematically, Bayes' factors are defined as the ratio of the marginal probability assigned to the data by one hypothesis to the marginal probability assigned to data by the other hypothesis. In many parametric statistical tests, hypotheses are defined by the specification of prior densities on unknown parameters. When the prior density on a parameter concentrates on a single value under both the null and alternative hypotheses (resulting in two ‘simple’ or ‘point’ hypotheses), the Bayes factor equals the likelihood ratio. More generally, Bayes' factors represent the ratio of an averaged likelihood function, averaged with respect to the different prior densities assigned to the unknown parameter under each hypothesis. Inconsistencies in the limiting behaviour of Bayes' factors may arise when hypotheses are defined with respect to prior densities that have overlapping support.

Key Concepts

  • Bayes' factors are used in Bayesian hypothesis tests and are the key factors through which experimental data determine the posterior probability assigned to scientific hypotheses.
  • Bayes' factors represent the ratio of the probability assigned by competing hypotheses to a common set of data.
  • When testing whether one of the two simple hypotheses is true, the Bayes factor equals the likelihood ratio between the hypotheses.
  • Bayes' factors equal the posterior odds divided by the prior odds between hypotheses.
  • The natural logarithm of a Bayes' factor is called the weight of evidence.

Keywords: Bayesian hypothesis test; integrated likelihood; intrinsic prior; likelihood ratio; local prior density; marginal likelihood; nonlocal prior density; weight of evidence

Figure 1. Prior densities for binomial success probability under composite hypotheses H1 (flat line) and H2 (curved line).
Figure 2. Bayes' factor as a function of the number of successes y in n = 10 trials when testing two composite binomial hypotheses.
Figure 3. Illustration of prior densities and a likelihood function based on 450 successes in 1000 trials. The likelihood function is the curve with a sharp mode at 0.45.
Figure 4. Weight of evidence in favour of local and nonlocal hypotheses for normal data based on a sample size of 100. The upper curve represents the logarithm of the Bayes factor (i.e. weight of evidence) in favour of H2a and the lower curve the weight of evidence in favour of H2b.
Figure 5. Plot of Bayes factors obtained under the uniformly most powerful Bayesian test versus p‐values for a 5% one‐sided test of a normal mean. The evidence threshold for the uniformly most powerful Bayesian test was determined so that the rejection region of that test matched the rejection region of a classical 5% test.
close

References

Berger JO and Pericchi LR (1996) The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association 91: 109–122.

Cover JA and Curd M (1998) Philosophy of Science: The Central Issues. New York: WM Norton & Company.

Jeffreys H (1998) Theory of Probability, 3rd edn. Oxford: Oxford University Press.

Johnson VE and Rossell D (2010) On the use of non‐local prior densities in Bayesian hypothesis tests. Journal of the Royal Statistical Society, Series B 72 (2): 143–170.

Johnson VE (2013a) Uniformly most powerful Bayesian tests. Annals of Statistics 41 (4): 1716–1741.

Johnson VE (2013b) Revised standards for statistical evidence. Proceedings of the National Academy of Sciences of the United States of America 110 (48): 19313–19317.

Moreno E, Bertolino F and Racugno W (1998) An intrinsic limiting procedure for model selection and hypotheses testing. Journal of the American Statistical Association 93: 1451–1460.

Wasserstein R and Lazar N (2016) The ASA's statement on p‐values: context, process, and purpose. The American Statistician 70 (2): 129–133.

Further Reading

Berger JO and Delampady M (1987) Testing precise hypotheses. Statistical Science 2: 317–335.

Edwards W, Lindman H and Savage LJ (1963) Bayesian statistical inference for psychological research. Psychological Review 70: 193–242.

Goodman SN (1999) Toward evidence‐based medical statistics. 2: The Bayes factor. Annals of Internal Medicine 130: 1005–1013.

Kass RE and Raftery AE (1995) Bayes factors. Journal of the American Statistical Association 90: 773–795.

O'Hagan A (1995) Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society, Series B 57: 99–118.

O'Hagan A (1997) Properties of intrinsic and fractional Bayes factors. Test 6: 101–118.

Contact Editor close
Submit a note to the editor about this article by filling in the form below.

* Required Field

How to Cite close
Johnson, Valen E(Nov 2016) Bayes' Factors. In: eLS. John Wiley & Sons Ltd, Chichester. http://www.els.net [doi: 10.1002/9780470015902.a0005851.pub2]