On Model Comparison: Application of Savage-Dickey Density Ratio to Bayes Factor

Main Article Content

Olawale B. Akanbi
Olusanya E. Olubusoye
Samuel A. Babatunde

Abstract

Bayes factor is a major Bayesian tool for model comparison especially when the model priors are the same. In this paper, the Savage-Dickey Density Ratio (SDDR) is used to derive the Bayes factor to select a model from two competing models under consideration in a normal linear regression with an independent normal-gamma prior. The Gibbs sampling technique for the joint posterior distribution with equal prior precision for both the unrestricted and restricted models is used to obtain the model estimates. The result shows that the Bayes factor gave more support to the unrestricted model against the restricted and was consistent irrespective of changes in sample size.

Keywords:
Nested model, prior precision, Savage-Dickey density ratio, Gibbs sampling.

Article Details

How to Cite
Akanbi, O. B., Olubusoye, O. E., & Babatunde, S. A. (2020). On Model Comparison: Application of Savage-Dickey Density Ratio to Bayes Factor. Asian Journal of Probability and Statistics, 6(3), 25-46. https://doi.org/10.9734/ajpas/2020/v6i330162
Section
Original Research Article

References

Anderson CJ. The analysis of multivariate, longitudinal categorical data by Log-Multilinear models. Ph.D Thesis, University of Illinois at Urbana-Champaign; 1993.

Jeffreys H. Theory of probability (3rd Edition). New York: Oxford University Press; 1961. Joyce, James. Zalta, Edward N, (Ed.), "Bayes' Theorem", The Stanford Encyclopedia of Philosophy (Spring 2019 Ed.), Metaphysics Research Lab, Stanford University; 2003.
(Retrieved 2020-01-17)

Schonbrodt FD, Wagenmakers EJ. Bayes factor design analysis: Planning for compelling evidence. 2018;25(1):128-142.

Gronau QF, Wagenmakers EJ, Heck DW, Matzke D. A simple method for comparing complex models: Bayesian model comparison for hierarchical multinomial processing tree models using Warp-III bridge sampling. Psychometrika. 2019a;84(1):261–284.

Kass RE, Raftery AE. Bayes factors. Journal of the American Statistical Association. 1995;90:773–795.

Berger JO, Mortera J. Interpreting the stars in precise hypothesis testing. International Statistical Review / Revue Internationale de Statistique. 1991;59(3):337-353.

Edwards W, Lindman H, Savage LJ. Bayesian statistical inference for psychological research. Psychological Review. 1963;70:193-242.

Liang F, Paulo R, Molina G, Clyde MA, Berger JO. Mixtures of g-priors for Bayesian variable selection. Journal of the American Statistical Association. 2008;103:410-423.

Gronau QF, Heathcote A, Matzke D. Computing Bayes factors for evidence-accumulation models using Warp-III bridge sampling. Behavior Research Methods. 2019b;1–20.

Wagenmakers EJ, Morey RD, Lee MD. Bayesian benefits for the pragmatic researcher. Current Directions in Psychological Science. 2016;25:169–176.

Dickey J, Lientz B. The weighted likelihood ratio, sharp hypothesis about chances, the order of a Markov chain. The Annals of Mathematical Statistics. 1970;41(1):214-226.

Dickey J. The weighted likelihood ratio linear hypotheses on normal location parameters. The Annals of Mathematical Statistics. 1971;42:204–223.

Dickey J. Approximate posterior distributions. Journal of the American Statistical Association. 1976;71:680–689.

Gunel E, Dickey J. Bayes factors for independence in contingency tables. Biometrika. 1974;61(3):545–557.

Sander Van der Linden, Chryst B. No need for Bayes factors: A fully Bayesian evidence synthesis. Frontiers in Appl. Math. Stat. 2017;3(12):1-3.

Newton MA, Raftery AE. Approximate Bayesian inference by the weighted likelihood bootstrap. J. Roy. Statist. Soc. B. 1994;56:3–48.

Gelfand A, Dey D. Bayesian model choice: Asymptotics and exact calculations. Journal of the Royal Statistical Society Series B. 1994;56:501–514.

Ewan, Cameron. A generalized Savage-Dickey density ratio. Electronic Journal of Statistics. 2013;176:1311–1292.

Feller W. An introduction to probability theory and its applications. Third Edition, John Wiley & Sons Inc. 1971;1.

Verdinelli I, Wasserman L. Computing Bayes factors using a generalization of the Savage Dickey density ratio. Journal of the American Statistical Association. 1995;90:614-618.