In DataScienceSalon/Bayesian-Regression: Bayesian Linear Regression and Default Priors

Introduction

Bayesian model averaging (BMA), now widely accepted as a principled way of accounting for model uncertainty, involves the challenge of the elicitation of priors for all parameters of each model and the probability of each model. It is this elicitation of prior knowledge about the parameters of interest which often leads to more reliable estimates and smaller uncertainties. In some cases, the elicitation of prior knowledge is essential to obtaining meaningful results. If the substantial prior information is available in the form of a probability distribution, it should be used. In many cases; however, prior information may be unavailable or minuscule with respect to the information provided by the data. In such cases, default priors can be used to characterize prior probability distributions of model parameters. Still, inappropriate priors may unduly influence posterior-based inferences and decision making.

Research Question

This paper documents an examination of the following question.

What are the effects of default priors on Bayesian regression model selection, model size, posterior inclusion probabilities of regressors, inference and predictive performance?

These questions were explored within the context of a linear regression model to predict audience scores for movies, given a data set of 645 films and 16 regression parameters. Nine candidate default parameter priors were evaluated.

Background

The effect of default priors on BMA, model selection, and predictive performance has been studied in a range of disciplines including, econometrics, social sciences, biostatistics. Ley and Steel [@Ley2008] evaluated the predictive performance of several priors including Maruyama-George [@Maruyama2011], Bottolo-Richardson [@Bottolo2010], Hyper g, Hyper g/n, and Zellner Siow [@Zellner1980] priors. Combining Binomial-Beta priors on model sizes with g-priors on the coefficients of each model, they proposed a benchmark Beta prior as well as a hyper-g/n prior for econometric applications, specifically cross-country growth regression. Eicher, Papageorgiou, and Raftery [@Eicher2011] evaluated 12 priors which have been proposed in statistics and economics literature and found that the Unit Information Prior (UIP), which corresponds to the Bayesian Information Criteria (BIC) approximation of the marginal likelihood, combined with the uniform prior over the model space, outperformed, the 11 other priors. Liang et al. assessed the predictive performance of a range of g-priors and mixtures of g-priors using the highest probability models under each prior, rather than BMA [@Liang2008]. Based upon reported mean squared error (MSE) of the predictions, they concluded that there were no statistically significant differences in the predictive performance of the mixtures vis-a-vis the fixed g priors. Fernandez et al. [@FernahNdez2001] evaluated nine priors based upon Zellner's g-prior structure [@ZELLNER1985187] and recommended a g-prior equal to $1/k^2$ when $n\leq k^2$ and $1/n$ when $n > k^2$, where $n$ is the number of observations and $k$ is the number of regressors.

This paper is organized as follows. Section 2 summarizes BMA key concepts, with a focus on prior parameter specification. Section 3 introduces the motivating example and the data used for this experiment. Section 4 includes univariate and bivariate exploratory data analyses (EDA). Section 5 covers model prior specification, comparison, evaluation, and selection. Section 6 reports the prediction accuracy of the top performing models on unseen data. Finally, section 7 includes concluding remarks.

BMA modeling and prediction functionality were provided by the BAS package [@Clyde2017]. Core scripts and functions used in this study are included in the appendix. The complete source code used to produce this report is available on github at https://github.com/DataScienceSalon/Bayesian-Regression.

---

DataScienceSalon/Bayesian-Regression documentation built on May 29, 2019, 12:06 a.m.