Description Usage Arguments Details Value Author(s) References Examples
This function implements the variational EM approach for the normal-beta prime (NBP) model in the standard linear regression model,
y = X β + ε,
where ε \sim N_n (0, σ^2 I_n). This is achieved by placing the normal-beta prime (NBP) prior of Bai and Ghosh (2019) on the coefficients of β. Mean field variational Bayes (MFVB) is used to approximate the posterior π(β | y) with an appropriate variational density q (β). The hyperparameters can be set deterministically by the user or they may be automatically selected by marginal maximum likelihood (MML).
It is recommended that the user use the Monte Carlo implementation, nbp
, rather than the variational Bayes (VB) function, nbp.VB
, for more accurate point esimates and posterior approximations.
1 2 |
X |
n \times p design matrix. Should be centered. |
y |
n \times 1 response vector. Should be centered. |
method.hyperparameters |
The method for estimating the shape parameters (a,b). If |
a |
Shape parameter for β'(a,b). The default is 0.5. The user may specify a different value for |
b |
Shape parameter for β'(a,b). The default is 0.5. The user may specify a different value for |
c |
The shape parameter for the IG(c,d) prior on unknown variance parameter, σ^2. The default is 10^{-5}. |
d |
The rate parameter for the IG(c,d) prior on the unknown variance parameter, σ^2. The default is 10^{-5}. |
selection |
The method of variable selection. |
tol |
The convergence criterion. If the absolute value of the difference between the current ELBO and the previous ELBO falls below |
n.iter |
The maximum number of coordinate ascent iterations to run. Defaults to 1000. |
The function implements the normal-beta prime (NBP) model of Bai and Ghosh (2019) using mean field variational Bayes (MFVB). The posterior variances and 95 percent credible intervals for each of the p covariates are also returned so that the user may assess uncertainty quantification. The full model is:
Y | (X, β ) \sim N_n(X β, σ^2 I_n),
β_i | ω_i^2 \sim N(0, σ^2 ω_i^2), i = 1, ..., p,
ω_i^2 \sim β'(a,b), i = 1, ..., p,
σ^2 \sim IG(c,d),
where β'(a,b) denotes the beta prime density,
π(ω_i^2) = \frac{Γ(a+b)}{Γ(a) Γ(b)} (ω_i^2)^{a-1} (1+ω_i^2)^{-a-b}.
The function returns a list containing the following components:
beta.hat |
The posterior mean estimate of β. |
beta.var |
The posterior variance estimates for each β_i, i=1, …, p. |
beta.intervals |
The 95 percent credible intervals for all p estimates in β. |
nbp.classifications |
A p-dimensional binary vector with "1" if the covariate is selected and "0" if it is deemed irrelevant. |
sigma2.esimate |
Estimate of unknown variance component σ^2. |
a.estimate |
MML estimate of shape parameter a. If a was fixed a priori, returns fixed a. |
b.estimate |
MML estimate of shape parameter b. If b was fixed a prior, returns fixed b. |
Ray Bai and Malay Ghosh
Bai, R. and Ghosh, M. (2019). "On the beta prime prior for scale parameters in high-dimensional Bayesian Regression models." Pre-print, arXiv:1807.06539.
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017). "Variational inference: A review for statisticians." Journal of the American Statistical Association, 112(518):859-877.
Hahn, P. R. and Carvalho, C. M. (2015). "Decoupling shrinkage and selection in Bayesian linear models: A posterior summary perspective." Journal of the American Statistical Association, 110(509):435-448.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | #################################
## Example on synethetic data. ##
#################################
n <- 50
p <- 80
X <- matrix(rnorm(n*p,-3,3), nrow=n, ncol=p)
beta.true <- c(rep(2,5), rep(0,p-5)) # True beta has five entries of '2' and rest '0'
X <- scale(X) # Center and scale X
y <- crossprod(t(X), beta.true) + rnorm(n)
################################
# Fit the NBP regression model #
# using variational Bayes #
################################
nbp.model <- nbp.VB(X=X, y=y, method.hyperparameters="mml", selection="dss")
nbp.model$beta.hat # posterior mean estimates
nbp.model$beta.var # posterior variance estimates
nbp.model$nbp.classifications # Variables selected
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.