dist.Zellner: Hyperprior-g Prior and Zellner's g-Prior
In LaplacesDemon: Complete Environment for Bayesian Inference

Description Usage Arguments Details Value References See Also Examples

These functions provide the density of the hyper-g prior (Liang et al., 2008), and both the density and random generation of Zellner's g-prior (Zellner, 1986).

1
2
3

dhyperg(g, alpha=3, log=FALSE)
dzellner(beta, g, sigma, X, log=FALSE)
rzellner(n, g, sigma, X)

`alpha`	This is a positive scale hyperhyperparameter that is proper when alpha > 2. The default is `alpha=3`.
`beta`	This is regression effects beta, a vector of length J.
`g`	This is hyperparameter g, a positive scalar.
`n`	This is the number of random deviates to generate.
`sigma`	This is the residual standard deviation sigma, a positive scalar.
`X`	This is a full-rank N x J design matrix X for N records and J predictors, where J+1 < N. Zellner's g-prior has been extended (elsewhere) via singular value decomposition (SVD) to the case where J > N.
`log`	Logical. If `log=TRUE`, then the logarithm of the density is returned.

Application: Continuous Multivariate
Density: p(theta) = (1/((2*pi)^(J/2)*|Sigma|^(1/2))) * exp(-(1/2)*(theta-mu)'*Sigma^(-1)*(theta-mu))
Inventor: Zellner, A. (1986)
Notation 1: theta ~ N[J](0, g sigma^2 (X^T X)^(-1))
Notation 2: p(theta) = N[J](theta, g, sigma^2, X)
Parameter 1: location parameter beta
Parameter 2: scale parameter g > 0
Parameter 3: scale parameter sigma^2 > 0
Mean:
Variance:
Mode:

Zellner's g-prior is a popular, data-dependent, elliptical, improper, least-informative prior distribution on regression effects beta in a Gaussian regression model. It is a particular form in the conjugate Normal-Gamma family. Zellner's g-prior is also used for estimating Bayes factors (for hypothesis testing) with a simpler form, as well as in model selection and variable selection. The marginal posterior distribution of regression effects beta is multivariate t.

One of many nice properties of Zellner's g-prior is that it adapts automatically to near-collinearity between different predictors. Zellner's g-prior puts most of its prior mass in the direction that causes the regression coefficients of correlated predictors to be smoothed away from each other. When coupled with model selection, Zellner's g-prior discourages highly collinear predictors from entering the models simultaneously by inducing a negative correlation between the coefficients. However, when it is desirable for collinear predictors to enter simultaneously, a modification has been proposed (though not included here) in which (X^T X)^(-1) is replaced with (X^T X)^lambda. For more information, see Krishna et al. (2009).

For variable selection, large values of g, with a prior mean of zero for beta, encourage models with few, large coefficients. Conversely, small values of g encourage saturated models with many, small coefficients.

The design matrix X is converted to Fisher's information matrix, which is used as a covariance matrix for beta. This is computationally efficient, because each element of the covariance matrix does not need to be estimated as a parameter. When X is nearly singular, regression effects beta may be poorly estimated.

Hyperparameter g acts as an inverse relative prior sample size, or as a dimensionality penalty. Zellner (1986) recommended that a hyperprior distribution is assigned to g so that it is estimated from the data, although in practice g has often been fixed, usually to N when no information is available, since it has the interpretation of adding prior information equivalent to one observation. A variety of hyperpriors have been suggested for g, such as in Bove and Held (2011), Liang et al. (2008), and Maruyama and George (2011). g becomes diffuse as it approaches infinity, and the Bayes factor approaches zero. The hyper-g prior of Liang et al. (2008) is proper when alpha > 2, and any value in the interval (2,4] may be reasonable.

dhyperg gives the density of the hyper-g prior of Liang et al. (2008), dzellner gives the density of Zellner's g-prior, and rzellner generates random deviates.

Bove, D.S. and Held, L. (2011). "Hyper-g Priors for Generalized Linear Models". Bayesian Analysis, 6(3), p. 387–410.

Krishna, A., Bondell, H.D., and Ghosh, S.K. (2009). "Bayesian Variable Selection Using an Adaptive Powered Correlation Prior". Journal of Statistical Planning Inference, 139(8), p. 2665-2674..

Liang, F., Paulo, R., Molina, G., Clyde, M.A., and Berger, J.O. (2008). "Mixtures of g Priors for Bayesian Variable Selection". Journal of the American Statistical Association, 103, p. 410–423.

Maruyama, Y. and George, E.I. (2011). "Fully Bayes Factors with a Generalised g-Prior". Annals of Statistics, 39, p. 2740–2765.

Zellner, A. (1986). "On Assessing Prior Distributions and Bayesian Regression Analysis with g-Prior Distributions". In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, p. 233–243. Elsevier: Amsterdam, North Holland.

BayesFactor and dmvt

library(LaplacesDemon)
set.seed(667)
beta <- rnorm(10)
g <- 100
sigma <- 2
X <- cbind(1,matrix(rnorm(100*9),100,9))
dhyperg(g, alpha=3)
dzellner(beta, g, sigma, X)
rzellner(1, g, sigma, X)

[1] 0.0004925927
[1] 5.445798e-08
        [,1]       [,2]     [,3]      [,4]      [,5]      [,6]     [,7]
mu 0.1957988 -0.6647923 2.152082 -2.126497 0.4301386 -2.412332 1.674801
        [,8]      [,9]      [,10]
mu 0.5850825 -1.437327 -0.8022153