Fast Bayesian horseshoe regression by the ellipitical slice sampler

Share:

Description

The main function of the ellpitical slice sampler for Bayesian shrinakge regression. This function has several built-in priors and user can also provide their own prior function (written as a R function).

Usage

1
2
3
fastHorseshoe(Y, X, beta_hat = NULL, penalize = NULL, prior = "horseshoe", 
  user_prior_function = NULL, sigma = NULL, s2 = 1, kap2 = 1, N = 20000L, 
  burnin = 0L, thinning = 1L, vglobal = 100, verb = FALSE)

Arguments

Y

data.frame, matrix, or vector of inputs Y. Response variable.

X

data.frame, matrix, or vector of inputs X. Regressors.

beta_hat

matrix, or vector of inputs. Initial value of coefficients. The default value is OLS estimator.

penalize

A bool vector with the length the same as number of regressors. 1 indicates shrink corresponding regressor. The default value is to penalize all coefficients.

prior

Indicating shrinkage prior to use. "horseshoe" for approximate horseshoe prior (default), "doubleexp" for double exponential prior, "normal" for normal prior and "cauchy" for Cauchy prior.

user_prior_function

User can provide R log-prior function. See below for examples.

sigma

Initial value of residual standard error. The default value is half of standard error of Y.

s2, kap2

Parameter of prior over sigma, an inverse gamma prior with rate s2 and shape s2.

N

Number of posterior samples (after burn-in).

burnin

Number of burn-in samples.

thinning

Number of thinnings. thinning = 1 means no thinning.

vglobal

Initial value of global shrinkage parameter.

verb

Bool, if TRUE, print out sampling progress.

Details

The Bayesian lasso model and Gibbs Sampling algorithm is described in detail in Park & Casella (2008).

Value

loops

A vector of number of ellpitical slice sampler loops for each posterior sample.

sigma

A vector of posterior samples of residual standard error.

vglobal

A vector of posterior samples of the global shrinkage parameter.

beta

A matrix of posterior samples of coefficients.

Note

When using user-specified prior function, it should take a vector coefficient and a vector of global shrinkage parameters as arguments with same length as coefficients vector, and return the evaluation of logrithm of prior. See below for an example.

Author(s)

Jingyu He jingyu.he@chicagobooth.edu

References

Hahn, P. Richard, Jingyu He, and Hedibert Lopes. Elliptical slice sampling for Bayesian shrinkage regression with applications to causal inference. (2016).

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
## follow the lars diabetes example
library(lars)
data(diabetes)
attach(diabetes)

## fast horseshoe
fit1 = fastHorseshoe(y, x, N = 30000, burnin = 5000, thinning = 5)
## posterior mean of beta
colMeans(fit1$beta)
## posterior mean of global shrinakge parameter
mean(fit1$v)
## posterior mean of residual variance
mean(fit1$sigma)
## average loops of ellipitical slice sampler for each sample
mean(fit1$loops)



############################################
## only penalize the first coefficient
fit2 = fastHorseshoe(y, x, penalize = c(1, rep(0, 9)))

## Not run: 
############################################
## use user's own prior function
## approximate horseshoe prior, R code, which is the same as package default
log_horseshoe_approx_prior <- function(beta, v){
    beta = beta / v;
    ll = sum(log(log(1 + 2 / beta^2))) - sum(log(v))
    return(ll)
}

fit3 = fastHorseshoe(y, x, user_prior_function = log_horseshoe_approx_prior)


## End(Not run)