AdditiveHierBasis: Estimating Sparse Additive Models
In asadharis/HierBasis: Nonparametric Regression and Sparse Additive Modeling

Description Usage Arguments Details Value Author(s) References See Also Examples

View source: R/hier_basis_additive.R

The main function for fitting sparse additive models via the additive HierBasis estimator

AdditiveHierBasis(x, y, nbasis = 10, max.lambda = NULL,
  lam.min.ratio = 0.01, nlam = 50, beta.mat = NULL, alpha = NULL,
  m.const = 3, max.iter = 100, tol = 1e-04, active.set = TRUE,
  type = c("gaussian", "binomial"), intercept = NULL,
  line.search.par = 0.5, step.size = 1, use.fista = TRUE,
  refit = FALSE)

`x`	An n x p matrix of covariates.
`y`	The response vector size n.
`nbasis`	The maximum number of basis functions.
`max.lambda`	The largest lambda value used for model fitting.
`lam.min.ratio`	The ratio of smallest and largest lambda values.
`nlam`	The number of lambda values.
`beta.mat`	An initial estimate of the parameter beta, a `ncol(x)`-by-`nbasis` matrix. If NULL (default), the initial estimate is the zero matrix.
`alpha`	The scalar tuning parameter controlling the additional smoothness, see details. The default is `NULL`.
`m.const`	The order of smoothness, usually not more than 3 (default).
`max.iter`	Maximum number of iterations for block coordinate descent.
`tol`	Tolerance for block coordinate descent, stopping precision.
`active.set`	Specify if the algorithm should use an active set strategy.
`type`	Specifies type of regression, "gaussian" is for linear regression with continuous response and "binomial" is for logistic regression with binary response.
`intercept`	For logistic regression, this specifies an initial value for the intercept. If `NULL` (default), then the initial value is the coefficient of the null model obtained by `glm` function.
`line.search.par`	For logistic regression, the parameter for the line search within the proximal gradient descent algorithm, this must be within the interval (0,\, 1).
`step.size`	For logistic regression, an initial step size for the line search algorithm.
`use.fista`	For using a proximal gradient descent algorithm, this specifies the use of accelerated proximal gradient descent.
`refit`	If `TRUE`, function returns the re-fitted model, i.e. the least squares estimates based on the sparsity pattern. Currently the functionality is only available for the least squares loss, i.e. `type == "gaussian"`

This function fits an additive nonparametric regression model. This is achieved by minimizing the following function of β:

minimize_{β_1,…, β_p} (1/2n)||y - ∑ Ψ_l β_l||^2 + ∑ (1-α)λ || β_l ||_2 + λα Ω_m( β_l ) ,

when α is non-null and

minimize_{β_1,…, β_p} (1/2n)||y - ∑ Ψ_l β_l||^2 + ∑ λ || β_l ||_2 + λ^2 Ω_m( β_l ) ,

when α is NULL, where β_l is a vector of length J = nbasis and the summation is over the index l. The penalty function Ω_m is given by

∑ a_{j,m}β[j:J],

where β[j:J] is beta[j:J] for a vector beta and the sum is over the index j. Finally, the weights a_{j,m} are given by

a_{j,m} = j^m - (j-1)^m,

where m denotes the 'smoothness level'. For details see Haris et al. (2018).

An object of class addHierBasis with the following elements:

`beta`	The `(nbasis * p)` x `nlam` matrix of estimated beta vectors.
`intercept`	The vector of size `nlam` of estimated intercepts.
`fitted.values`	The `n` x `nlam` matrix of fitted values.
`lambdas`	The sequence of lambda values used for fitting the different models.
`x, y`	The original `x` and `y` values used for estimation.
`m.const`	The `m.const` value used for defining 'order' of smoothness.
`nbasis`	The maximum number of basis functions used for additive HierBasis.
`xbar`	The `nbasis` x `p` matrix of means of the full design matrix.
`ybar`	The mean of the vector y.
`refit.mod`	An additional refitted model, including yhat, beta and intercepts. Only if `refit == TRUE`.

Asad Haris (aharis@uw.edu), Ali Shojaie and Noah Simon

Haris, A., Shojaie, A. and Simon, N. (2018). Nonparametric Regression with Adaptive Smoothness via a Convex Hierarchical Penalty. Available on request by authors.

Meier, L., Van de Geer, S., and Buhlmann, P. (2009). High-dimensional additive modeling. The Annals of Statistics 37.6B (2009): 3779-3821.

predict.addHierBasis, plot.addHierBasis

require(Matrix)

set.seed(1)

# Generate the points x.
n <- 100
p <- 30

x <- matrix(rnorm(n*p), ncol = p)

# A simple model with 3 non-zero functions.
y <- rnorm(n, sd = 0.1) + sin(x[, 1]) + x[, 2] + (x[, 3])^3

mod <- AdditiveHierBasis(x, y, nbasis = 50, max.lambda = 30,
                         beta.mat = NULL,
                         nlam = 50, alpha = 0.5,
                         lam.min.ratio = 1e-4, m.const = 3,
                         max.iter = 300, tol = 1e-4)

# Obtain predictions for new.x.
preds <- predict(mod, new.x = matrix(rnorm(n*p), ncol = p))

# Plot the individual functions.
xs <- seq(-3,3,length = 300)
plot(mod,1,30, type  ="l",col = "red", lwd = 2, xlab = "x", ylab = "f_1(x)",
  main = "Estimating the Sine function")
lines(xs, sin(xs), type = "l", lwd = 2)
legend("topleft", c("Estimated Function", "True Function"),
      col = c("red", "black"), lwd = 2, lty = 1)

plot(mod,2,30, type  ="l",col = "red", lwd = 2, xlab = "x", ylab = "f_2(x)",
  main = "Estimating the Linear function")
lines(xs, xs, type = "l", lwd = 2)
legend("topleft", c("Estimated Function", "True Function"),
      col = c("red", "black"), lwd = 2, lty = 1)

plot(mod,3,30, type  ="l",col = "red", lwd = 2, xlab = "x", ylab = "f_3(x)",
     main = "Estimating the cubic polynomial")
lines(xs, xs^3, type = "l", lwd = 2)
legend("topleft", c("Estimated Function", "True Function"),
       col = c("red", "black"), lwd = 2, lty = 1)