hdsvm: Solve Penalized SVM

In hdsvm: Fast Algorithm for Support Vector Machine

Solve Penalized SVM

Description

Fits a penalized support vector machine (SVM) model using a range of lambda values, allowing for detailed control over regularization parameters and model complexity.

Usage

hdsvm(
  x,
  y,
  nlambda = 100,
  lambda.factor = ifelse(nobs < nvars, 0.01, 1e-04),
  lambda = NULL,
  lam2 = 0,
  hval = 1,
  pf = rep(1, nvars),
  pf2 = rep(1, nvars),
  exclude,
  dfmax = nvars + 1,
  pmax = min(dfmax * 1.2, nvars),
  standardize = TRUE,
  eps = 1e-08,
  maxit = 1e+06,
  sigma = 0.9,
  is_exact = FALSE
)

Arguments

x	Matrix of predictors, with dimensions (n observations by p variables).
y	Response variable vector of length n.
nlambda	Number of lambda values to consider (default is 100).
lambda.factor	The factor for getting the minimal value in the lambda sequence, where min(lambda) = lambda.factor * max(lambda) and max(lambda) is the smallest value of lambda for which all coefficients (except the intercept when it is present) are penalized to zero. The default depends on the relationship between n (the number of rows in the design matrix) and p (the number of predictors). If n < p, it defaults to 0.05. If n > p, the default is 0.001, closer to zero. A very small value of lambda.factor will lead to a saturated fit. The argument takes no effect if there is a user-supplied lambda sequence.
lambda	A user-supplied lambda sequence. Typically, by leaving this option unspecified, users can have the program compute its own lambda sequence based on nlambda and lambda.factor. It is better to supply, if necessary, a decreasing sequence of lambda values than a single (small) value. The program will ensure that the user-supplied lambda sequence is sorted in decreasing order before
lam2	Regularization parameter lambda2 for the quadratic penalty of the coefficients. Unlike lambda, only one value of lambda2 is used for each fitting process.
hval	Smoothing parameter for the smoothed hinge loss, default is 1.
pf	L1 penalty factor of length p used for the adaptive LASSO or adaptive elastic net. Separate L1 penalty weights can be applied to each coefficient to allow different L1 shrinkage. Can be 0 for some variables (but not all), which imposes no shrinkage, and results in that variable always being included in the model. Default is 1 for all variables (and implicitly infinity for variables in the exclude list).
pf2	L2 penalty factor of length p used for adaptive elastic net. Separate L2 penalty weights can be applied to each coefficient to allow different L2 shrinkage. Can be 0 for some variables, which imposes no shrinkage. Default is 1 for all variables.
exclude	Indices of variables to be excluded from the model. Default is none. Equivalent to an infinite penalty factor.
dfmax	The maximum number of variables allowed in the model. Useful for very large p when a partial path is desired. Default is p+1.
pmax	Maximum count of non-zero coefficients across the solution path.
standardize	Logical flag for variable standardization, prior to fitting the model sequence. The coefficients are always returned to the original scale. Default is TRUE.
eps	Convergence criterion for stopping the algorithm.
maxit	Maximum number of iterations permitted.
sigma	Penalty parameter in the quadratic term of the augmented Lagrangian.
is_exact	If TRUE, solutions are computed exactly; otherwise, approximations are used.

Details

The function utilizes the hinge loss function combined with elastic net penalization:

1'[\max\{1 - y_i (\beta_0 + X_i^\top \beta), 0\}]/N + \lambda_1 \cdot |pf_1 \circ \beta|_1 + 0.5 \cdot \lambda_2 \cdot (\sqrt{pf_2} \circ \beta)^2,

where \circ denotes the Hadamard product.

For faster computation, if the algorithm is not converging or running slow, consider increasing eps, increasing sigma, decreasing nlambda, or increasing lambda.factor before increasing maxit.

Value

An object with S3 class hdsvm consisting of

call	the call that produced this object
b0	intercept sequence of length length(lambda)
beta	a p*length(lambda) matrix of coefficients, stored as a sparse matrix (dgCMatrix class, the standard class for sparse numeric matrices in the Matrix package.). To convert it into normal type matrix, use as.matrix().
lambda	the actual sequence of lambda values used
df	the number of nonzero coefficients for each value of lambda.
npasses	the number of iterations for every lambda value
jerr	error flag, for warnings and errors, 0 if no error.

Examples

set.seed(315)
n <- 100
p <- 400
x1 <- matrix(rnorm(n / 2 * p, -0.25, 0.1), n / 2)
x2 <- matrix(rnorm(n / 2 * p, 0.25, 0.1), n / 2)
x <- rbind(x1, x2)
beta <- 0.1 * rnorm(p)
prob <- plogis(c(x %*% beta))
y <- 2 * rbinom(n, 1, prob) - 1
lam2 <- 0.01
fit <- hdsvm(x, y, lam2=lam2)

hdsvm documentation built on April 12, 2025, 1:27 a.m.