svmmaj: SVM-Maj Algorithm

In SVMMaj: Implementation of the SVM-Maj Algorithm

SVM-Maj Algorithm

Description

SVM-Maj is an algorithm to compute a support vector machine (SVM) solution. In its most simple form, it aims at finding hyperplane that optimally separates two given classes. This objective is equivalent to finding a linear combination of k predictor variables to predict the two classes for n observations. SVM-Maj minimizes the standard support vector machine (SVM) loss function. The algorithm uses three efficient updates for three different situations: primal method which is efficient in the case of n > k, the decomposition method, used when the matrix of predictor variables is not of full rank, and a dual method, that is efficient when n < k. Apart from the standard absolute hinge error, SVM-Maj can also handle the quadratic and the Huber hinge.

Usage

## S3 method for class 'q.svmmaj'
print(x, ...)

svmmaj(X, y, lambda = 1, weights.obs = 1, weights.var = 1,
  scale = c("interval", "zscore", "none"), spline.knots = 0,
  spline.degree = 1L, kernel = vanilladot, kernel.sigma = 1,
  kernel.scale = 1, kernel.degree = 1, kernel.offset = 1,
  hinge = c("absolute", "quadratic", "huber", "logitistic"),
  hinge.delta = 1e-08, options = setSVMoptions(),
  initial.point = NULL, verbose = FALSE, na.action = na.omit, ...)

## Default S3 method:
svmmaj(X, y, lambda = 1, weights.obs = 1,
  weights.var = 1, scale = c("interval", "zscore", "none"),
  spline.knots = 0, spline.degree = 1L, kernel = vanilladot,
  kernel.sigma = 1, kernel.scale = 1, kernel.degree = 1,
  kernel.offset = 1, hinge = c("absolute", "quadratic", "huber",
  "logitistic"), hinge.delta = 1e-08, options = setSVMoptions(),
  initial.point = NULL, verbose = FALSE, na.action = na.omit, ...)

Arguments

x	the svmmaj object as result of svmmaj
...	Other arguments passed to methods.
X	A data frame (or object coercible by as.data.frame to a data frame) consisting the attributes, the class of each attribute can be either numeric, logical or factor.
y	A factor (or object coercible by factor to a factor) consisting the class labels.
lambda	Regularization parameter of the penalty term.
weights.obs	a vector of length n with the nonnegative weight for the residual of each object (with length n). If the length is 2, then it specifies the weight per class.
weights.var	a vector of length k with weights for each attribute.
scale	Specifies whether the columns of attribute matrix X needs to be standardized into zscores or to the interval [0 1]. Possible values are: none, zscore and interval. Moreover, the standardization parameters can be given instead.
spline.knots	equals the number of internal knots of the spline basis. When the number of knots exceeds the number of (categorical) values of an explanatory variable, the duplicate knots will be removed using unique. For no splines, use spline.knots = 0.
spline.degree	equals the polynomial degree of the splines, for no splines:spline.degree = 1.
kernel	Specifies which kernel function to be used (see dots of package kernlab). Default kernel is the linear kernel.
kernel.sigma	additional parameters used for the kernel function (see dots)
kernel.scale	additional parameters used for the kernel function (see dots)
kernel.degree	additional parameters used for the kernel function (see dots)
kernel.offset	additional parameters used for the kernel function (see dots)
hinge	Specifies with hinge function from getHinge should be used.
hinge.delta	The parameter of the huber hinge (only if hinge = 'huber').
options	additional settings used in the svmmaj algorithm
initial.point	Initial solution.
verbose	TRUE shows the progress of the iteration.
na.action	Generic function for handling NA values.

Details

The following settings can be added as element in the options parameter: decomposition Specifies whether the QR decomposition should be used for efficient updates. Possible values are 'svd' for Singular value decomposition (Eigenvalue decomposition for non-linear kernel) or 'chol' for Cholesky (or QR decomposition in case of linear kernel)

convergence Specifies the convergence criterion of the algorithm. Default is 1e-08. increase.step The iteration number from which relaxed update will be used. eps The relaxation of the majorization function for absolute hinge: .25 * eps^-1 is the maximum steepness of the majorization function.

check.positive Specifies whether a check has to be made for positive input values. max.iter maximum number of iterations to use

Value

Returns a svmmaj-class object, of which the methods plot, plotWeights, summary and predict can be applied. (see also predict.svmmaj and print.svmmaj)

Author(s)

Hok San Yip, Patrick J.F. Groenen, Georgi Nalbantov

References

P.J.F. Groenen, G. Nalbantov and J.C. Bioch (2008) SVM-Maj: a majorization approach to linear support vector machines with different hinge errors.

See Also

dots for the computations of the kernels. predict.svmmaj normalize isb getHinge

Examples

## using default settings
model1 <- svmmaj(
 diabetes$X, diabetes$y, hinge = 'quadratic', lambda = 1)
summary(model1)

weights.obs = list(positive = 2, negative = 1)
## using radial basis kernel
library(kernlab)
model2 <- svmmaj(
  diabetes$X, diabetes$y, hinge = 'quadratic', lambda = 1, 
  weights.obs = weights.obs, scale = 'interval',
  kernel = rbfdot,
  kernel.sigma = 1
)
summary(model2)
## I-spline basis
library(ggplot2)
model3 <- svmmaj(
  diabetes$X, diabetes$y, weight.obs = weight.obs,
  spline.knots = 3, spline.degree = 2
)
plotWeights(model3, plotdim = c(2, 4))

SVMMaj documentation built on May 23, 2022, 9:05 a.m.