SVM-Maj Algorithm

Description

SVM-Maj is an algorithm to compute a support vector machine (SVM) solution. In its most simple form, it aims at finding hyperplane that optimally separates two given classes. This objective is equivalent to finding a linear combination of k predictor variables to predict the two classes for n observations. SVM-Maj minimizes the standard support vector machine (SVM) loss function. The algorithm uses three efficient updates for three different situations: primal method which is efficient in the case of n>k, the decomposition method, used when the matrix of predictor variables is not of full rank, and a dual method, that is efficient when n<k. Apart from the standard absolute hinge error, SVM-Maj can also handle the quadratic and the Huber hinge.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
svmmaj(X,y,lambda=1,...)
## Default S3 method:
svmmaj(X, y, lambda=1, weights.obs = 1, weights.var= 1, scale = 'none',
spline.knots=0L, spline.degree=1L, kernel=vanilladot, kernel.sigma=1, kernel.degree=1,
kernel.scale=1, kernel.offset=0, hinge='absolute', hinge.k=5, convergence=1e-8,
print.step=FALSE, initial.point=NULL, increase.step = 20, eps=1e-8,
check.positive=TRUE,na.action=na.omit,...)

plotWeights(object,plotdim=c(3,3))
## S3 method for class 'svmmaj'
plot(x,titletext=NULL,...)

Arguments

X

A data frame (or object coercible by as.data.frame to a data frame) consisting the attribues, the class of each attribute can be either numeric, logical or factor.

y

A factor (or object coercible by factor to a factor) consisting the class labels.

lambda

Regularization parameter of the penalty term.

weights.obs

Vector of length n with the nonnegative weight for the residual of each object (with length n). If the length is 2, then it specifies the weight per class.

weights.var

Vector of length k with weights for each attribute.

scale

Specifies whether the columns of attribute matrix X needs to be standardized into zscores or to the interval [0 1]. Possible values are: none, zscore and interval. Moreover, the standardization parameters can be given instead.

spline.knots

Number of internal knots of the spline basis. When the number of knots exceeds the number of (categorical) values of an explanatory variable, the duplicate knots will be removed using link[base]{unique}. spline.knots = 0 and spline.degree=1 in case of no splines.

spline.degree

The polynomial degree of the splines, for no splines: spline.knots = 0 and spline.degree=1.

kernel

Specifies which kernel function to be used (see dots of package kernlab). Default kernel is the linear kernel .

kernel.sigma

The kernel parameter sigma for the RBF kernel (see rbfdot). Default value is 1.

kernel.degree

The degree parameter of the polynomial kernel (see polydot).

kernel.scale

The scale parameter of the polynomial kernel (see polydot).

kernel.offset

The offset used in a polynomial kernel (polydot).

hinge

Specifies which hinge term to be used. Possible values are: absolute, quadratic, huber.

hinge.k

The parameter of the huber hinge, if the huber hinge is used.

convergence

Specifies the convergence criterion of the algorithm. Default is 1e-08.

print.step

print.step=TRUE shows the progress of the iteration.

initial.point

Initial solution.

increase.step

The iteration number from which relaxed update will be used.

eps

The relaxation of the majorization function for absolute hinge: .25 * eps^-1 is the maximum steepness of the majorization function.

check.positive

Specifies whether a check has to be made for positive input values.

na.action

Generic function for handling NA values.

object

The model returned from svmmaj.

x

The model returned from svmmaj used to plot the distribution of the objects per class using plot.svmmaj.

titletext

An optional title for plot.svmmaj.

plotdim

A vector of the form c(nr, nc). Subsequent figures will be drawn in an nr-by-nc array on the device.

...

Other arguments passed to methods.

Value

Returns a svmmaj-class, of which the methods plot, plotWeights, summary and predict can be applied. (see also predict.svmmaj) The returning object consist of the following values:

call

The function specifications which has been called.

lambda

The regularization parameter of the penalty term which has been used.

loss

The corresponding loss function value of the final solution.

iteration

Number of iterations needed to evaluate the algorithm.

X

The attribute matrix of dim(X) = c(n,k).

y

The vector of length n with the actual class labels. These labels can be numeric [0 1] or two strings.

classes

A vector of length n with the predicted class labels of each object, derived from q.tilde

Xtrans

The attribute matrix X after standardization and (if specified) spline transformation.

norm.param

The applied normalization parameters (see normalize).

splineInterval

The spline knots which has been used (see isb).

splineLength

Denotes the number of spline basis of each explanatory variable in X.

method

The decomposition matrices used in estimating the model.

hinge

The hinge function which has been used (see hinge).

beta

If identified, the beta parameters for the linear combination (only available for linear kernel).

q

A vector of length n with predicted values of each object including the intercept.

nSV

Number of support vectors.

plotWeights shows, one graph per attribute, the weights of all attributes. The type of graph depends on the type of the attribute: the spline line of the corresponding attribute in case a spline has been used, a bar plot for categorical and logical values, and a linear line for all other type of the attribute values. This function cannot be used in a model with a non-linear kernel.

Author(s)

Hok San Yip, Patrick J.F. Groenen, Georgi Nalbantov

References

P.J.F. Groenen, G. Nalbantov and J.C. Bioch (2008) SVM-Maj: a majorization approah to linear support vector machines with different hinge errors.

See Also

dots for the computations of the kernels. predict.svmmaj normalize isb hinge

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
attach(diabetes)

## using default settings
model1 <- model <- svmmaj(X,y,hinge='quadratic', lambda=1)
summary(model1)

weights.obs = list(positive=2,negative=1)
## using radial basis kernel
model2 <- svmmaj(X,y,hinge='quadratic', lambda=1, weights.obs=weights.obs,
              standardize='interval',kernel=rbfdot,
              kernel.sigma=1)
summary(model2)
## I-spline basis
model3 <- svmmaj(X,y,,weights.obs=weights.obs,spline.knots=3,spline.degree=2)
plotWeights(model3,plotdim=c(2,4))