krr: Non Linear Kernel Ridge Regression

In mlesnoff/rnirs: Dimension reduction, Regression and Discrimination for Chemometrics

Non Linear Kernel Ridge Regression

Description

Function krr fits KRR models, also referred to as LS-SVM (Suykens et al. 2000, Bennett & Embrechts 2003, Krell 2018).

The kernel Gram matrices K are internally centered before the analyses, but the data are not column-wise scaled (there is no argument scale in the function). If needed, the user has to do the scaling before using the function.

Row observations can eventually be weighted with a priori weights (using argument weights).

Note: An alternative to krr is to run a "direct" kernel approach (e.g. Bennett & Embrechts 2003), i.e. to build preliminary kernel Gram matrices (such as doing a pre-processing on X), and then run usual RR on them. See examples in function kgram.

See also the tuning facility with splitpar.

Usage

krr(Xr, Yr, Xu, Yu = NULL, lambda = 0, unit = 1, 
                 kern = kpol, weights = NULL, print = TRUE, ...)

Arguments

Xr	A n x p matrix or data frame of reference (= training) observations.
Yr	A n x q matrix or data frame, or a vector of length n, of reference (= training) responses.
Xu	A m x p matrix or data frame of new (= test) observations to predict.
Yu	A m x q matrix or data frame, or a vector of length m, of the true responses for Xu. Default to NULL.
lambda	A value, or vector of values, of the regularization parameter lambda.
unit	A scalar. Unit used for lambda (Default to unit = 1). For instance, lambda = 12, unit = 1e-6, ... corresponds to a value lambda = 12e-6.
kern	A function defining the considered kernel (Default to kpol). See kpol for syntax and other available kernel functions.
weights	A vector of length n defining a priori weights to apply to the observations. Internally, weights are "normalized" to sum to 1. Default to NULL (weights are set to 1 / n).
print	Logical (default = TRUE). If TRUE, fitting information are printed.
...	Optionnal arguments to pass in the kernel function defined in kern. The value set in the kernel parameters (e.g. degree for kpol) can be a scalar or a vector of several values.

Value

A list of outputs (see examples), such as:

y	Responses for the test data.
fit	Predictions for the test data.
r	Residuals for the test data.
tr	The trace of the hat matrix (estimated df).

References

Bennett, K.P., Embrechts, M.J., 2003. An optimization perspective on kernel partial least squares regression, in: Advances in Learning Theory: Methods, Models and Applications, NATO Science Series III: Computer & Systems Sciences. IOS Press Amsterdam, pp. 227-250.

Cawley, G.C., Talbot, N.L.C., 2002. Reduced Rank Kernel Ridge Regression. Neural Processing Letters 16, 293-302. https://doi.org/10.1023/A:1021798002258

Krell, M.M., 2018. Generalizing, Decoding, and Optimizing Support Vector Machine Classification. arXiv:1801.04929.

Saunders, C., Gammerman, A., Vovk, V., 1998. Ridge Regression Learning Algorithm in Dual Variables, in: In Proceedings of the 15th International Conference on Machine Learning. Morgan Kaufmann, pp. 515–521.

Suykens, J.A.K., Lukas, L., Vandewalle, J., 2000. Sparse approximation using least squares support vector machines. 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353). https://doi.org/10.1109/ISCAS.2000.856439

Welling, M., n.d. Kernel ridge regression. Department of Computer Science, University of Toronto, Toronto, Canada. https://www.ics.uci.edu/~welling/classnotes/papers_class/Kernel-Ridge.pdf

Examples

n <- 10
p <- 6
set.seed(1)
X <- matrix(rnorm(n * p, mean = 10), ncol = p, byrow = TRUE)
y1 <- 100 * rnorm(n)
y2 <- 100 * rnorm(n)
Y <- cbind(y1, y2)
set.seed(NULL)

Xr <- X[1:8, ] ; Yr <- Y[1:8, ] 
Xu <- X[9:10, ] ; Yu <- Y[9:10, ] 

fm <- krr(Xr, Yr, Xu, Yu, lambda = c(.1, .2), degree = 2:3, offset = c(0, 1))
## Same as:
## fm <- krr(Xr, Yr, Xu, Yu, lambda = c(1, 2), unit = .1, degree = 2:3, offset = c(0, 1))

fm$y
fm$fit
fm$r

mse(fm, ~ lambda + unit + degree + scale + offset)
mse(fm, ~ lambda + unit + degree + scale + offset, nam = "y2")

mlesnoff/rnirs documentation built on April 24, 2023, 4:17 a.m.