# krr: Non Linear Kernel Ridge Regression In mlesnoff/rnirs: Dimension reduction, Regression and Discrimination for Chemometrics

 krr R Documentation

## Non Linear Kernel Ridge Regression

### Description

Function `krr` fits KRR models, also referred to as LS-SVM (Suykens et al. 2000, Bennett & Embrechts 2003, Krell 2018).

The kernel Gram matrices `K` are internally centered before the analyses, but the data are not column-wise scaled (there is no argument `scale` in the function). If needed, the user has to do the scaling before using the function.

Row observations can eventually be weighted with a priori weights (using argument `weights`).

Note: An alternative to `krr` is to run a "direct" kernel approach (e.g. Bennett & Embrechts 2003), i.e. to build preliminary kernel Gram matrices (such as doing a pre-processing on `X`), and then run usual RR on them. See examples in function `kgram`.

See also the tuning facility with `splitpar`.

### Usage

``````
krr(Xr, Yr, Xu, Yu = NULL, lambda = 0, unit = 1,
kern = kpol, weights = NULL, print = TRUE, ...)

``````

### Arguments

 `Xr` A `n x p` matrix or data frame of reference (= training) observations. `Yr` A `n x q` matrix or data frame, or a vector of length `n`, of reference (= training) responses. `Xu` A `m x p` matrix or data frame of new (= test) observations to predict. `Yu` A `m x q` matrix or data frame, or a vector of length `m`, of the true responses for `Xu`. Default to `NULL`. `lambda` A value, or vector of values, of the regularization parameter `lambda`. `unit` A scalar. Unit used for lambda (Default to `unit = 1`). For instance, `lambda = 12, unit = 1e-6, ...` corresponds to a value `lambda = 12e-6`. `kern` A function defining the considered kernel (Default to `kpol`). See `kpol` for syntax and other available kernel functions. `weights` A vector of length `n` defining a priori weights to apply to the observations. Internally, weights are "normalized" to sum to 1. Default to `NULL` (weights are set to `1 / n`). `print` Logical (default = `TRUE`). If `TRUE`, fitting information are printed. `...` Optionnal arguments to pass in the kernel function defined in `kern`. The value set in the kernel parameters (e.g. `degree` for `kpol`) can be a scalar or a vector of several values.

### Value

A list of outputs (see examples), such as:

 `y` Responses for the test data. `fit` Predictions for the test data. `r` Residuals for the test data. `tr` The trace of the hat matrix (estimated df).

### References

Bennett, K.P., Embrechts, M.J., 2003. An optimization perspective on kernel partial least squares regression, in: Advances in Learning Theory: Methods, Models and Applications, NATO Science Series III: Computer & Systems Sciences. IOS Press Amsterdam, pp. 227-250.

Cawley, G.C., Talbot, N.L.C., 2002. Reduced Rank Kernel Ridge Regression. Neural Processing Letters 16, 293-302. https://doi.org/10.1023/A:1021798002258

Krell, M.M., 2018. Generalizing, Decoding, and Optimizing Support Vector Machine Classification. arXiv:1801.04929.

Saunders, C., Gammerman, A., Vovk, V., 1998. Ridge Regression Learning Algorithm in Dual Variables, in: In Proceedings of the 15th International Conference on Machine Learning. Morgan Kaufmann, pp. 515â521.

Suykens, J.A.K., Lukas, L., Vandewalle, J., 2000. Sparse approximation using least squares support vector machines. 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353). https://doi.org/10.1109/ISCAS.2000.856439

Welling, M., n.d. Kernel ridge regression. Department of Computer Science, University of Toronto, Toronto, Canada. https://www.ics.uci.edu/~welling/classnotes/papers_class/Kernel-Ridge.pdf

### Examples

``````
n <- 10
p <- 6
set.seed(1)
X <- matrix(rnorm(n * p, mean = 10), ncol = p, byrow = TRUE)
y1 <- 100 * rnorm(n)
y2 <- 100 * rnorm(n)
Y <- cbind(y1, y2)
set.seed(NULL)

Xr <- X[1:8, ] ; Yr <- Y[1:8, ]
Xu <- X[9:10, ] ; Yu <- Y[9:10, ]

fm <- krr(Xr, Yr, Xu, Yu, lambda = c(.1, .2), degree = 2:3, offset = c(0, 1))
## Same as:
## fm <- krr(Xr, Yr, Xu, Yu, lambda = c(1, 2), unit = .1, degree = 2:3, offset = c(0, 1))

fm\$y
fm\$fit
fm\$r

mse(fm, ~ lambda + unit + degree + scale + offset)
mse(fm, ~ lambda + unit + degree + scale + offset, nam = "y2")

``````

mlesnoff/rnirs documentation built on April 24, 2023, 4:17 a.m.