tuning: Calculating Tuning Parameters

In statmlhb/CVEK: Cross-Validated Kernel Ensemble

Description

Calculate tuning parameters based on given criteria.

Usage

tuning(Y, X, K_mat, mode, lambda)

Arguments

Y	(vector of length n) Reponses of the dataframe.
X	(dataframe, n*p) Fixed effects variables in the dataframe (could contains several subfactors).
K_mat	(matrix, n*n) Estimated ensemble kernel matrix.
mode	(character) A character string indicating which tuning parameter criteria is to be used.
lambda	(numeric) A numeric string specifying the range of tuning parameter to be chosen. The lower limit of lambda must be above 0.

Details

There are seven tuning parameter selections here:

leave-one-out Cross Validation

λ_{n-CV}=\underset{λ \in Λ}{argmin}\;\Big\{log\;y^{\star T}[I-diag(A_λ)-\frac{1}{n}I]^{-1}(I-A_λ)^2[I-diag(A_λ)- \frac{1}{n}I]^{-1}y^\star \Big\}

Akaike Information Criteria

λ_{AIC}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)^2y^\star+\frac{2[tr(A_λ)+2]}{n}\Big\}

Akaike Information Criteria (small sample size)

λ_{AICc}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)^2y^\star+\frac{2[tr(A_λ)+2]}{n-tr(A_λ)-3}\Big\}

Bayesian Information Criteria

λ_{BIC}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)^2y^\star+\frac{log(n)[tr(A_λ)+2]}{n}\Big\}

Generalized Cross Validation

λ_{GCV}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)^2y^\star-2log[1-\frac{tr(A_λ)}{n}-\frac{1}{n}]_+\Big\}

Generalized Cross Validation (small sample size)

λ_{GCVc}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)^2y^\star-2log[1-\frac{tr(A_λ)}{n}-\frac{2}{n}]_+\Big\}

Generalized Maximum Profile Marginal Likelihood

λ_{GMPML}=\underset{λ \in Λ}{argmin}\Big\{log\; y^{\star T}(I-A_λ)y^\star-\frac{1}{n-1}log \mid I-A_λ \mid \Big\}

Value

lambda0

(numeric) The estimated tuning parameter.

Author(s)

Wenying Deng

References

Philip S. Boonstra, Bhramar Mukherjee, and Jeremy M. G. Taylor. A Small-Sample Choice of the Tuning Parameter in Ridge Regression. July 2015.

Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition. Springer Series in Statistics. Springer- Verlag, New York, 2 edition, 2009.

Hirotogu Akaike. Information Theory and an Extension of the Maximum Likelihood Princi- ple. In Selected Papers of Hirotugu Akaike, Springer Series in Statistics, pages 199–213. Springer, New York, NY, 1998.

Clifford M. Hurvich and Chih-Ling Tsai. Regression and time series model selection in small samples. June 1989.

Hurvich Clifford M., Simonoff Jeffrey S., and Tsai Chih-Ling. Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. January 2002.

Examples

lambda0 <- tuning(Y = CVEK:::model_matrices$y, 
X = CVEK:::model_matrices$X, K_mat = CVEK:::K_ens, 
mode = "loocv", lambda = exp(seq(-10, 5)))

statmlhb/CVEK documentation built on May 5, 2019, 3:47 a.m.