f2fSP_cv: Cross-validation for Overlap Group Least Absolute Shrinkage...
In fdaSP: Sparse Functional Data Analysis Methods
f2fSP_cv R Documentation
Cross-validation for Overlap Group Least Absolute Shrinkage and Selection Operator for function-on-function regression model
Description
Overlap Group-LASSO for function-on-function regression model solves the following optimization problem
\textrm{min}_{\psi} ~ \frac{1}{2} \sum_{i=1}^n \int \left( y_i(s) - \int x_i(t) \psi(t,s) dt \right)^2 ds + \lambda \sum_{g=1}^{G} \Vert S_{g}T\psi \Vert_2
to obtain a sparse coefficient vector \psi=\mathsf{vec}(\Psi)\in\mathbb{R}^{ML} for the functional penalized predictor x(t), where the coefficient matrix \Psi\in\mathbb{R}^{M\times L},
the regression function \psi(t,s)=\varphi(t)^\intercal\Psi\theta(s),
\varphi(t) and \theta(s) are two B-splines bases of order d and dimension M and L, respectively. For each group g, each row of
the matrix S_g\in\mathbb{R}^{d\times ML} has non-zero entries only for those bases belonging
to that group. These values are provided by the arguments groups and group_weights (see below).
Each basis function belongs to more than one group. The diagonal matrix T\in\mathbb{R}^{ML\times ML} contains
the basis-specific weights. These values are provided by the argument var_weights (see below).
The regularization path is computed for the overlap group-LASSO penalty at a grid of values for the regularization
parameter \lambda using the alternating direction method of multipliers (ADMM). See Boyd et al. (2011) and Lin et al. (2022)
for details on the ADMM method.
Usage
f2fSP_cv(
mY,
mX,
L,
M,
group_weights = NULL,
var_weights = NULL,
standardize.data = FALSE,
splOrd = 4,
lambda = NULL,
lambda.min.ratio = NULL,
nlambda = NULL,
cv.fold = 5,
overall.group = FALSE,
control = list()
)
Arguments
mY
an (n\times r_y) matrix of observations of the functional response variable.
mX
an (n\times r_x) matrix of observations of the functional covariate.
L
number of elements of the B-spline basis vector \theta(s).
M
number of elements of the B-spline basis vector \varphi(t).
group_weights
a vector of length G containing group-specific weights. The default is square root of the group cardinality, see Bernardi et al. (2022).
var_weights
a vector of length ML containing basis-specific weights. The default is a vector where
each entry is the reciprocal of the number of groups including that basis. See Bernardi et al. (2022) for details.
standardize.data
logical. Should data be standardized?
splOrd
the order d of the spline basis.
lambda
either a regularization parameter or a vector of regularization parameters.
In this latter case the routine computes the whole path. If it is NULL values for lambda are provided by the routine.
lambda.min.ratio
smallest value for lambda, as a fraction of the maximum lambda value. If nr_y>LM, the default is 0.0001, and if nr_y<LM, the default is 0.01.
nlambda
the number of lambda values - default is 30.
cv.fold
the number of folds - default is 5.
overall.group
logical. If it is TRUE, an overall group including all penalized covariates is added.
control
a list of control parameters for the ADMM algorithm. See ‘Details’.
Value
A named list containing
- sp.coefficients
an (M\times L) solution matrix for the parameters \Psi, which corresponds to the minimum cross-validated MSE.
- sp.fun
an (r_x\times r_y) matrix providing the estimated functional coefficient for \psi(t,s) corresponding to the minimum cross-validated MSE.
- lambda
sequence of lambda.
- lambda.min
value of lambda that attains the cross-validated minimum mean squared error.
- indi.min.mse
index of the lambda sequence corresponding to lambda.min.
- mse
cross-validated mean squared error.
- min.mse
minimum value of the cross-validated MSE for the sequence of lambda.
- mse.sd
standard deviation of the cross-validated mean squared error.
- convergence
logical. 1 denotes achieved convergence.
- elapsedTime
elapsed time in seconds.
- iternum
number of iterations.
Iteration stops when both r_norm and s_norm values
become smaller than eps_pri and eps_dual, respectively.
Details
The control argument is a list that can supply any of the following components:
- adaptation
logical. If it is TRUE, ADMM with adaptation is performed. The default value is TRUE. See Boyd et al. (2011) for details.
- rho
an augmented Lagrangian parameter. The default value is 1.
- tau.ada
an adaptation parameter greater than one. Only needed if adaptation = TRUE. The default value is 2. See Boyd et al. (2011) and Lin et al. (2022) for details.
- mu.ada
an adaptation parameter greater than one. Only needed if adaptation = TRUE. The default value is 10. See Boyd et al. (2011) and Lin et al. (2022) for details.
- abstol
absolute tolerance stopping criterion. The default value is sqrt(sqrt(.Machine$double.eps)).
- reltol
relative tolerance stopping criterion. The default value is sqrt(.Machine$double.eps).
- maxit
maximum number of iterations. The default value is 100.
- print.out
logical. If it is TRUE, a message about the procedure is printed. The default value is TRUE.
References
\insertRefbernardi_etal.2022fdaSP
\insertRefboyd_etal.2011fdaSP
\insertRefjenatton_etal.2011fdaSP
\insertReflin_etal.2022fdaSP
Examples
## generate sample data
set.seed(4321)
s <- seq(0, 1, length.out = 100)
t <- seq(0, 1, length.out = 100)
p1 <- 5
p2 <- 6
r <- 10
n <- 50
beta_basis1 <- splines::bs(s, df = p1, intercept = TRUE) # first basis for beta
beta_basis2 <- splines::bs(s, df = p2, intercept = TRUE) # second basis for beta
data_basis <- splines::bs(s, df = r, intercept = TRUE) # basis for X
x_0 <- apply(matrix(rnorm(p1 * p2, sd = 1), p1, p2), 1,
fdaSP::softhresh, 1.5) # regression coefficients
x_fun <- beta_basis2 %*% x_0 %*% t(beta_basis1)
fun_data <- matrix(rnorm(n*r), n, r) %*% t(data_basis)
b <- fun_data %*% x_fun + rnorm(n * 100, sd = sd(fun_data %*% x_fun )/3)
## set the hyper-parameters
maxit <- 1000
rho_adaptation <- FALSE
rho <- 0.01
reltol <- 1e-5
abstol <- 1e-5
## fit functional regression model
mod_cv <- f2fSP_cv(mY = b, mX = fun_data, L = p1, M = p2,
group_weights = NULL, var_weights = NULL,
standardize.data = FALSE, splOrd = 4,
lambda = NULL, nlambda = 30, cv.fold = 5,
lambda.min.ratio = NULL,
control = list("abstol" = abstol,
"reltol" = reltol,
"maxit" = maxit,
"adaptation" = rho_adaptation,
"rho" = rho,
"print.out" = FALSE))
### graphical presentation
plot(log(mod_cv$lambda), mod_cv$mse, type = "l", col = "blue", lwd = 2, bty = "n",
xlab = latex2exp::TeX("$\\log(\\lambda)$"), ylab = "Prediction Error",
ylim = range(mod_cv$mse - mod_cv$mse.sd, mod_cv$mse + mod_cv$mse.sd),
main = "Cross-validated Prediction Error")
fdaSP::confband(xV = log(mod_cv$lambda), yVmin = mod_cv$mse - mod_cv$mse.sd,
yVmax = mod_cv$mse + mod_cv$mse.sd)
abline(v = log(mod_cv$lambda[which(mod_cv$lambda == mod_cv$lambda.min)]), col = "red", lwd = 1.0)
### comparison with oracle error
mod <- f2fSP(mY = b, mX = fun_data, L = p1, M = p2,
group_weights = NULL, var_weights = NULL,
standardize.data = FALSE, splOrd = 4,
lambda = NULL, nlambda = 30, lambda.min.ratio = NULL,
control = list("abstol" = abstol,
"reltol" = reltol,
"maxit" = maxit,
"adaptation" = rho_adaptation,
"rho" = rho,
"print.out" = FALSE))
err_mod <- apply(mod$sp.coef.path, 1, function(x) sum((x - x_0)^2))
plot(log(mod$lambda), err_mod, type = "l", col = "blue", lwd = 2,
xlab = latex2exp::TeX("$\\log(\\lambda)$"),
ylab = "Estimation Error", main = "True Estimation Error", bty = "n")
abline(v = log(mod$lambda[which(err_mod == min(err_mod))]), col = "red", lwd = 1.0)
abline(v = log(mod_cv$lambda[which(mod_cv$lambda == mod_cv$lambda.min)]),
col = "red", lwd = 1.0, lty = 2)
fdaSP documentation built on Oct. 6, 2023, 1:07 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| f2fSP_cv | R Documentation |
Cross-validation for Overlap Group Least Absolute Shrinkage and Selection Operator for function-on-function regression model
Description
Overlap Group-LASSO for function-on-function regression model solves the following optimization problem
\textrm{min}_{\psi} ~ \frac{1}{2} \sum_{i=1}^n \int \left( y_i(s) - \int x_i(t) \psi(t,s) dt \right)^2 ds + \lambda \sum_{g=1}^{G} \Vert S_{g}T\psi \Vert_2
to obtain a sparse coefficient vector \psi=\mathsf{vec}(\Psi)\in\mathbb{R}^{ML} for the functional penalized predictor x(t), where the coefficient matrix \Psi\in\mathbb{R}^{M\times L},
the regression function \psi(t,s)=\varphi(t)^\intercal\Psi\theta(s),
\varphi(t) and \theta(s) are two B-splines bases of order d and dimension M and L, respectively. For each group g, each row of
the matrix S_g\in\mathbb{R}^{d\times ML} has non-zero entries only for those bases belonging
to that group. These values are provided by the arguments groups and group_weights (see below).
Each basis function belongs to more than one group. The diagonal matrix T\in\mathbb{R}^{ML\times ML} contains
the basis-specific weights. These values are provided by the argument var_weights (see below).
The regularization path is computed for the overlap group-LASSO penalty at a grid of values for the regularization
parameter \lambda using the alternating direction method of multipliers (ADMM). See Boyd et al. (2011) and Lin et al. (2022)
for details on the ADMM method.
Usage
f2fSP_cv(
mY,
mX,
L,
M,
group_weights = NULL,
var_weights = NULL,
standardize.data = FALSE,
splOrd = 4,
lambda = NULL,
lambda.min.ratio = NULL,
nlambda = NULL,
cv.fold = 5,
overall.group = FALSE,
control = list()
)
Arguments
mY |
an |
mX |
an |
L |
number of elements of the B-spline basis vector |
M |
number of elements of the B-spline basis vector |
group_weights |
a vector of length |
var_weights |
a vector of length |
standardize.data |
logical. Should data be standardized? |
splOrd |
the order |
lambda |
either a regularization parameter or a vector of regularization parameters. In this latter case the routine computes the whole path. If it is NULL values for lambda are provided by the routine. |
lambda.min.ratio |
smallest value for lambda, as a fraction of the maximum lambda value. If |
nlambda |
the number of lambda values - default is 30. |
cv.fold |
the number of folds - default is 5. |
overall.group |
logical. If it is TRUE, an overall group including all penalized covariates is added. |
control |
a list of control parameters for the ADMM algorithm. See ‘Details’. |
Value
A named list containing
- sp.coefficients
an
(M\times L)solution matrix for the parameters\Psi, which corresponds to the minimum cross-validated MSE.- sp.fun
an
(r_x\times r_y)matrix providing the estimated functional coefficient for\psi(t,s)corresponding to the minimum cross-validated MSE.- lambda
sequence of lambda.
- lambda.min
value of lambda that attains the cross-validated minimum mean squared error.
- indi.min.mse
index of the lambda sequence corresponding to lambda.min.
- mse
cross-validated mean squared error.
- min.mse
minimum value of the cross-validated MSE for the sequence of lambda.
- mse.sd
standard deviation of the cross-validated mean squared error.
- convergence
logical. 1 denotes achieved convergence.
- elapsedTime
elapsed time in seconds.
- iternum
number of iterations.
Iteration stops when both r_norm and s_norm values
become smaller than eps_pri and eps_dual, respectively.
Details
The control argument is a list that can supply any of the following components:
- adaptation
logical. If it is TRUE, ADMM with adaptation is performed. The default value is TRUE. See Boyd et al. (2011) for details.
- rho
an augmented Lagrangian parameter. The default value is 1.
- tau.ada
an adaptation parameter greater than one. Only needed if adaptation = TRUE. The default value is 2. See Boyd et al. (2011) and Lin et al. (2022) for details.
- mu.ada
an adaptation parameter greater than one. Only needed if adaptation = TRUE. The default value is 10. See Boyd et al. (2011) and Lin et al. (2022) for details.
- abstol
absolute tolerance stopping criterion. The default value is sqrt(sqrt(.Machine$double.eps)).
- reltol
relative tolerance stopping criterion. The default value is sqrt(.Machine$double.eps).
- maxit
maximum number of iterations. The default value is 100.
- print.out
logical. If it is TRUE, a message about the procedure is printed. The default value is TRUE.
References
\insertRefbernardi_etal.2022fdaSP
\insertRefboyd_etal.2011fdaSP
\insertRefjenatton_etal.2011fdaSP
\insertReflin_etal.2022fdaSP
Examples
## generate sample data
set.seed(4321)
s <- seq(0, 1, length.out = 100)
t <- seq(0, 1, length.out = 100)
p1 <- 5
p2 <- 6
r <- 10
n <- 50
beta_basis1 <- splines::bs(s, df = p1, intercept = TRUE) # first basis for beta
beta_basis2 <- splines::bs(s, df = p2, intercept = TRUE) # second basis for beta
data_basis <- splines::bs(s, df = r, intercept = TRUE) # basis for X
x_0 <- apply(matrix(rnorm(p1 * p2, sd = 1), p1, p2), 1,
fdaSP::softhresh, 1.5) # regression coefficients
x_fun <- beta_basis2 %*% x_0 %*% t(beta_basis1)
fun_data <- matrix(rnorm(n*r), n, r) %*% t(data_basis)
b <- fun_data %*% x_fun + rnorm(n * 100, sd = sd(fun_data %*% x_fun )/3)
## set the hyper-parameters
maxit <- 1000
rho_adaptation <- FALSE
rho <- 0.01
reltol <- 1e-5
abstol <- 1e-5
## fit functional regression model
mod_cv <- f2fSP_cv(mY = b, mX = fun_data, L = p1, M = p2,
group_weights = NULL, var_weights = NULL,
standardize.data = FALSE, splOrd = 4,
lambda = NULL, nlambda = 30, cv.fold = 5,
lambda.min.ratio = NULL,
control = list("abstol" = abstol,
"reltol" = reltol,
"maxit" = maxit,
"adaptation" = rho_adaptation,
"rho" = rho,
"print.out" = FALSE))
### graphical presentation
plot(log(mod_cv$lambda), mod_cv$mse, type = "l", col = "blue", lwd = 2, bty = "n",
xlab = latex2exp::TeX("$\\log(\\lambda)$"), ylab = "Prediction Error",
ylim = range(mod_cv$mse - mod_cv$mse.sd, mod_cv$mse + mod_cv$mse.sd),
main = "Cross-validated Prediction Error")
fdaSP::confband(xV = log(mod_cv$lambda), yVmin = mod_cv$mse - mod_cv$mse.sd,
yVmax = mod_cv$mse + mod_cv$mse.sd)
abline(v = log(mod_cv$lambda[which(mod_cv$lambda == mod_cv$lambda.min)]), col = "red", lwd = 1.0)
### comparison with oracle error
mod <- f2fSP(mY = b, mX = fun_data, L = p1, M = p2,
group_weights = NULL, var_weights = NULL,
standardize.data = FALSE, splOrd = 4,
lambda = NULL, nlambda = 30, lambda.min.ratio = NULL,
control = list("abstol" = abstol,
"reltol" = reltol,
"maxit" = maxit,
"adaptation" = rho_adaptation,
"rho" = rho,
"print.out" = FALSE))
err_mod <- apply(mod$sp.coef.path, 1, function(x) sum((x - x_0)^2))
plot(log(mod$lambda), err_mod, type = "l", col = "blue", lwd = 2,
xlab = latex2exp::TeX("$\\log(\\lambda)$"),
ylab = "Estimation Error", main = "True Estimation Error", bty = "n")
abline(v = log(mod$lambda[which(err_mod == min(err_mod))]), col = "red", lwd = 1.0)
abline(v = log(mod_cv$lambda[which(mod_cv$lambda == mod_cv$lambda.min)]),
col = "red", lwd = 1.0, lty = 2)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.