calibration_control: A function that controls the calibration of models

In proximetricsR: Spectral Preprocessing and Chemometric Calibration of NIR Sensors

A function that controls the calibration of models

Description

\loadmathjax

This function is used to further control some aspects of the calibration of models (with the calibrate function) such as cross-validation and outlier detection.

Usage

calibration_control(validation_type = c("lgo", "loo", "kfold", "none"),
                    number = ifelse(validation_type == "lgo", 100, 10),
                    p = 0.75,
                    folds = c("random", "sequential"),
                    tuning_parameter = c("rmse", "rsq", "none"),
                    learning_rates = c(maximum = 1.1, sequential = 1.05),
                    remove_outliers = 0,
                    cal_residual_limit = 2.5,
                    mahalanobis_limit = 5,
                    val_residual_limit = 3.5,
                    allow_parallel = TRUE,
                    fix_pls_factors = TRUE,
                    fixed_components = 0,
                    replacements = TRUE,
                    seed = NULL)

Arguments

validation_type	a character string indicating the type of cross-validation (cv) to be conducted. Options are: "lgo" for leave-group-out cv (default), "loo" for leave-one-out cv and "kfold" for k-fold cv. See details.
number	an integer indicating the number of sampling iterations or sub-sample groups for the selected validation_type argument. Default is 100 for leave-group out cv and 10 for k-fold cross-validation. This parameter is ignored for leave-one-out cv.
p	a numeric value indicating the percentage of calibration observations to be retained at each sampling iteration at each local segment when "lgo" is selected in the validation_type argument. Default is 0.75 (i.e. 75 percent of the observations).
folds	a character string indicating the way folds are created (valid only when validation_type = "kfold"). Options are: "random" (default) or "sequential".
tuning_parameter	a character string indicating which cross-validation statistic to use for the optimization of the included number of components. Options are: "rmse" (default, minimization of the root mean squared error), "rsq" (maximization of the coefficient of determination) or "none" (no tuning). Does not apply when validation_type = "none".
learning_rates	a vector of length 2 for additional control over the selection of the optimal number of components. See details for its use. Defaults to c(1.1, 1.05).
remove_outliers	an integer indicating the number of times the model should automatically detect and remove outliers. Each time, a new model is fitted with the outliers removed, until either no more outliers are found or the remove_outliers has been reached. Outliers found and removed in each step, as well as the first and last computed models are recorded. Outliers are detected based on the limits set in the arguments cal_residual_limit, mahalanobis_limit and val_residual_limit. Setting remove_outliers to 0 (default) disables automatic outlier removal, whereas selecting it as Inf removes outliers until no more are found.
cal_residual_limit	a numeric value which indicates the upper limit of the standardized residuals for the fitted response variable. Observations with absolute residuals above this limit are labeled as "calibration outliers". The standardized calibration residuals are calculated as the absolute differences between the reference values and their corresponding fitted values divided by the standard deviation of these absolute differences. Default is 2.5 (as in NIRWise PLUS calibration software).
mahalanobis_limit	a numeric value which indicates the upper limit of the squared Mahalanobis distances of each sample in the score space to zero. Observations with squared Mahalanobis distance above this limit are labeled as "Mahalanobis outliers". The squared Mahalanobis distances are calculated as the squared Euclidean distance of the standardized scores to the origin. Default is 5 (as in NIRWise PLUS calibration software).
val_residual_limit	a numeric value which indicates the upper limit of the standardized residuals for cross-validation predictions of the response variable. This applies only to "kfold" or "loo" cross-validation. Observations with absolute residuals above this limit are labeled as "validation outliers". The standardized validation residuals are calculated as the absolute differences between the reference values and their corresponding cross-validated predictions divided by the standard deviation of these absolute differences. Default is 3.5 (as in NIRWise PLUS calibration software).
allow_parallel	a logical indicating if parallel execution is allowed. If TRUE, parallelization is applied to the cross-validation procedure. The parallelization of this for loop is implemented using the foreach function of the foreach package. Default is TRUE.
fix_pls_factors	a logical. This parameter only has an influence on the produced application files, where it indicates whether the final number of factors of the model should be fixed. Note that this has no influence on the model in R itself, as the optimal number of components inside the model remains the same (but it does influence the exported files). Default is TRUE.
fixed_components	a numerical value indicating a fixed number of components to be used in the model (i.e. no optimization of the components). The default value is 0, which indicates that the number of components is not fixed and it uses the one selected by the function.
replacements	a logical. Only used in case validation_type is selected as "lgo". Specifies if the sampling for the calibration sets must be done with replacements. See details for a more thorough explanation. Defaults to TRUE.
seed	an integer that can be used in any of the validation methods to obtain reproducible results, using the set.seed function. In case it is selected as NULL, no seed will be set. Note that the seed will not be reset, and future random computations can be affected. Furthermore, this parameter is meant as a way to provide reproducibility, and should not be used to simply select the seed with the best results. Default is NULL.

Details

This package extends the cross-validation methods implemented in the NIRWise PLUS software, which is based only on k-fold cross validation.

The validation methods available for assessing the predictive performance of the models are:

• Leave-group-out cross-validation ("lgo"): The data is partitioned into different subsets of similar size. Each partition is based on a stratified random sampling using the distribution of the response variable. When p \mjeqn\ge\geqslant 0.5 (i.e. the number of calibration observations to retain is larger than 50% of the total samples), the sampling is conducted for selecting the validation samples, and when p is below 0.5 the sampling is conducted for selecting the calibration samples (samples used for model training). The model fitted with the selected calibration samples is used to predict the target response variable values of the validation samples. The accuracy and precision, indicated by the root mean square error (RMSE) and the coefficient of determination (\mjeqnR^2R^2) respectively, are computed. This process is repeated \mjeqnmm times (where \mjeqnmm is controlled by the number argument), and the final RMSE and \mjeqnR^2R^2 are computed as the average over all respective results of the \mjeqnmm iterations. In case the parameter replacements is set to TRUE, the selection of the calibration sets is done by using sampling with replacement.

• Leave-one-out cross-validation ("loo"): The number of iterations is equal to the number of observations in the calibration set. In each iteration, one single observation is held out, while the remaining samples are used to fit a model, which is used to predict the response variable of the held out observation. The predictions are then compared to the reference ones and both the RMSE and the (\mjeqnR^2R^2) are computed.

• k-fold cross-validation ("kfold"): The data is split (either randomly or sequentially) into \mjeqnkk disjoint blocks of similar size, where \mjeqnkk is controlled by number. In the sequential splits, every block \mjeqnB_iB_i is selected as follows:

  \mjdeqn

  B_i = \lbrace i + k(j - 1) | j \in N, \ i + k(j - 1) \leq n \rbraceB_i = \lbrace i + k(j - 1)| j \in \mathbbN, \ i + k(j - 1) \leq n \rbrace

  where \mjeqnnn is the total number of observations. In other words, the observations are put sequentially into the blocks until all observations have a block assigned.
  A total of \mjeqnkk iterations is conducted. In each iteration, one block is considered as the validation set, while the remaining samples are used to fit a model, which is then used to predict the response variable of the held-out block.
  The number observations in each block is given by the total number of observations divided by the number of blocks. Note that the maximum number of folds is limited to half of the number of observations. Note also that this implementation of k-fold cross-validation is an improved version of the one in the NIRWise PLUS software, where only the sequential sample selection is supported.

• No validation ("none"): No validation is carried out.

For each validation type (except "none"), the optimal number of factors is not necessarily chosen to be the minimum of RMSE or the maximum of \mjeqnR^2R^2 (depending on the tuning_parameter). Instead, since both are often monotonically decreasing respectively monotonically increasing as the number of components increases, an additional parameter learning_rates \mjeqn\gamma\gamma for fine-tuning of the determination of the number of factors is included:

For RMSE, consider the index where the minimum of all computed RMSE is attained:

\mjdeqn

n_min = arg\min_n \ RMSE_nn_min = arg\min_n \ RMSE_n,

Then, among all \mjteqn1 < n < n_min1 \lt n \lt n_min1 \lt n \lt n_min fulfilling

\mjtdeqn

RMSE_n < RMSE_n_min \cdot \gamma_maxRMSE_n \lt RMSE_n_min \cdot \gamma_maxRMSE_n \lt RMSE_n_min \cdot \gamma_max

\mjtdeqn

RMSE_n < RMSE_n+1 \cdot \gamma_seqRMSE_n \lt RMSE_n+1 \cdot \gamma_seqRMSE_n \lt RMSE_n+1 \cdot \gamma_seq

we take the smallest \mjeqnnn as the optimal number of components.
For \mjeqnR^2R^2, a similar approach is taken, but with maxima instead of minima: \mjeqnn_max = arg\max_n R^2_nn_max = arg\max_n R^2_n Then, take the smallest \mjteqn1 < n < n_max1 \lt n \lt n_max1 \lt n \lt n_max still satisfying

\mjtdeqn

R^2_n > R^2_n_max \cdot \gamma_max^-1R^2_n \gt R^2_n_max \cdot \gamma_max^-1R^2_n \gt R^2_n_max \cdot \gamma_max^-1

\mjtdeqn

R^2_n > R^2_n+1 \cdot \gamma_seq^-1R^2_n \gt R^2_n+1 \cdot \gamma_seq^-1R^2_n \gt R^2_n+1 \cdot \gamma_seq^-1

Note that in this case, we take the inverse of the learning rates. Furthermore, setting learning_rates = c(1, 1) retains the global minimum for RMSE, respectively maximum for \mjeqnR^2R^2.

Value

a list of class calibration_control mirroring the specified parameters

Author(s)

Leonardo Ramirez-Lopez

See Also

calibrate, calibrate_models

Examples

# 5-fold cross-validation with sequential sampling
calibration_control(
  validation_type = "kfold",
  number = 5,
  folds = "sequential"
)

# leave-one-out cross_validation
calibration_control(validation_type = "loo")

# 100 leave-group-out validations with 60% samples retained, with replacements
calibration_control(
  validation_type = "lgo",
  number = 100,
  p = 0.6,
  replacements = TRUE
)

# 2-fold leave-group-out cross-validation with 75% samples retained, no replacements
calibration_control(
  validation_type = "lgo",
  number = 2,
  p = 0.75,
  replacements = FALSE
)

# Same as before, but removing any outlier that is found
calibration_control(
  validation_type = "lgo",
  number = 2,
  p = 0.75,
  replacements = FALSE,
  remove_outliers = Inf
)

# no validation, gives warning
calibration_control(validation_type = "none")

proximetricsR documentation built on Sept. 4, 2026, 5:08 p.m.