cross_validation: Perform cross-validation on a list of fitted models.

In rSPDE: Rational Approximations of Fractional Stochastic Partial Differential Equations

Perform cross-validation on a list of fitted models.

Description

Obtain several scores for a list of fitted models according to a folding scheme.

Usage

cross_validation(
  models,
  model_names = NULL,
  scores = c("mse", "crps", "scrps", "dss"),
  cv_type = c("k-fold", "loo", "lpo"),
  k = 5,
  percentage = 20,
  number_folds = 10,
  n_samples = 1000,
  return_scores_folds = FALSE,
  orientation_results = c("negative", "positive"),
  include_best = TRUE,
  train_test_indexes = NULL,
  return_train_test = FALSE,
  parallelize_RP = FALSE,
  n_cores_RP = parallel::detectCores() - 1,
  true_CV = FALSE,
  save_settings = FALSE,
  print = TRUE,
  fit_verbose = FALSE
)

Arguments

models	A fitted model obtained from calling the bru() function or a list of models fitted with the bru() function.
model_names	A vector containing the names of the models to appear in the returned data.frame. If NULL, the names will be of the form ⁠Model 1⁠, ⁠Model 2⁠, and so on. By default, it will try to obtain the name from the models list.
scores	A vector containing the scores to be computed. The options are "mse", "crps", "scrps" and "dss". By default, all scores are computed.
cv_type	The type of the folding to be carried out. The options are k-fold for k-fold cross-validation, in which case the parameter k should be provided, loo, for leave-one-out and lpo for leave-percentage-out, in this case, the parameter percentage should be given, and also the number_folds with the number of folds to be done. The default is k-fold.
k	The number of folds to be used in k-fold cross-validation. Will only be used if cv_type is k-fold.
percentage	The percentage (from 1 to 99) of the data to be used to train the model. Will only be used if cv_type is lpo.
number_folds	Number of folds to be done if cv_type is lpo.
n_samples	Number of samples to compute the posterior statistics to be used to compute the scores.
return_scores_folds	If TRUE, the scores for each fold will also be returned.
orientation_results	character vector. The options are "negative" and "positive". If "negative", the smaller the scores the better. If "positive", the larger the scores the better.
include_best	Should a row indicating which model was the best for each score be included?
train_test_indexes	A list containing two entries train, which is a list whose elements are vectors of indexes of the training data, and test, which is a list whose elements are vectors of indexes of the test data. Typically this will be returned list obtained by setting the argument return_train_test to TRUE.
return_train_test	Logical. Should the training and test indexes be returned? If 'TRUE' the train and test indexes will the 'train_test' element of the returned list.
parallelize_RP	Logical. Should the computation of CRPS and SCRPS be parallelized?
n_cores_RP	Number of cores to be used if parallelize_rp is TRUE.
true_CV	Should a TRUE cross-validation be performed? If TRUE the models will be fitted on the training dataset. If FALSE, the parameters will be kept fixed at the ones obtained in the result object.
save_settings	Logical. If TRUE, the settings used in the cross-validation will also be returned.
print	Should partial results be printed throughout the computation?
fit_verbose	Should INLA's run during cross-validation be verbose?

Value

A data.frame with the fitted models and the corresponding scores.

rSPDE documentation built on Nov. 6, 2023, 1:06 a.m.