Description Usage Arguments Details Value Author(s) References Examples
Control the computational nuances of the train
function
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | trainControl(method = "boot",
number = ifelse(grepl("cv", method), 10, 25),
repeats = ifelse(grepl("cv", method), 1, number),
p = 0.75,
initialWindow = NULL,
horizon = 1,
fixedWindow = TRUE,
verboseIter = FALSE,
returnData = TRUE,
returnResamp = "final",
savePredictions = FALSE,
classProbs = FALSE,
summaryFunction = defaultSummary,
selectionFunction = "best",
preProcOptions = list(thresh = 0.95, ICAcomp = 3, k = 5),
index = NULL,
indexOut = NULL,
timingSamps = 0,
predictionBounds = rep(FALSE, 2),
seeds = NA,
adaptive = list(min = 5, alpha = 0.05,
method = "gls", complete = TRUE),
allowParallel = TRUE)
|
method |
The resampling method: |
number |
Either the number of folds or number of resampling iterations |
repeats |
For repeated k-fold cross-validation only: the number of complete sets of folds to compute |
verboseIter |
A logical for printing a training log. |
returnData |
A logical for saving the data |
returnResamp |
A character string indicating how much of the resampled summary metrics should be saved. Values can be “final”, “all” or “none” |
savePredictions |
a logical to save the hold-out predictions for each resample |
p |
For leave-group out cross-validation: the training percentage |
initialWindow, horizon, fixedWindow |
possible arguments to |
classProbs |
a logical; should class probabilities be computed for classification models (along with predicted values) in each resample? |
summaryFunction |
a function to compute performance metrics across resamples. The arguments to the function should be the same as those in |
selectionFunction |
the function used to select the optimal tuning parameter. This can be a name of the function or the function itself. See |
preProcOptions |
A list of options to pass to |
index |
a list with elements for each resampling iteration. Each list element is the sample rows used for training at that iteration. |
indexOut |
a list (the same length as |
timingSamps |
the number of training set samples that will be used to measure the time for predicting samples (zero indicates that the prediction time should not be estimated. |
predictionBounds |
a logical or numeric vector of length 2 (regression only). If logical, the predictions can be constrained to be within the limit of the training set outcomes. For example, a value of |
seeds |
an optional set of integers that will be used to set the seed at each resampling iteration. This is useful when the models are run in parallel. A value of |
adaptive |
a list used when |
allowParallel |
if a parallel backend is loaded and available, should the function use it? |
When setting the seeds manually, the number of models being evaluated is required. This may not be obvious as train
does some optimizations for certain models. For example, when tuning over PLS model, the only model that is fit is the one with the largest number of components. So if the model is being tuned over comp in 1:10
, the only model fit is ncomp = 10
. However, if the vector of integers used in the seeds
arguments is longer than actually needed, no error is thrown.
Using method = "none"
and specifying model than one model in train
's tuneGrid
or tuneLength
arguments will result in an error.
Using adaptive resampling when method
is either "adaptive_cv"
, "adaptive_boot"
or "adaptive_LGOCV"
, the full set of resamples is not run for each model. As resampling continues, a futility analysis is conducted and models with a low probability of being optimal are removed. These features are experimental. See Kuhn (2014) for more details. The options for this procedure are:
min
: the minimum number of resamples used before models are removed
alpha
: the confidence level of the one-sided intervals used to measure futility
method
: either generalized least squares (method = "gls"
) or a Bradley-Terry model (method = "BT"
)
complete
: if a single parameter value is found before the end of resampling, should the full set of resamples be computed for that parameter. )
An echo of the parameters specified
Max Kuhn
Kuhn (2014), “Futility Analysis in the Cross-Validation of Machine Learning Models” http://arxiv.org/abs/1405.6974
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | ## Not run:
## Do 5 repeats of 10-Fold CV for the iris data. We will fit
## a KNN model that evaluates 12 values of k and set the seed
## at each iteration.
set.seed(123)
seeds <- vector(mode = "list", length = 51)
for(i in 1:50) seeds[[i]] <- sample.int(1000, 22)
## For the last model:
seeds[[51]] <- sample.int(1000, 1)
ctrl <- trainControl(method = "repeatedcv",
repeats = 5,
seeds = seeds)
set.seed(1)
mod <- train(Species ~ ., data = iris,
method = "knn",
tuneLength = 12,
trControl = ctrl)
ctrl2 <- trainControl(method = "adaptive_cv",
repeats = 5,
verboseIter = TRUE,
seeds = seeds)
set.seed(1)
mod2 <- train(Species ~ ., data = iris,
method = "knn",
tuneLength = 12,
trControl = ctrl2)
## End(Not run)
|
Loading required package: lattice
Loading required package: ggplot2
+ Fold01.Rep1: k= 5
- Fold01.Rep1: k= 5
+ Fold01.Rep1: k= 7
- Fold01.Rep1: k= 7
+ Fold01.Rep1: k= 9
- Fold01.Rep1: k= 9
+ Fold01.Rep1: k=11
- Fold01.Rep1: k=11
+ Fold01.Rep1: k=13
- Fold01.Rep1: k=13
+ Fold01.Rep1: k=15
- Fold01.Rep1: k=15
+ Fold01.Rep1: k=17
- Fold01.Rep1: k=17
+ Fold01.Rep1: k=19
- Fold01.Rep1: k=19
+ Fold01.Rep1: k=21
- Fold01.Rep1: k=21
+ Fold01.Rep1: k=23
- Fold01.Rep1: k=23
+ Fold01.Rep1: k=25
- Fold01.Rep1: k=25
+ Fold01.Rep1: k=27
- Fold01.Rep1: k=27
+ Fold02.Rep1: k= 5
- Fold02.Rep1: k= 5
+ Fold02.Rep1: k= 7
- Fold02.Rep1: k= 7
+ Fold02.Rep1: k= 9
- Fold02.Rep1: k= 9
+ Fold02.Rep1: k=11
- Fold02.Rep1: k=11
+ Fold02.Rep1: k=13
- Fold02.Rep1: k=13
+ Fold02.Rep1: k=15
- Fold02.Rep1: k=15
+ Fold02.Rep1: k=17
- Fold02.Rep1: k=17
+ Fold02.Rep1: k=19
- Fold02.Rep1: k=19
+ Fold02.Rep1: k=21
- Fold02.Rep1: k=21
+ Fold02.Rep1: k=23
- Fold02.Rep1: k=23
+ Fold02.Rep1: k=25
- Fold02.Rep1: k=25
+ Fold02.Rep1: k=27
- Fold02.Rep1: k=27
+ Fold03.Rep1: k= 5
- Fold03.Rep1: k= 5
+ Fold03.Rep1: k= 7
- Fold03.Rep1: k= 7
+ Fold03.Rep1: k= 9
- Fold03.Rep1: k= 9
+ Fold03.Rep1: k=11
- Fold03.Rep1: k=11
+ Fold03.Rep1: k=13
- Fold03.Rep1: k=13
+ Fold03.Rep1: k=15
- Fold03.Rep1: k=15
+ Fold03.Rep1: k=17
- Fold03.Rep1: k=17
+ Fold03.Rep1: k=19
- Fold03.Rep1: k=19
+ Fold03.Rep1: k=21
- Fold03.Rep1: k=21
+ Fold03.Rep1: k=23
- Fold03.Rep1: k=23
+ Fold03.Rep1: k=25
- Fold03.Rep1: k=25
+ Fold03.Rep1: k=27
- Fold03.Rep1: k=27
+ Fold04.Rep1: k= 5
- Fold04.Rep1: k= 5
+ Fold04.Rep1: k= 7
- Fold04.Rep1: k= 7
+ Fold04.Rep1: k= 9
- Fold04.Rep1: k= 9
+ Fold04.Rep1: k=11
- Fold04.Rep1: k=11
+ Fold04.Rep1: k=13
- Fold04.Rep1: k=13
+ Fold04.Rep1: k=15
- Fold04.Rep1: k=15
+ Fold04.Rep1: k=17
- Fold04.Rep1: k=17
+ Fold04.Rep1: k=19
- Fold04.Rep1: k=19
+ Fold04.Rep1: k=21
- Fold04.Rep1: k=21
+ Fold04.Rep1: k=23
- Fold04.Rep1: k=23
+ Fold04.Rep1: k=25
- Fold04.Rep1: k=25
+ Fold04.Rep1: k=27
- Fold04.Rep1: k=27
+ Fold05.Rep1: k= 5
- Fold05.Rep1: k= 5
+ Fold05.Rep1: k= 7
- Fold05.Rep1: k= 7
+ Fold05.Rep1: k= 9
- Fold05.Rep1: k= 9
+ Fold05.Rep1: k=11
- Fold05.Rep1: k=11
+ Fold05.Rep1: k=13
- Fold05.Rep1: k=13
+ Fold05.Rep1: k=15
- Fold05.Rep1: k=15
+ Fold05.Rep1: k=17
- Fold05.Rep1: k=17
+ Fold05.Rep1: k=19
- Fold05.Rep1: k=19
+ Fold05.Rep1: k=21
- Fold05.Rep1: k=21
+ Fold05.Rep1: k=23
- Fold05.Rep1: k=23
+ Fold05.Rep1: k=25
- Fold05.Rep1: k=25
+ Fold05.Rep1: k=27
- Fold05.Rep1: k=27
o 3 eliminated;9 remain
+ Fold06.Rep1: k= 9
- Fold06.Rep1: k= 9
+ Fold06.Rep1: k=11
- Fold06.Rep1: k=11
+ Fold06.Rep1: k=13
- Fold06.Rep1: k=13
+ Fold06.Rep1: k=15
- Fold06.Rep1: k=15
+ Fold06.Rep1: k=17
- Fold06.Rep1: k=17
+ Fold06.Rep1: k=19
- Fold06.Rep1: k=19
+ Fold06.Rep1: k=21
- Fold06.Rep1: k=21
+ Fold06.Rep1: k=23
- Fold06.Rep1: k=23
+ Fold06.Rep1: k=25
- Fold06.Rep1: k=25
o 8 models of 9 were eliminated due to linear dependencies
x parameter filtering failed:[1] "Error in t.test.default(x2$value, alternative = \"greater\") : \n not enough 'x' observations\n"
attr(,"class")
[1] "try-error"
attr(,"condition")
<simpleError in t.test.default(x2$value, alternative = "greater"): not enough 'x' observations>
o 1 eliminated;0 remains
o 1 eliminated;8 remain
+ Fold08.Rep1: k= 9
- Fold08.Rep1: k= 9
+ Fold08.Rep1: k=11
- Fold08.Rep1: k=11
+ Fold08.Rep1: k=13
- Fold08.Rep1: k=13
+ Fold08.Rep1: k=15
- Fold08.Rep1: k=15
+ Fold08.Rep1: k=17
- Fold08.Rep1: k=17
+ Fold08.Rep1: k=19
- Fold08.Rep1: k=19
+ Fold08.Rep1: k=21
- Fold08.Rep1: k=21
+ Fold08.Rep1: k=25
- Fold08.Rep1: k=25
o 1 eliminated;7 remain
+ Fold09.Rep1: k= 9
- Fold09.Rep1: k= 9
+ Fold09.Rep1: k=11
- Fold09.Rep1: k=11
+ Fold09.Rep1: k=13
- Fold09.Rep1: k=13
+ Fold09.Rep1: k=15
- Fold09.Rep1: k=15
+ Fold09.Rep1: k=17
- Fold09.Rep1: k=17
+ Fold09.Rep1: k=19
- Fold09.Rep1: k=19
+ Fold09.Rep1: k=21
- Fold09.Rep1: k=21
o 6 eliminated;1 remains
+ Fold10.Rep1: k=13
- Fold10.Rep1: k=13
+ Fold01.Rep2: k=13
- Fold01.Rep2: k=13
+ Fold02.Rep2: k=13
- Fold02.Rep2: k=13
+ Fold03.Rep2: k=13
- Fold03.Rep2: k=13
+ Fold04.Rep2: k=13
- Fold04.Rep2: k=13
+ Fold05.Rep2: k=13
- Fold05.Rep2: k=13
+ Fold06.Rep2: k=13
- Fold06.Rep2: k=13
+ Fold07.Rep2: k=13
- Fold07.Rep2: k=13
+ Fold08.Rep2: k=13
- Fold08.Rep2: k=13
+ Fold09.Rep2: k=13
- Fold09.Rep2: k=13
+ Fold10.Rep2: k=13
- Fold10.Rep2: k=13
+ Fold01.Rep3: k=13
- Fold01.Rep3: k=13
+ Fold02.Rep3: k=13
- Fold02.Rep3: k=13
+ Fold03.Rep3: k=13
- Fold03.Rep3: k=13
+ Fold04.Rep3: k=13
- Fold04.Rep3: k=13
+ Fold05.Rep3: k=13
- Fold05.Rep3: k=13
+ Fold06.Rep3: k=13
- Fold06.Rep3: k=13
+ Fold07.Rep3: k=13
- Fold07.Rep3: k=13
+ Fold08.Rep3: k=13
- Fold08.Rep3: k=13
+ Fold09.Rep3: k=13
- Fold09.Rep3: k=13
+ Fold10.Rep3: k=13
- Fold10.Rep3: k=13
+ Fold01.Rep4: k=13
- Fold01.Rep4: k=13
+ Fold02.Rep4: k=13
- Fold02.Rep4: k=13
+ Fold03.Rep4: k=13
- Fold03.Rep4: k=13
+ Fold04.Rep4: k=13
- Fold04.Rep4: k=13
+ Fold05.Rep4: k=13
- Fold05.Rep4: k=13
+ Fold06.Rep4: k=13
- Fold06.Rep4: k=13
+ Fold07.Rep4: k=13
- Fold07.Rep4: k=13
+ Fold08.Rep4: k=13
- Fold08.Rep4: k=13
+ Fold09.Rep4: k=13
- Fold09.Rep4: k=13
+ Fold10.Rep4: k=13
- Fold10.Rep4: k=13
+ Fold01.Rep5: k=13
- Fold01.Rep5: k=13
+ Fold02.Rep5: k=13
- Fold02.Rep5: k=13
+ Fold03.Rep5: k=13
- Fold03.Rep5: k=13
+ Fold04.Rep5: k=13
- Fold04.Rep5: k=13
+ Fold05.Rep5: k=13
- Fold05.Rep5: k=13
+ Fold06.Rep5: k=13
- Fold06.Rep5: k=13
+ Fold07.Rep5: k=13
- Fold07.Rep5: k=13
+ Fold08.Rep5: k=13
- Fold08.Rep5: k=13
+ Fold09.Rep5: k=13
- Fold09.Rep5: k=13
+ Fold10.Rep5: k=13
- Fold10.Rep5: k=13
Aggregating results
Selecting tuning parameters
Fitting k = 13 on full training set
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.