aggregate.cv: Aggregate cross-validation results

In cvTools: Cross-validation tools for regression models

Description

Compute summary statistics of results from repeated K-fold cross-validation.

Usage

  ## S3 method for class 'cv'
 aggregate(x, FUN = mean, select = NULL,
    ...)

  ## S3 method for class 'cvSelect'
 aggregate(x, FUN = mean,
    select = NULL, ...)

  ## S3 method for class 'cvTuning'
 aggregate(x, ...)

Arguments

x	an object inheriting from class "cv" or "cvSelect" that contains cross-validation results (note that the latter includes objects of class "cvTuning").
FUN	a function to compute the summary statistics.
select	a character, integer or logical vector indicating the columns of cross-validation results for which to compute the summary statistics.
...	for the "cvTuning" method, additional arguments to be passed to the "cvSelect" method. Otherwise additional arguments to be passed to FUN.

Value

The "cv" method returns a vector or matrix of aggregated cross-validation results, depending on whether FUN returns a single value or a vector.

For the other methods, a data frame containing the aggregated cross-validation results for each model is returned. In the case of the "cvTuning" method, the data frame contains the combinations of tuning parameters rather than a column describing the models.

Author(s)

Andreas Alfons

See Also

cvFit, cvSelect, cvTuning, aggregate

Examples

library("robustbase")
data("coleman")
set.seed(1234)  # set seed for reproducibility

## set up folds for cross-validation
folds <- cvFolds(nrow(coleman), K = 5, R = 10)


## compare raw and reweighted LTS estimators for 
## 50% and 75% subsets

# 50% subsets
fitLts50 <- ltsReg(Y ~ ., data = coleman, alpha = 0.5)
cvFitLts50 <- cvLts(fitLts50, cost = rtmspe, folds = folds, 
    fit = "both", trim = 0.1)

# 75% subsets
fitLts75 <- ltsReg(Y ~ ., data = coleman, alpha = 0.75)
cvFitLts75 <- cvLts(fitLts75, cost = rtmspe, folds = folds, 
    fit = "both", trim = 0.1)

# combine results into one object
cvFitsLts <- cvSelect("0.5" = cvFitLts50, "0.75" = cvFitLts75)
cvFitsLts

# summary of the results with the 50% subsets
aggregate(cvFitLts50, summary)
# summary of the combined results
aggregate(cvFitsLts, summary)


## evaluate MM regression models tuned for 
## 80%, 85%, 90% and 95% efficiency
tuning <- list(tuning.psi=c(3.14, 3.44, 3.88, 4.68))

# set up function call
call <- call("lmrob", formula = Y ~ .)
# perform cross-validation
cvFitsLmrob <- cvTuning(call, data = coleman, 
    y = coleman$Y, tuning = tuning, cost = rtmspe, 
    folds = folds, costArgs = list(trim = 0.1))
cvFitsLmrob

# summary of results
aggregate(cvFitsLmrob, summary)

Example output

Loading required package: lattice
Loading required package: robustbase

5-fold CV results:
   Fit reweighted      raw
1  0.5   1.140772 1.511817
2 0.75   0.963192 1.165930

Best model:
reweighted        raw 
    "0.75"     "0.75" 
        reweighted      raw
Min.     0.8426695 1.121681
1st Qu.  0.9832825 1.476071
Median   1.1400160 1.502916
Mean     1.1407723 1.511817
3rd Qu.  1.2835384 1.623199
Max.     1.5522526 1.875849
   Fit reweighted.Min. reweighted.1st Qu. reweighted.Median reweighted.Mean
1  0.5       0.8426695          0.9832825         1.1400160       1.1407723
2 0.75       0.8360649          0.8979938         0.9829863       0.9631920
  reweighted.3rd Qu. reweighted.Max. raw.Min. raw.1st Qu. raw.Median raw.Mean
1          1.2835384       1.5522526 1.121681    1.476071   1.502916 1.511817
2          1.0241433       1.0850610 1.046983    1.097745   1.145762 1.165930
  raw.3rd Qu. raw.Max.
1    1.623199 1.875849
2    1.234328 1.298372

5-fold CV results:
  tuning.psi        CV
1       3.14 0.9683601
2       3.44 0.9654708
3       3.88 0.9681099
4       4.68 0.9865325

Optimal tuning parameter:
   tuning.psi
CV       3.44
  tuning.psi   CV.Min. CV.1st Qu. CV.Median   CV.Mean CV.3rd Qu.   CV.Max.
1       3.14 0.8387515  0.9198769 0.9743848 0.9683601  1.0278803 1.0829762
2       3.44 0.8375457  0.9203998 0.9701659 0.9654708  1.0197471 1.0855025
3       3.88 0.8375048  0.9220203 0.9713238 0.9681099  1.0395408 1.0982037
4       4.68 0.8087384  0.9208760 0.9892362 0.9865325  1.0601024 1.1479685

cvTools documentation built on May 2, 2019, 12:53 a.m.