crossvaldata: Computes k-fold cross validation for rminer models.

Description Usage Arguments Details Value Note Author(s) References See Also Examples

View source: R/estimate.R

Description

Computes k-fold cross validation for rminer models.

Usage

1
2
3
crossvaldata(x, data, theta.fit, theta.predict, ngroup = 10, 
             mode = "stratified", seed = NULL, model, task, feature = "none",
             ...)

Arguments

x

See fit for details.

data

See fit for details.

theta.fit

fitting function

theta.predict

prediction function

ngroup

number of folds

mode

Possibilities are: "stratified", "random" or "order" (see holdout for details).

seed

if NULL then a random seed is used; else a fixed seed is adopted (will return always the same result for the same seed).

model

See fit for details.

task

See fit for details.

feature

See fit for details.

...

Additional parameters sent to theta.fit or theta.predic (e.g. search)

Details

Standard k-fold cross-validation adopted for rminer models. By default, for classification tasks ("class" or "prob") a stratified sampling is used (the class distributions are identical for each fold), unless mode is set to random or order (see holdout for details).

Value

Returns a list with:

Note

A better control (e.g. use of several Runs) is achieved using the simpler mining function.

Author(s)

This function was adapted by Paulo Cortez from the crossval function of the bootstrap library (S original by R. Tibshirani and R port by F. Leisch).

References

Check the crossval function of the bootstrap library.

See Also

holdout, fit, mining and predict.fit.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
 data(iris)
 # 3-fold cross validation using fit and predict
 # the control argument is sent to rpart function 
 # rpart.control() is from the rpart package
 M=crossvaldata(Species~.,iris,fit,predict,ngroup=3,seed=12345,model="rpart",
                task="prob", control = rpart::rpart.control(cp=0.05))
 print("cross validation object:")
 print(M)
 C=mmetric(iris$Species,M$cv.fit,metric="CONF")
 print("confusion matrix:")
 print(C)

Example output

[1] "cross validation object:"
$cv.fit
       [,1]       [,2]      [,3]
  [1,]    1 0.00000000 0.0000000
  [2,]    1 0.00000000 0.0000000
  [3,]    1 0.00000000 0.0000000
  [4,]    1 0.00000000 0.0000000
  [5,]    1 0.00000000 0.0000000
  [6,]    1 0.00000000 0.0000000
  [7,]    1 0.00000000 0.0000000
  [8,]    1 0.00000000 0.0000000
  [9,]    1 0.00000000 0.0000000
 [10,]    1 0.00000000 0.0000000
 [11,]    1 0.00000000 0.0000000
 [12,]    1 0.00000000 0.0000000
 [13,]    1 0.00000000 0.0000000
 [14,]    1 0.00000000 0.0000000
 [15,]    1 0.00000000 0.0000000
 [16,]    1 0.00000000 0.0000000
 [17,]    1 0.00000000 0.0000000
 [18,]    1 0.00000000 0.0000000
 [19,]    1 0.00000000 0.0000000
 [20,]    1 0.00000000 0.0000000
 [21,]    1 0.00000000 0.0000000
 [22,]    1 0.00000000 0.0000000
 [23,]    1 0.00000000 0.0000000
 [24,]    1 0.00000000 0.0000000
 [25,]    1 0.00000000 0.0000000
 [26,]    1 0.00000000 0.0000000
 [27,]    1 0.00000000 0.0000000
 [28,]    1 0.00000000 0.0000000
 [29,]    1 0.00000000 0.0000000
 [30,]    1 0.00000000 0.0000000
 [31,]    1 0.00000000 0.0000000
 [32,]    1 0.00000000 0.0000000
 [33,]    1 0.00000000 0.0000000
 [34,]    1 0.00000000 0.0000000
 [35,]    1 0.00000000 0.0000000
 [36,]    1 0.00000000 0.0000000
 [37,]    1 0.00000000 0.0000000
 [38,]    1 0.00000000 0.0000000
 [39,]    1 0.00000000 0.0000000
 [40,]    1 0.00000000 0.0000000
 [41,]    1 0.00000000 0.0000000
 [42,]    1 0.00000000 0.0000000
 [43,]    1 0.00000000 0.0000000
 [44,]    1 0.00000000 0.0000000
 [45,]    1 0.00000000 0.0000000
 [46,]    1 0.00000000 0.0000000
 [47,]    1 0.00000000 0.0000000
 [48,]    1 0.00000000 0.0000000
 [49,]    1 0.00000000 0.0000000
 [50,]    1 0.00000000 0.0000000
 [51,]    0 0.93750000 0.0625000
 [52,]    0 0.93750000 0.0625000
 [53,]    0 0.12121212 0.8787879
 [54,]    0 0.96875000 0.0312500
 [55,]    0 0.93750000 0.0625000
 [56,]    0 0.89473684 0.1052632
 [57,]    0 0.93750000 0.0625000
 [58,]    0 0.89473684 0.1052632
 [59,]    0 0.96875000 0.0312500
 [60,]    0 0.93750000 0.0625000
 [61,]    0 0.89473684 0.1052632
 [62,]    0 0.93750000 0.0625000
 [63,]    0 0.96875000 0.0312500
 [64,]    0 0.96875000 0.0312500
 [65,]    0 0.93750000 0.0625000
 [66,]    0 0.89473684 0.1052632
 [67,]    0 0.96875000 0.0312500
 [68,]    0 0.93750000 0.0625000
 [69,]    0 0.89473684 0.1052632
 [70,]    0 0.96875000 0.0312500
 [71,]    0 0.00000000 1.0000000
 [72,]    0 0.89473684 0.1052632
 [73,]    0 0.12121212 0.8787879
 [74,]    0 0.93750000 0.0625000
 [75,]    0 0.93750000 0.0625000
 [76,]    0 0.89473684 0.1052632
 [77,]    0 0.89473684 0.1052632
 [78,]    0 0.93750000 0.0625000
 [79,]    0 0.89473684 0.1052632
 [80,]    0 0.96875000 0.0312500
 [81,]    0 0.93750000 0.0625000
 [82,]    0 0.96875000 0.0312500
 [83,]    0 0.96875000 0.0312500
 [84,]    0 0.89473684 0.1052632
 [85,]    0 0.89473684 0.1052632
 [86,]    0 0.93750000 0.0625000
 [87,]    0 0.96875000 0.0312500
 [88,]    0 0.93750000 0.0625000
 [89,]    0 0.93750000 0.0625000
 [90,]    0 0.96875000 0.0312500
 [91,]    0 0.93750000 0.0625000
 [92,]    0 0.89473684 0.1052632
 [93,]    0 0.93750000 0.0625000
 [94,]    0 0.93750000 0.0625000
 [95,]    0 0.89473684 0.1052632
 [96,]    0 0.96875000 0.0312500
 [97,]    0 0.89473684 0.1052632
 [98,]    0 0.93750000 0.0625000
 [99,]    0 0.89473684 0.1052632
[100,]    0 0.96875000 0.0312500
[101,]    0 0.12121212 0.8787879
[102,]    0 0.12121212 0.8787879
[103,]    0 0.12121212 0.8787879
[104,]    0 0.12121212 0.8787879
[105,]    0 0.03225806 0.9677419
[106,]    0 0.03225806 0.9677419
[107,]    0 0.93750000 0.0625000
[108,]    0 0.03225806 0.9677419
[109,]    0 0.12121212 0.8787879
[110,]    0 0.00000000 1.0000000
[111,]    0 0.03225806 0.9677419
[112,]    0 0.03225806 0.9677419
[113,]    0 0.12121212 0.8787879
[114,]    0 0.00000000 1.0000000
[115,]    0 0.03225806 0.9677419
[116,]    0 0.03225806 0.9677419
[117,]    0 0.00000000 1.0000000
[118,]    0 0.03225806 0.9677419
[119,]    0 0.00000000 1.0000000
[120,]    0 0.93750000 0.0625000
[121,]    0 0.03225806 0.9677419
[122,]    0 0.12121212 0.8787879
[123,]    0 0.03225806 0.9677419
[124,]    0 0.12121212 0.8787879
[125,]    0 0.12121212 0.8787879
[126,]    0 0.00000000 1.0000000
[127,]    0 0.00000000 1.0000000
[128,]    0 0.12121212 0.8787879
[129,]    0 0.12121212 0.8787879
[130,]    0 0.12121212 0.8787879
[131,]    0 0.12121212 0.8787879
[132,]    0 0.00000000 1.0000000
[133,]    0 0.03225806 0.9677419
[134,]    0 0.89473684 0.1052632
[135,]    0 0.93750000 0.0625000
[136,]    0 0.12121212 0.8787879
[137,]    0 0.12121212 0.8787879
[138,]    0 0.03225806 0.9677419
[139,]    0 0.00000000 1.0000000
[140,]    0 0.12121212 0.8787879
[141,]    0 0.12121212 0.8787879
[142,]    0 0.03225806 0.9677419
[143,]    0 0.00000000 1.0000000
[144,]    0 0.12121212 0.8787879
[145,]    0 0.03225806 0.9677419
[146,]    0 0.12121212 0.8787879
[147,]    0 0.03225806 0.9677419
[148,]    0 0.00000000 1.0000000
[149,]    0 0.12121212 0.8787879
[150,]    0 0.00000000 1.0000000

$mpar
$mpar[[1]]
$mpar[[1]]$control
$mpar[[1]]$control$minsplit
[1] 20

$mpar[[1]]$control$minbucket
[1] 7

$mpar[[1]]$control$cp
[1] 0.05

$mpar[[1]]$control$maxcompete
[1] 4

$mpar[[1]]$control$maxsurrogate
[1] 5

$mpar[[1]]$control$usesurrogate
[1] 2

$mpar[[1]]$control$surrogatestyle
[1] 0

$mpar[[1]]$control$maxdepth
[1] 30

$mpar[[1]]$control$xval
[1] 10



$mpar[[2]]
$mpar[[2]]$control
$mpar[[2]]$control$minsplit
[1] 20

$mpar[[2]]$control$minbucket
[1] 7

$mpar[[2]]$control$cp
[1] 0.05

$mpar[[2]]$control$maxcompete
[1] 4

$mpar[[2]]$control$maxsurrogate
[1] 5

$mpar[[2]]$control$usesurrogate
[1] 2

$mpar[[2]]$control$surrogatestyle
[1] 0

$mpar[[2]]$control$maxdepth
[1] 30

$mpar[[2]]$control$xval
[1] 10



$mpar[[3]]
$mpar[[3]]$control
$mpar[[3]]$control$minsplit
[1] 20

$mpar[[3]]$control$minbucket
[1] 7

$mpar[[3]]$control$cp
[1] 0.05

$mpar[[3]]$control$maxcompete
[1] 4

$mpar[[3]]$control$maxsurrogate
[1] 5

$mpar[[3]]$control$usesurrogate
[1] 2

$mpar[[3]]$control$surrogatestyle
[1] 0

$mpar[[3]]$control$maxdepth
[1] 30

$mpar[[3]]$control$xval
[1] 10




$sen
NULL

$sresponses
[1] FALSE

$attributes
NULL

$ngroup
[1] 3

$leave.out
[1] 50

$groups
$groups[[1]]
 [1] 109 131 113 149  67  25  47  73 104 140   5  22 102   1  54  63  53 136  24
[20] 125  59  43 124  90  82  49  87 146  28 130  96 137  23  80 129  42 100 103
[39]  70  15 128 144 101  83 122  34   7 141   6  64

$groups[[2]]
 [1]  97 126  31  21  71  48  69   8  41 139  72 132 110  66  85  19 117  13  50
[20]  77  56  40  30  26   4 150 127  95  37  11  61  36 119   2  10  20 134  32
[39]  99  76  84 143 148  35  92  29  39  79  58 114

$groups[[3]]
 [1] 111  98  75  78  89  60  55  38 120   9 145  62  44 133 116 135 105  65  68
[20]  88  17  74   3  86  27  51 118  52 142 121  57 107  94  12  16  14 147 106
[39] 123  45 138  91 108  46  33 115 112  93  81  18


[1] "confusion matrix:"
$res
NULL

$conf
            pred
target       setosa versicolor virginica
  setosa         50          0         0
  versicolor      0         47         3
  virginica       0          4        46

$roc
NULL

$lift
NULL

rminer documentation built on Dec. 16, 2019, 5:41 p.m.