Description Usage Arguments Details Value References See Also Examples
Estimates the relative overfitting of a model as the ratio of the difference in test and train performance to the difference of test performance in the no-information case and train performance. In the no-information case the features carry no information with respect to the prediction. This is simulated by permuting features and predictions.
1 2 3 4 5 | estimateRelativeOverfitting(rdesc, measures, task, learner)
## S3 method for class 'ResampleDesc'
estimateRelativeOverfitting(rdesc, measures, task,
learner)
|
rdesc |
[ |
measures |
[ |
task |
[ |
learner |
[ |
Currently only support for classification and regression tasks is implemented.
[data.frame
]. Relative overfitting estimate(s), named by measure(s), for each resampling iteration.
Bradley Efron and Robert Tibshirani; Improvements on Cross-Validation: The .632+ Bootstrap Method, Journal of the American Statistical Association, Vol. 92, No. 438. (Jun., 1997), pp. 548-560.
Other performance: ConfusionMatrix
,
calculateConfusionMatrix
,
calculateROCMeasures
,
makeCostMeasure
,
makeCustomResampledMeasure
,
makeMeasure
, measures
,
performance
1 2 3 4 | task = makeClassifTask(data = iris, target = "Species")
rdesc = makeResampleDesc("CV", iters = 2)
estimateRelativeOverfitting(rdesc, acc, task, makeLearner("classif.knn"))
estimateRelativeOverfitting(rdesc, acc, task, makeLearner("classif.lda"))
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.