Description Usage Arguments Details Value Examples
View source: R/model_selection.R
compare pairs of models using statistics such as t.test, correlation, evaluation.
1 2 3 4 5 6 7 8 9 |
list_of_algos |
a list of model objects (first use the subset_mods to select the best and then re-run) |
on_Train |
if TRUE, then it applies the test-statistics on train-data |
regression |
is it a regression or a classification task |
evaluation_metric |
one of the evaluation metrics (accuracy, rmse, etc.) |
t.test.conf.int |
specify confidence interval for the t.test statistic (0.95, 0.99, etc.) |
cor_test |
one of spearman, pearson, kendal |
sort_decreasing |
sorts the resulted data.frame by the evaluation metric of the first algorithm in either increasing or decreasing order |
This function takes a list of objects after they were subset and re-run on the same resampling method. It returns a data frame with statistics for each pair of them.
a data frame
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ## Not run:
#..............
# random-forest
#..............
res_rf = random_search_resample(as.factor(y1), tune_iters = 30,
resampling_method = list(method = 'cross_validation',
repeats = NULL,
sample_rate = NULL,
folds = 5),
ALGORITHM = list(package = require(randomForest),
algorithm = randomForest),
grid_params = bst_m$rf,
DATA = list(x = X, y = as.factor(y1)),
Args = NULL,
regression = FALSE, re_run_params = TRUE)
#..............
# RWeka Bagging
#..............
res_logitBoost = random_search_resample(as.factor(y1), tune_iters = 30,
resampling_method = list(method = 'cross_validation',
repeats = NULL,
sample_rate = NULL,
folds = 5),
ALGORITHM = list(package = require(RWeka),
algorithm = LogitBoost),
grid_params = bst_m$logitboost_weka,
DATA = list(formula = form, data = ALL_DATA),
Args = NULL,
regression = FALSE, re_run_params = TRUE)
tmp_lst = list(rf = res_rf, LogBoost = res_logitBoost)
res = model_selection(tmp_lst,
on_Train = FALSE,
regression = FALSE,
evaluation_metric = 'acc',
t.test.conf.int = 0.95,
cor_test = list(method = 'spearman'),
sort_decreasing = TRUE)
res
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.