modeling_procedure: Setup a modeling procedure
In emil: Evaluation of Modeling without Information Leakage

Description Usage Arguments Value Author(s) See Also Examples

A modeling procedure is an object containing all information necessary to carry out and evaluate the performance of a predictive modeling task with fit, tune, or evaluate. To use an out-of-the box algorithm with default values, only the method argument needs to be set. See emil for a list of available methods. To deviate from the defaults, e.g. by tuning parameters or using a custom function for model fitting, set the appropriate parameters as described below. For a guide on how to implement a custom method see the documentaion page extension.

1 2	modeling_procedure(method, parameter = list(), error_fun = NULL, fit_fun, predict_fun, importance_fun)

`method`	The name of the modeling method. Only needed to identify plug-in functions, i.e. if you supply them yourself there is no need to set `method`.
`parameter`	A list of model parameters. These will be fed to the fitting function after the dataset (`x` and `y` parameters). To tune a parameter, supply the candidate values in a vector or list. When tuning more than one parameter, all combinations of parameter values will be tested, if the elements of `parameter` are named. To manually specify which parameter value combinations to try, leave the the elements unnamed (see example 3 and 4). Parameters that should have vectors or lists as values, e.g. `trControl` when using `fit_caret` to train pkgcaret models, must be wrapped in an additional list. That is, to set a parameter value to a list, but not tune it, make it a list of length 1 containing the list to be used (see example 6).
`error_fun`	Performance measure used to evaluate procedures and to tune parameters. See `error_fun` for details.
`fit_fun`	The function to be used for model fitting.
`predict_fun`	The function to be used for model prediction.
`importance_fun`	The function to be used for calculating or extracting feature importances. See `get_importance` for details.

An object of class modeling_procedure.

Christofer Bäcklin

emil, evaluate, fit, tune, predict, get_importance

# 1: Fit linear discriminants without tuning any parameter,
# since it has none
modeling_procedure("lda")

# 2: Tune random forest's `mtry` parameter, with 3 possible values
modeling_procedure("randomForest", list(mtry = list(100, 250, 1000)))

# 3: Tune random forest's `mtry` and `maxnodes` parameters simultaneously,
# with 3 values each, testing all 9 possible combinations
modeling_procedure("randomForest", list(mtry = list(100, 250, 1000),
                                        maxnodes = list(5, 10, 25)))

# 4: Tune random forest's `mtry` and `maxnodes` parameters simultaneously,
# but only test 3 manually specified combinations of the two
modeling_procedure("randomForest", list(list(mtry = 100, maxnodes = 5),
                                   list(mtry = 250, maxnodes = 10),
                                   list(mtry = 1000, maxnodes = 25)))

# 5: Tune elastic net's `alpha` and `lambda` parameters. Since elastic net's
# fitting function can tune `lambda` internally in a more efficient way
# than the general framework is able to do, only tune `alpha` and pass all
# `lambda` values as a single argument.
modeling_procedure("glmnet", list(alpha = seq(0, 1, length.out=6),
                                  lambda = list(seq(0, 5, length.out=30))))

# 6: Train elastic nets using the caret package's model fitting framework
if(requireNamespace("caret", quitely = TRUE)){
    modeling_procedure("caret", list(method = "glmnet",
        trControl = list(trainControl(verboseIter = TRUE, classProbs = TRUE))))
}