RGF_Regressor: Regularized Greedy Forest regressor

In mlampros/RGF: Regularized Greedy Forest

Description

Regularized Greedy Forest regressor

Regularized Greedy Forest regressor

Usage

# init <- RGF_Regressor$new(max_leaf = 500, test_interval = 100,
#                                  algorithm = "RGF", loss = "LS", reg_depth = 1.0,
#                                  l2 = 0.1, sl2 = NULL, normalize = TRUE,
#                                  min_samples_leaf = 10, n_iter = NULL,
#                                  n_tree_search = 1, opt_interval = 100,
#                                  learning_rate = 0.5, memory_policy = "generous",
#                                  verbose = 0, init_model = NULL)

Details

the fit function builds a regressor from the training set (x, y).

the predict function predicts the regression target for x.

the cleanup function removes tempfiles used by this model. See the issue https://github.com/RGF-team/rgf/issues/75, which explains in which cases the cleanup function applies.

the get_params function returns the parameters of the model.

the score function returns the coefficient of determination ( R^2 ) for the predictions.

the feature_importances function returns the feature importances for the data.

the dump_model function currently prints information about the fitted model in the console

the save_model function saves a model to a file from which training can do warm-start in the future.

Methods

RGF_Regressor$new(max_leaf = 500, test_interval = 100, algorithm = "RGF", loss = "LS", reg_depth = 1.0, l2 = 0.1, sl2 = NULL, normalize = TRUE, min_samples_leaf = 10, n_iter = NULL, n_tree_search = 1, opt_interval = 100, learning_rate = 0.5, memory_policy = "generous", verbose = 0, init_model = NULL)

--------------

fit(x, y, sample_weight = NULL)

--------------

predict(x)

--------------

cleanup()

--------------

get_params(deep = TRUE)

--------------

score(x, y, sample_weight = NULL)

--------------

feature_importances()

--------------

dump_model()

--------------

save_model(filename)

--------------

Super class

RGF::Internal_class -> RGF_Regressor

Methods

Public methods

• RGF_Regressor$new()

• RGF_Regressor$clone()

Inherited methods

• RGF::Internal_class$cleanup()

• RGF::Internal_class$dump_model()

• RGF::Internal_class$feature_importances()

• RGF::Internal_class$fit()

• RGF::Internal_class$get_params()

• RGF::Internal_class$predict()

• RGF::Internal_class$predict_proba()

• RGF::Internal_class$save_model()

• RGF::Internal_class$score()

---

Method new()

Usage

RGF_Regressor$new(
  max_leaf = 500,
  test_interval = 100,
  algorithm = "RGF",
  loss = "LS",
  reg_depth = 1,
  l2 = 0.1,
  sl2 = NULL,
  normalize = TRUE,
  min_samples_leaf = 10,
  n_iter = NULL,
  n_tree_search = 1,
  opt_interval = 100,
  learning_rate = 0.5,
  memory_policy = "generous",
  verbose = 0,
  init_model = NULL
)

Arguments

max_leaf

an integer. Training will be terminated when the number of leaf nodes in the forest reaches this value.

test_interval

an integer. Test interval in terms of the number of leaf nodes.

algorithm

a character string specifying the Regularization algorithm. One of "RGF" (RGF with L2 regularization on leaf-only models), "RGF_Opt" (RGF with min-penalty regularization) or "RGF_Sib" (RGF with min-penalty regularization with the sum-to-zero sibling constraints).

loss

a character string specifying the Loss function. One of "LS" (Square loss), "Expo" (Exponential loss) or "Log" (Logistic loss).

reg_depth

a float. Must be no smaller than 1.0. Meant for being used with the algorithm RGF Opt or RGF Sib. A larger value penalizes deeper nodes more severely.

l2

a float. Used to control the degree of L2 regularization.

sl2

a float or NULL. Override L2 regularization parameter l2 for the process of growing the forest. That is, if specified, the weight correction process uses l2 and the forest growing process uses sl2. If NULL, no override takes place and l2 is used throughout training.

normalize

a boolean. If True, training targets are normalized so that the average becomes zero.

min_samples_leaf

an integer or a float. Minimum number of training data points in each leaf node. If an integer, then consider min_samples_leaf as the minimum number. If a float, then min_samples_leaf is a percentage and ceil(min_samples_leaf * n_samples) are the minimum number of samples for each node.

n_iter

an integer or NULL. The number of iterations of coordinate descent to optimize weights. If NULL, 10 is used for loss = "LS" and 5 for loss = "Expo" or "Log".

n_tree_search

an integer. The number of trees to be searched for the nodes to split. The most recently grown trees are searched first.

opt_interval

an integer. Weight optimization interval in terms of the number of leaf nodes. For example, by default, weight optimization is performed every time approximately 100 leaf nodes are newly added to the forest.

learning_rate

a float. Step size of Newton updates used in coordinate descent to optimize weights.

memory_policy

a character string. One of "conservative" (it uses less memory at the expense of longer runtime. Try only when with default value it uses too much memory) or "generous" (it runs faster using more memory by keeping the sorted orders of the features on memory for reuse). Memory using policy.

verbose

an integer. Controls the verbosity of the tree building process.

init_model

either NULL or a character string, optional (default=NULL). Filename of a previously saved model from which training should do warm-start. If model has been saved into multiple files, do not include numerical suffixes in the filename. NOTE: Make sure you haven't forgotten to increase the value of the max_leaf parameter regarding to the specified warm-start model because warm-start model trees are counted in the overall number of trees.

---

Method clone()

The objects of this class are cloneable with this method.

Usage

RGF_Regressor$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

References

https://github.com/RGF-team/rgf/tree/master/python-package, Rie Johnson and Tong Zhang, Learning Nonlinear Functions Using Regularized Greedy Forest

Examples

try({
    if (reticulate::py_available() && reticulate::py_module_available("rgf.sklearn")) {

        library(RGF)

        set.seed(1)
        x = matrix(runif(1000), nrow = 100, ncol = 10)

        y = runif(100)

        RGF_regr = RGF_Regressor$new(max_leaf = 50)

        RGF_regr$fit(x, y)

        preds = RGF_regr$predict(x)
    }
}, silent=TRUE)

mlampros/RGF documentation built on March 17, 2021, 1:50 p.m.