Learning to Rank Bagging Workflows with Metalearning
Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques together with the market scarcity on ML experts, has created the need for systems that enable a fast and easy drop-in replacement for ML libraries. Automated machine learning (autoML) is the field of ML that attempts to answers these needs. Typically, these systems rely on optimization techniques such as bayesian optimization to lead the search for the best model. Our approach differs from these systems by making use of the most recent advances on metalearning and a learning to rank approach to learn from metadata. We propose autoBagging, an autoML system that automatically ranks 63 bagging workflows by exploiting past performance and dataset characterization. Results on 140 classification datasets from the OpenML platform show that autoBagging can yield better performance than the Average Rank method and achieve results that are not statistically different from an ideal model that systematically selects the best workflow for each dataset.
formula. Currently supporting only categorical target variables (classification tasks)
training dataset with a categorical target variable
The underlying model leverages the performance of the workflows in historical data. It ranks and recommends workflows for a given classification task. A bagging workflow is comprised by the following steps:
the number of trees to grow
the pruning of low performing trees in the ensemble
a parameter of the previous step
the dynamic selection method used to aggregate predictions. If none is recommended, majority voting is used.
abmodel class object
Pinto, F., Cerqueira, V., Soares, C., Mendes-Moreira, J.: "autoBagging: Learning to Rank Bagging Workflows with Metalearning" arXiv preprint arXiv:1706.09367 (2017).
1 2 3 4 5 6 7 8 9 10 11 12
## Not run: # splitting an example dataset into train/test: train <- iris[1:(.7*nrow(iris)), ] test <- iris[-c(1:(.7*nrow(iris))), ] # then apply autoBagging to the train, using the desired formula: # autoBagging will compute metafeatures on the dataset # and apply a pre-trained ranking model to recommend a workflow. model <- autoBagging(Species ~., train) # predictions are produced with the standard predict method preds <- predict(model, test) ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.