Challenge: Machine Learning Challenge.

In ja-thomas/mlComp: Machine Learning competitions and challenges in R

Description

Create a Machine Learning challenge objects that should be solved in a given time limit. Currently only classification tasks with accuracy as performance are supported.

Usage

Challenge

Format

R6Class object.

Usage

chall = Challenge$new(id = 3, difficulty = "easy", time.limit = 3600)
lrn = makeLearner("classif.rpart")
chall$submit(lrn)

Arguments

For Challenge$new():

id

[integer(1) | NULL | "daily"]
OpenML task.id used as challenge task. If NULL a random task from the OpenML-CC18 benchmarking suit is sampled. A daily challenge can be selected by setting id to "daily".

difficulty

[character(1) | numeric(1)]
Define the difficulty of the Challenge. very easy is the 0.5 quantile of runs submitted to OpenML, easy the 0.6, medium the 0.8, hard the 0.9 and very hard the 0.99 quantile. Alternatively the quantile can be directly specified as a numer value between 0 and 1. Default is easy.

time.limit

[numeric(1)]
Time limit, starting from the creation of the Challenge object until it is solved. Default is 1800, i.e., half an hour.

For Challenge$submit():

lrn

[Learner]
Learner submitted to solved the challenge. Preprocessing on the task can be done with mlrCPO.

Fields

id [integer(1)]

id of OpenML task used in challenge.

difficulty [numeric(1)]

Difficulty of the challenge.

goal [numeric(1)]

Threshold value to beat with Challenge$submit(lrn).

task [Task]

The Data for the challenge, useful for exploratory analysis.

time.limit

[numeric(1)]
See above.

Examples

chall = Challenge$new(id = 3, difficulty = "easy", time.limit = 3600)
lrn = makeLearner("classif.rpart")
chall$submit(lrn)

ja-thomas/mlComp documentation built on May 31, 2019, 7:41 a.m.