glicko2_ranking: Glicko2 raning of algorithms

Description Usage Arguments Value Examples

Description

This procedure ranks algorithms based on a glicko2-procedure. Every round (total nr_rounds), for every function and dimension of the datasetlist, each pair of algorithms competes. This competition samples a random runtime for the provided target (defaults to best achieved target). Whichever algorithm has the lower runtime wins the game. Then, from these games, the glicko2-rating is determined.

Usage

1
2
glicko2_ranking(dsl, nr_rounds = 100, which = "by_FV",
  target_dt = NULL)

Arguments

dsl

The DataSetList, can contain multiple functions and dimensions, but should have the same algorithms for all of them

nr_rounds

The number of rounds to run. More rounds leads to a more accurate ranking.

which

Whether to use fixed-target ('by_FV') or fixed-budget ('by_RT') perspective

target_dt

Custom data.table target value to use. When NULL, this is selected automatically.

Value

A dataframe containing the glicko2-ratings and some additional info

Examples

1
2
glicko2_ranking(dsl, nr_round = 25)
glicko2_ranking(dsl, nr_round = 25, which = 'by_RT')

IOHprofiler/IOHanalyzer documentation built on Aug. 8, 2020, 2:09 a.m.