glicko2_ranking: Glicko2 raning of algorithms

In IOHprofiler/IOHanalyzer: Data Analysis Part of 'IOHprofiler'

Glicko2 raning of algorithms

Description

This procedure ranks algorithms based on a glicko2-procedure. Every round (total nr_rounds), for every function and dimension of the datasetlist, each pair of algorithms competes. This competition samples a random runtime for the provided target (defaults to best achieved target). Whichever algorithm has the lower runtime wins the game. Then, from these games, the glicko2-rating is determined.

Usage

glicko2_ranking(dsl, nr_rounds = 100, which = "by_FV", target_dt = NULL)

Arguments

dsl	The DataSetList, can contain multiple functions and dimensions, but should have the same algorithms for all of them
nr_rounds	The number of rounds to run. More rounds leads to a more accurate ranking.
which	Whether to use fixed-target ('by_FV') or fixed-budget ('by_RT') perspective
target_dt	Custom data.table target value to use. When NULL, this is selected automatically.

Value

A dataframe containing the glicko2-ratings and some additional info

Examples

glicko2_ranking(dsl, nr_round = 25)
glicko2_ranking(dsl, nr_round = 25, which = 'by_RT')

IOHprofiler/IOHanalyzer documentation built on Feb. 1, 2024, 11:35 a.m.