get_dsc_rank: Get the matrix of rankings using the DSCtool api for a...

In IOHprofiler/IOHanalyzer: Data Analysis Part of 'IOHprofiler'

Get the matrix of rankings using the DSCtool api for a DataSetList

Description

Get the matrix of rankings using the DSCtool api for a DataSetList

Usage

get_dsc_rank(
  dsList,
  targets = NULL,
  which = "by_RT",
  test_type = "AD",
  alpha = 0.05,
  epsilon = 0,
  monte_carlo_iterations = 0,
  na.correction = NULL
)

Arguments

dsList	The DataSetList object
targets	Optional list of target values (Runtime or target value)
which	Whether to use a fixed-target 'by_RT' perspective or fixed-budget 'by_FV'
test_type	Either 'AD' for Anderson-Darling or KS for Kolmogorov-Smirnov tests
alpha	Threshold value for statistical significance
epsilon	Minimum threshold to have practical difference between algorithms (eDSC)
monte_carlo_iterations	How many monte-carlo-simulations to perform (set to 0 to use regular DSC)
na.correction	How to deal with missing values. Only used in fixed-target perspective. Options are: - 'NULL': No correction is done. This will likely result in an error, as the DSCtool does not allow for na values - 'PAR-1' Replace missing values with Budget (budget taken from relevant DataSet) - 'PAR-10' Replace missing values with 10*Budget (budget taken from relevant DataSet) - 'ERT' Replace NA values with the Expected Running Time. If all values are NA, this reverts to nr_runs * budget - 'Remove-na' Removes all NA values

Value

A named list containing a ranked-matrix which has the rankin of each algorithm on each problem, as well as a list of which omnibus tests can be used to further process this data. This can be further analyzed using 'get_dsc_omnibus'

Examples

get_dsc_rank(dsl, na.correction = 'PAR-10')

IOHprofiler/IOHanalyzer documentation built on Feb. 1, 2024, 11:35 a.m.