get_dsc_rank: Get the matrix of rankings using the DSCtool api for a...

Description Usage Arguments Value Examples

Description

Get the matrix of rankings using the DSCtool api for a DataSetList

Usage

1
2
3
get_dsc_rank(dsList, targets = NULL, which = "by_RT",
  test_type = "AD", alpha = 0.05, epsilon = 0,
  monte_carlo_iterations = 0, na.correction = NULL)

Arguments

dsList

The DataSetList object

targets

Optional list of target values (Runtime or target value)

which

Whether to use a fixed-target 'by_RT' perspective or fixed-budget 'by_FV'

test_type

Either 'AD' for Anderson-Darling or KS for Kolmogorov-Smirnov tests

alpha

Threshold value for statistical significance

epsilon

Minimum threshold to have practical difference between algorithms (eDSC)

monte_carlo_iterations

How many monte-carlo-simulations to perform (set to 0 to use regular DSC)

na.correction

How to deal with missing values. Only used in fixed-target perspective. Options are: - 'NULL': No correction is done. This will likely result in an error, as the DSCtool does not allow for na values - 'PAR-1' Replace missing values with Budget (budget taken from relevant DataSet) - 'PAR-10' Replace missing values with 10*Budget (budget taken from relevant DataSet) - 'ERT' Replace NA values with the Expected Running Time. If all values are NA, this reverts to nr_runs * budget - 'Remove-na' Removes all NA values

Value

A named list containing a ranked-matrix which has the rankin of each algorithm on each problem, as well as a list of which omnibus tests can be used to further process this data. This can be further analyzed using 'get_dsc_omnibus'

Examples

1

IOHanalyzer documentation built on Oct. 21, 2021, 5:06 p.m.