corr_t_test: Correlated t test

In RebeccaGroh/seqbtests: Sequential Bayesian Comparison of ML Algorithm Performances

Description

This function implements a two-sided t test for paired samples.

Usage

corr_t_test(
  df,
  problem,
  baseline,
  algorithm = NULL,
  measure = NULL,
  rho = 0.01
)

Arguments

df	('list') Data frame containing the performane measure.
problem	('character') Problem set used to evaluate the algorithms performance. Value in 'problem' column.
baseline	('character') First algorithm. Value in 'algorithm' column.
algorithm	('character') Second algorithm. Value in 'algorithm' column. If not defined, the baseline is tested against all algorithms in the data frame.
measure	('character') Name of the 'measure' column. If not defined, the first 'measure' column in the data frame is used.
rho	('double') Correlation factor. Default is 0.1. (For the case of cross validated results, the heuristic to set the correlation is size of test set divided by total size of the dataset.)

Details

The test has first been implemented in scmamp.

Value

A list containing the following components:

• measure ('character')
  A string with the name of the measure column.

• method ('character')
  A string with the name of the method.

• statistic ('double')
  The value of the statistic used in the test.

• p_value ('double')
  The p-value for the test.

References

https://github.com/b0rxa/scmamp

Examples

    results <- corr_t_test(df= test_benchmark_small, 
    problem = "problem_a", baseline = "algo_1", algorithm = "algo_2")

RebeccaGroh/seqbtests documentation built on Nov. 17, 2021, 8:50 a.m.