correlatedTtest: Correlated t-test

In b0rxa/scmamp: Statistical Comparison of Multiple Algorithms in Multiple Problems

Correlated t-test

Description

This function performs the correlated version of the t-test for paired samples

Usage

correlatedTtest(x, y = NULL, rho, alternative = "two.sided")

Arguments

x	First sample
y	Second sample (if not provided, x is tested aginst 0)
rho	Correlation factor (see details)
alternative	a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter.

Details

This version of the t-test takes into account the correlation between samples. for the particular case of cross validated results, the heuristic to set the correlation is size of test set divided by total size of the dataset.

Value

A list with class "htest" containing the following components: statistic, the value of the statistic used in the test; method, a character string indicating what type of test was performed; data.name, a character string giving the name of the data and diff.matirx, a matrix with all the pairwise absolute difference of average values.

References

C. Nadeau and Y. Bengio (2003) Inference for the Generalization Error. Machine Learning, 52(3), 239-281.

Examples

sample1 <- rnorm(25, 1, 1)
sample2 <- rnorm(25, 1.2, 1)
correlatedTtest (x=sample1, y=sample2, rho=0.1, alternative="less")

b0rxa/scmamp documentation built on Jan. 17, 2024, 10:49 a.m.