multipleComparisonTest: Tests for multiple comparisons
In scmamp: Statistical Comparison of Multiple Algorithms in Multiple Problems

Description Usage Arguments Value References Examples

This function is a wrapper to multiple comparison tests.

1 2	multipleComparisonTest(data, algorithms = NULL, group.by = NULL, test = "aligned ranks", correct = "finner", alpha = 0.05, ...)

`data`	A matrix or data frame containing the results obtained by the algorithms (columns) in each problem (rows). It can contain additional columns, but if any of the column has to be discarderd (not used neither to group the problems nor to be part of the comparison), then it is mandatory to indicate, in the `algorithms` parameter, which columns contain the algorithm information.
`algorithms`	Vector with either the names or the indices of the columns that contain the values to be tested. If not provided, the function assumes that all the columns except those indicated in `group.by` represent the results obtained by an algorithm.
`group.by`	Vector with either the names or the indices of the columns to be used to group the data. Each group is tested independently. If `NULL`, all the data is used for a single comparison.
`test`	Parameter that indicates the statistical test to be used. It can be either a string indicating one of the available test or a function. As a string, it can take the following values: `'friedman'` - Friedman test, as in Garcia and Herrera (2010) `'aligned ranks'` Friedman's Aligned Ranks test, as in Garcia and Herrera (2010) `'quade'` - Quade test, as in Garcia and Herrera (2010) `'anova'` - ANOVA test, as in Test 22 in Kanji (2006). If a function is provided, then it has to have as first argument a matrix containing the columns to be compared. The function has to return a list with, at least, an element named `p.value` (as the `htest` objects that are usually returned by R's test implementations).
`correct`	Either string indicating the type of correction that has to be applied or a function to correct the p-values for multiple testing; This parameter is only need in case the data is grouped. As a string, the valid values are: `holland` - Holland's procedure, as in Garcia and Herrera (2010) `finner` - Finner's procedure, as in Garcia and Herrera (2010) `rom` - Rom's procedure, as in Garcia and Herrera (2010) `li` - Li's procedure, as in Garcia and Herrera (2010) Any of the methods implemented in the `p.adjust` function. For a list of options, type `p.adjust.methods` . If a function is provided, the it has to recieve, as first argument, a vector of pvalues to be corrected and has to return a verctor with the corrected p-values in the same order as the input vector.
`alpha`	Alpha value used in Rom's correction. By default, set at 0.05.
`...`	Special argument used to pass additional parameters to the statistical test and the correction method.

In case the group.by argument is not provided (or it is NULL), the function return an object of class htest. If columns for grouping are provided, then the function returns a matrix that includes, for each group, the values of the group.by columns, the raw p-value and the corrected p-value.

#' @seealso friedmanTest, friedmanAlignedRanksTest, quadeTest, anovaTest, adjustShaffer, adjustBergmannHommel, adjustHolland, adjustFinner, adjustRom, adjustLi

S. Garcia and F. Herrera (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and ata mining: Experimental analysis of power. Information Sciences, 180, 2044-2064.

Kanji, G. K. (2006) 100 Statistical Tests. SAGE Publications Ltd, 3rd edition.

# Grouped data
data(data_blum_2015)
multipleComparisonTest (data=data.blum.2015, 
                        algorithms=c("FrogCOL", "FrogMIS", "FruitFly"), 
                        group.by=c("Size", "Radius"), 
                        test="quade", correct="finner")
# Not grouped data
data(data_gh_2008)
multipleComparisonTest (data=data.gh.2008, test="aligned ranks", 
                        correct="hochberg")