Description Usage Arguments Value Note See Also

Computes the ranks of all the algorithms from their (repeated) results measurements after grouping them by several factors combined simultaneosly.

1 2 3 4 |

`data` |
A dataframe object containing (at least) two columns for the target factor and the performance measure Additional columns are aimed at grouping the problem configuration by (at most) 3 different factors. |

`params` |
A vector with the column names in |

`target` |
Name of the target column of |

`performance` |
Name of the column of |

`pairing.col` |
Name of the column which links together the paired samples, in case we have set |

`test` |
The statistical test to be performed to compare the performance of every level of the target variable at each problem configuration. |

`fun` |
Function performing a custom statistical test, if |

`correction` |
The p-value adjust method. Must be one of "holm", "hochberg", "hommel", "bonferroni", "BH", "BY", "fdr", "none" (defaults to "holm").
This parameter will be ignored if |

`alpha` |
Significance threshold for pairwise comparisons. Defaults to 0.05. |

`maximize` |
Boolean indicating whether the higher the performance measure, the better (default), or vice-versa. |

`ncores` |
Number of physical CPUs available for computations. If |

`paired` |
Boolean indicating whether samples in the same problem configuration, which only differ in the target value, and in the same relative position (row) within their respective target values are paired or not. Defaults to FALSE. This should be set to TRUE, for instance, in Machine Learning problems in which, for a fixed problem configuration, the target variable (usually the algorithms being compared) is associated to a number of samples (results) coming from the Cross Validation process. If a K-fold CV is being done, then we would have, for a given problem configuration, K rows for each of the algorithms being compared, all of them identical in all the columns except for the performance column. In that case, the performance of the i-th row (1 <= i <= K) of all of those batches (groups of K rows) for that fixed problem configuration would be related, hence every pairwise comparison should take into account paired samples. |

`...` |
Further arguments to be passed to the function |

If `length(performance)`

equals 1, an object of classes `c("SRCS", "data.frame")`

with the following columns:
- A set of columns with the same names as the `params`

and `target`

arguments.
- Two columns called "mean" and "sd" containing the mean of the repeated peformance measurements for each problem configuration and the standard deviation.
- One column named "rank" with the actual rank of each level of the `target`

variable within that problem configuration. The lower the rank, the better the algorithm.
- |target| additional columns containing the p-values resulting of the comparison between the algorithm and the rest for the same problem configuration,
where |target| is the number of levels of the target variable.

If `length(performance)`

> 1 (let `P = length(performance)`

for the explanation that follows), an object of classes `c("SRCS","data.frame")`

with the following columns:
- A set of columns with the same names as the `params`

and `target`

arguments.
- One column per element of the `performance`

vector, named "rank1", ..., "rankP", containing, for each performance measure,
the rank of each level of the `target`

variable within that problem configuration for that performance measure.
The higher the rank, the better the algorithm.

Although it has no effect on the results of `SRCSranks`

, the user should preferably have set the order
of the factor levels explicitly by calling function `levels`

before calling this function, specially if he intends to subsequently apply `plot`

to the results,
because the level order does affect the way graphics are arranged in the plot.

`plot.SRCS`

for a full working example of `SRCSranks`

and plotting facilities. Also
`pairwise.wilcox.test`

, `t.test`

, `pairwise.t.test`

, `TukeyHSD`

, `p.adjust.methods`

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.