perf_wilcox: Compute column-wise statistics in a performance matrix
In b2slab/diffuStats: Diffusion scores on biological networks

Description Usage Arguments Details Value Examples

Function perf_wilcox compares all the columns of a matrix through a wilcox.test. The columns are assumed to be performance measures (e.g. AUROC) whereas the rows are instances.

perf_wilcox(
    perf_mat,
    adjust = function(p) stats::p.adjust(p, method = "fdr"),
    ci = 0.95,
    digits_ci = 2,
    digits_p = 3,
    ...
)

`perf_mat`	Numeric matrix whose columns contain performance metrics of different methods.
`adjust`	Function to adjust the p-values for multiple testing. By default, `p.adjust` with its default parameters is used.
`ci`	Numeric, confidence interval (defaults to `0.95`)
`digits_ci`	Integer, digits to display in the confidence interval
`digits_p`	Integer, digits to display in the p-value
`...`	further arguments for `format`

The statistical comparison of the columns is intended to ease comparisons between methods in a rigorous way. Methods are compared pairwise and a p-value for difference in performance. The function perf_wilcox returns a character matrix so that (1) the upper triangular matrix contains confidence intervals on the estimate of the difference between performances, and (2) the lower triangular matrix contains the two-tailed p-value that tests difference in performance, with multiple testing correction. The comparison takes place between row and column in that precise order: a positive difference favours the row and a negative one, the column.

Character matrix. The upper triangular matrix contains a confidence interval and the estimate of the pairwise difference in performance. The lower triangular matrix shows the associated two-tailed p-value, with multiple testing correction.

# Dummy data frame to test 
n <- 100
perf_mat <- cbind(
    good = runif(n = n, min = 0.5, max = 1), 
    so_so = runif(n = n, min = 0.2, max = 0.7), 
    bad = runif(n = n, min = 0, max = 0.5)
)
wilcox_mat <- perf_wilcox(perf_mat)

# See how the methods in the rows compare to those
# in the columns, confidence interval 
# (upper) and p-value (lower)
wilcox_mat