Description Usage Arguments Details Value Examples
Function perf_wilcox
compares all the columns of a matrix
through a wilcox.test
.
The columns are assumed to be performance measures (e.g. AUROC)
whereas the rows are instances.
1 2 3 4 5 6 7 8 | perf_wilcox(
perf_mat,
adjust = function(p) stats::p.adjust(p, method = "fdr"),
ci = 0.95,
digits_ci = 2,
digits_p = 3,
...
)
|
perf_mat |
Numeric matrix whose columns contain performance metrics of different methods. |
adjust |
Function to adjust the p-values for multiple testing.
By default, |
ci |
Numeric, confidence interval (defaults to |
digits_ci |
Integer, digits to display in the confidence interval |
digits_p |
Integer, digits to display in the p-value |
... |
further arguments for |
The statistical comparison of the columns is intended to
ease comparisons between methods in a rigorous way.
Methods are compared pairwise and a p-value for difference in
performance.
The function perf_wilcox
returns a character matrix so that
(1) the upper triangular matrix contains confidence intervals
on the estimate of the difference between performances, and
(2) the lower triangular matrix contains the two-tailed p-value
that tests difference in performance, with multiple testing correction.
The comparison takes place between row and column in that precise order:
a positive difference favours the row and a negative one, the column.
Character matrix. The upper triangular matrix contains a confidence interval and the estimate of the pairwise difference in performance. The lower triangular matrix shows the associated two-tailed p-value, with multiple testing correction.
1 2 3 4 5 6 7 8 9 10 11 12 13 | # Dummy data frame to test
n <- 100
perf_mat <- cbind(
good = runif(n = n, min = 0.5, max = 1),
so_so = runif(n = n, min = 0.2, max = 0.7),
bad = runif(n = n, min = 0, max = 0.5)
)
wilcox_mat <- perf_wilcox(perf_mat)
# See how the methods in the rows compare to those
# in the columns, confidence interval
# (upper) and p-value (lower)
wilcox_mat
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.