generateCritDifferencesData: Generate data for critical-differences plot.

Description Usage Arguments Value See Also

View source: R/plotCritDifferences.R


Generates data that can be used to plot a critical differences plot. Computes the critical differences according to either the "Bonferroni-Dunn" test or the "Nemenyi" test.
"Bonferroni-Dunn" usually yields higher power as it does not compare all algorithms to each other, but all algorithms to a baseline instead.
Learners are drawn on the y-axis according to their average rank.
For test = "nemenyi" a bar is drawn, connecting all groups of not significantly different learners.
For test = "bd" an interval is drawn arround the algorithm selected as baseline. All learners within this interval are not signifcantly different from the baseline.

CD = q_alpha sqrt(k(k+1)/(6N))

Where q_α is based on the studentized range statistic. See references for details.


generateCritDifferencesData(bmr, measure = NULL, p.value = 0.05,
  baseline = NULL, test = "bd")



Benchmark result.


Performance measure. Default is the first measure used in the benchmark experiment.


P-value for the critical difference. Default: 0.05


[character(1)]: []
Select a as baseline for the test = "bd" ("Bonferroni-Dunn") critical differences diagram.The critical difference Interval will then be positioned arround this learner. Defaults to best performing algorithm.
For test = "nemenyi", no baseline is needed as it performs all pairwise comparisons.


Test for which the critical differences are computed.
“bd” for the Bonferroni-Dunn Test, which is comparing all classifiers to a baseline, thus performing a comparison of one classifier to all others.
Algorithms not connected by a single line are statistically different. then the baseline.
“nemenyi” for the posthoc.friedman.nemenyi.test which is comparing all classifiers to each other. The null hypothesis that there is a difference between the classifiers can not be rejected for all classifiers that have a single grey bar connecting them.


[critDifferencesData]. List containing:


[data.frame] containing the info for the descriptive part of the plot


[list] of class pairwise.htest
contains the calculated posthoc.friedman.nemenyi.test

[list] containing info on the critical difference and its positioning


baseline chosen for plotting


p.value used for the posthoc.friedman.nemenyi.test and for computation of the critical difference

See Also

Other generate_plot_data: generateCalibrationData, generateFeatureImportanceData, generateFilterValuesData, generateFunctionalANOVAData, generateLearningCurveData, generatePartialDependenceData, generateThreshVsPerfData, getFilterValues, plotFilterValues

Other benchmark: BenchmarkResult, batchmark, benchmark, convertBMRToRankMatrix, friedmanPostHocTestBMR, friedmanTestBMR, getBMRAggrPerformances, getBMRFeatSelResults, getBMRFilteredFeatures, getBMRLearnerIds, getBMRLearnerShortNames, getBMRLearners, getBMRMeasureIds, getBMRMeasures, getBMRModels, getBMRPerformances, getBMRPredictions, getBMRTaskDescs, getBMRTaskIds, getBMRTuneResults, plotBMRBoxplots, plotBMRRanksAsBarChart, plotBMRSummary, plotCritDifferences, reduceBatchmarkResults

guillermozbta/s2 documentation built on Jan. 2, 2018, 12:25 a.m.