In wiesenfa/challengeR: Analyzing assessment data of biomedical image analysis competitions and visualization of results

\newpage

Visualization of cross-task insights

The algorithms are ordered according to consensus ranking.

Characterization of algorithms

Ranking stability: Variability of achieved rankings across tasks

Algorithms are color-coded, and the area of each blob at position $\left( A_i, \text{rank } j \right)$ is proportional to the relative frequency $A_i$ achieved rank $j$ across multiple tasks. The median rank for each algorithm is indicated by a black cross. This way, the distribution of ranks across tasks can be intuitively visualized.

\bigskip

#stability.ranked.list
stability(object,ordering=ordering_consensus,max_size=9,size=8,shape=4)+
  scale_color_manual(values=cols) +
  guides(color = 'none')

\newpage

Characterization of tasks

Cluster Analysis

Dendrogram from hierarchical cluster analysis and \textit{network-type graphs} for assessing the similarity of tasks based on challenge rankings.

A dendrogram is a visualization approach based on hierarchical clustering. It depicts clusters according to a chosen distance measure (here: Spearman's footrule) as well as a chosen agglomeration method (here: complete and average agglomeration). \bigskip

if (n.tasks>2) {
  dendrogram(object,
             dist = "symdiff",
             method="complete")
} else cat("\nCluster analysis only sensible if there are >2 tasks.\n\n")

\bigskip

if (n.tasks>2)   
  dendrogram(object,
             dist = "symdiff",
             method="average")

wiesenfa/challengeR documentation built on Aug. 25, 2023, 6:43 a.m.