quallmer.app: Interactive Validation App for 'quallmer'

Companion package to 'quallmer' providing an interactive 'shiny' application for manual coding, reviewing large language model (LLM) generated annotations, and computing inter-rater reliability metrics. Supports three modes: blind manual coding, LLM output validation, and agreement calculation. Computes standard reliability metrics including Krippendorff's alpha (Krippendorff 2019 <doi:10.4135/9781071878781>), Cohen's kappa, Fleiss' kappa (Fleiss 1971 <doi:10.1037/h0031619>), intraclass correlation coefficient (ICC), and percent agreement for nominal, ordinal, interval, and ratio data. Also computes gold-standard validation metrics including accuracy, precision, recall, and F1 scores following Sokolova and Lapalme (2009 <doi:10.1016/j.ipm.2009.03.002>).

install.packages("quallmer.app")

Try the quallmer.app package in your browser