results_2023_sample.md
In rtransparency: Identifies Indicators of Transparency

Independent 2023 OA PMC hand-labeled sample

Package version 0.9.9. 1000 open-access PMC articles published in 2023, sampled and hand-labeled for all eight transparency indicators. This is a modern, independent companion to the Serghiou et al. (2021) held-out set (which predates these indicators and the 2023-era reporting conventions).

Methods note. Conflicts of interest, funding and data labels were reconciled against the detector's extracted statement where the author's back matter was truncated in the labelling view, so those three are not independent of the detector and their agreement is near ceiling by construction. Novelty, replication, registration and code sharing were labelled independently and are the meaningful validation.

| Indicator | Labels | n | pos | Sens | Spec | PPV | Acc | |---|---|---|---|---|---|---|---| | coi | detector-adjudicated | 1000 | 927 | 100.0 | 91.8 | 99.4 | 99.4 | | fund | detector-adjudicated | 1000 | 537 | 95.0 | 95.2 | 95.9 | 95.1 | | reg | independent | 1000 | 52 | 84.6 | 99.2 | 84.6 | 98.4 | | nov | independent | 1000 | 205 | 90.2 | 93.3 | 77.7 | 92.7 | | rep | independent | 1000 | 17 | 82.4 | 98.5 | 48.3 | 98.2 | | data | detector-adjudicated | 1000 | 123 | 91.1 | 97.8 | 85.5 | 97.0 | | code | independent | 1000 | 33 | 93.9 | 99.0 | 75.6 | 98.8 | | ai | detector-adjudicated | 986 | 9 | 100.0 | 100.0 | 100.0 | 100.0 |