Description Usage Arguments Details Value Author(s) References Examples
reliability with progressive rater inclusion
1 2 3 4 | raterprog(winner, loser, raterID, runs=100, ratershuffle=1, progbar=TRUE, kval=100,
startvalue=0, normprob=FALSE)
raterprogplot(xdata)
|
winner |
character, vector with the IDs of the winning (preferred) stimuli |
loser |
character, vector with the IDs of the losing (not preferred) stimuli |
raterID |
a vector (numeric, character, factor) with rater IDs |
runs |
numeric, number of randomizations |
ratershuffle |
numeric, number of times rater order is reshuffled/randomized |
progbar |
logical, should a progress bar be displayed |
kval |
numeric, k-value, which determines the maximum number of points a stimulus' rating can change after a single rating event, by default 100 |
startvalue |
numeric, start value around which ratings are centered, by default |
normprob |
logical, by default |
xdata |
results from |
raterprog()
calculates reliability
, increasing the number of raters to be included in the rating process in a step-wise fashion. In the first (and by default only one) run, the first rater is the one that appears first in the data set, and in subsequent steps raters are added by the order in which they occur. If ratershuffle=
is set to values larger than 1, the order in which raters are included is randomized.
raterprogplot()
plots the matrix resulting from raterprog()
. If ratershuffle=
is larger than 1, the average reliability index is plotted alongside quartiles and results from the original rater inclusion sequence.
Note that the function currently only calculates the weighted version of the reliability
index.
a numeric matrix. Rows correspond to number of raters in the data set, while columns reflect the number of times the rater order is reshuffled.
Christof Neumann after suggestion by TF
clark2018EloChoice
1 2 3 4 5 6 7 8 9 10 | data("physical")
# limit to 12 raters
physical <- physical[physical$raterID < 14, ]
x <- raterprog(physical$Winner, physical$Loser, physical$raterID, ratershuffle = 1)
raterprogplot(x)
# with multiple orders in which raters are added
x <- raterprog(physical$Winner, physical$Loser, physical$raterID, ratershuffle = 10)
raterprogplot(x)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.