Computes classification accuracy and consistency using Lathrop and Cheng's (2014) nonparametric approach.
Description
Computes classification accuracy and consistency with Lathrop & Cheng's (2014) approach. First, the kernelsmoothed estimate of the probability of a correct response, conditional on observed total score, is found with pnr()
. Then, the method proceeds similar to class.Lee()
. Using the nonparametric approach does not require a parametric IRT model, keeps the problem on the total score scale, and can produce more accurate CA and CC estimates when the IRT model's assumptions are violated (see Lathrop & Cheng, 2014).
Usage
1 2 
Arguments
cutscore 
A scalar or vector of cut scores on the total score scale. Should not include 0 or the max total score, as the end points are added internally. 
pnr.out 
The output from

resp 
The response data matrix with rows as subjects and columns as items. Because the method is based on total score, the method is not robust to missing data. Any 
bw.g 
The global bandwidth parameter. The default of NULL will estimate the global bandwidth with the optimal (in terms of MSE) estimate of the bandwidth for normally distributed variables. The default is generally a good starting point. 
alpha 
The adaptivity of the bandwidth parameter. A value of 0 means no adaptation and each evaluation point uses the value in 
Value
Marginal 
A matrix with two columns of marginal accuracy and consistency per cut score (and simultaneous if multiple cutscores are given) 
Conditional 
A list of two matrixes, one for conditional accuracy and one for conditional consistency. Each matrix has one row per evaluation point. 
Note
The function pnr()
is modified from Ramsay's (1991) kernelsmoothed response functions, specifically because they occur conditional total score (and not conditional on a latent trait) and the addition of an adaptive bandwidth (which helps performance when the distribution of total scores is not normal.)
There is no "D" method of marginalization (as there is for class.Rud
and class.Lee
). But if there is a theoretical distribution of total scores, the pnr.out[[2]]
can be adjusted to match this theoretical distribution.
Author(s)
Quinn N. Lathrop
References
Lathrop, Q. N., & Cheng, Y. (2014). A Nonparametric Approach to Estimate Classification Accuracy and Consistency. Journal of Educational Measurement, 51(3), 318334.
Lee, W. (2010) Classification consistency and accuracy for complex assessments using item response theory. Journal of Educational Measurement, 47, 117.
Ramsay, J. O. (1991). Kernel Smoothing Approaches to Item Characteristic Curve Estimation. Psychometrika, 56(4), 611630.
Examples
1 2 3 4 5 6 7 8 9 