fullDist: Full distribution of ability estimator (dichotomous models...

In catR: Generation of IRT Response Patterns under Computerized Adaptive Testing

Full distribution of ability estimator (dichotomous models only)

Description

This command returns the full distribution of the selected ability estimator, that is, the set of all possible ability estimates and related probabilities, for a given matrix of item parameters and ability estimate (or true value).

Usage

fullDist(th, it, method = "BM", priorDist = "norm", priorPar = c(0,1), 
 	weight = "Huber", tuCo  = 1, range = c(-4 ,4), parInt = c(-4, 4, 33))
 

Arguments

th	numeric: the ability estimate of interest (can be a vector of estimates too).
it	numeric: a suitable matrix of item parameters. See Details.
method	character: the ability estimator. Possible values are "BM" (default), "ML", "WL", "EAP" and "ROB". See Details.
priorDist	character: specifies the prior distribution. Possible values are "norm" (default), "unif" and "Jeffreys". Ignored if method is neither "BM" nor "EAP". See Details.
priorPar	numeric: vector of two components specifying the prior parameters (default is c(0,1)) of the prior ability distribution. Ignored if method is neither "BM" nor "EAP", or if priorDist="Jeffreys". See Details.
weight	character: the type of weight function for the robust estimator. Possible values are "Huber" (default) and "Tukey". Ignored if method is not "ROB" or if model is not NULL. See Details.
tuCo	numeric: the value of the tuning constant for the weight function (default is 1, suitable with "Huber" weight). Ignored if method is not "ROB" or if model is not NULL. See Details.
range	numeric: vector of two components specifying the range wherein the ability estimate must be looked for (default is c(-4,4)). Ignored if method=="EAP".
parInt	numeric: vector of three components, holding respectively the values of the arguments lower, upper and nqp of the eapEst command. Default vector is (-4, 4, 33). Ignored if method is not "EAP".

Details

The computation of the full distribution of an ability estimator is required to determine the exact standard error of ability, compared to the asympotic SE (ASE) displayed by former versions of the semTheta function. This distribution is computed as follows.

First, all possible response patterns (for the given test length fixed by the number of items specified by it argument) are generated through the internal dataGen function. Second, ability estimation is performed with each generated pattern, and the corresponding pattern probability is computed using the set of item parameters it and the predefined ability level th of interest. These to components are eventually returned as the full distribution.

In case of the Rasch (1PL) model, this long process can be shortened as the total test score is a proxy for ability estimation. Hence, with n items, only (n+1) patterns must be created, one for each test score from zero to n. Related probabilities can be derived using the Lord-Wingersky algorithm (1984) that is implemented internally through the LW() function.

Dichotomous IRT models are considered whenever model is set to NULL (default value). In this case, it must be a matrix with one row per item and four columns, with the values of the discrimination, the difficulty, the pseudo-guessing and the inattention parameters (in this order). These are the parameters of the four-parameter logistic (4PL) model (Barton and Lord, 1981).

Five ability estimators are available: the maximum likelihood (ML) estimator (Lord, 1980), the Bayes modal (BM) estimator (Birnbaum, 1969), the expected a posteriori (EAP) estimator (Bock and Mislevy, 1982), the weighted likelihood (WL) estimator (Warm, 1989) and the robust estimator (Schuster & Yuan, 2011). The selected estimator is specified by the method argument, with values "ML", "BM", "EAP", "WL" and "ROB" respectively.

For the BM and EAP estimators, three prior distributions are available: the normal distribution, the uniform distribution and the Jeffreys' prior distribution (Jeffreys, 1939, 1946). The prior distribution is specified by the argument priorPar, with values "norm", "unif" and "Jeffreys", respectively. The priorPar argument is ignored if method="ML" or method="WL".

The argument priorPar determines either: the prior mean and standard deviation of the normal prior distribution (if priorDist="norm"), or the range for defining the prior uniform distribution (if priorDist="unif"). This argument is ignored if priorDist="Jeffreys".

The eapPar argument sets the range and the number of quadrature points for numerical integration in the EAP process. By default, it takes the vector value (-4, 4, 33), that is, 33 quadrature points on the range [-4; 4] (or, by steps of 0.25). See eapEst for further details.

Robust estimation requires an appropriate weight function that depends on an accurate tuning constant. Suggested functions are the Huber weight (Schuester and Yuan, 2011) and the Tukey weight (Mosteller and Tukey, 1977). Both can be set by the weight argument, with respective values "Huber" and "Tukey". Default function is Huber. Moreover, the tuCo argument specifies the tuning constant for the weight function. Default value is 1 and suggested for Huber weight (also by default), and value 4 is suggested for Tukey weight (Schuester and Yuan, 2011).

The ability level of interest (that is, for which the probability distribution must be computed) is specified by theth argument. Note that it can hold a vector of ability levels too; in this case, the probability distribution is computed for each component of th.

Value

A matrix with (t+1) columns (where t is the length of vector th), the first column holding all observable ability estimates and columns 2 to (t+1) with related probabilities of the corresponding components of th.

Note

Under the Rasch (1PL) model, the computation of the full distribution is very efficient even for a large test: due to the Lord-Wingersky algorithm, only (n+1) patterns (where n is the number of items) must be created, one for each possible test score. But for other models, 2^n patterns must be generated and used for ability estimation, which can become computationally intensive with long tests. Therefore it is recommended not to make use of this function with more than ten items (except under the Rasch model).

Author(s)

David Magis
Department of Psychology, University of Liege, Belgium
[e-mail]

Lianne Ippel
Department of Psychology, University of Liege, Belgium
[e-mail]

References

Barton, M.A., and Lord, F.M. (1981). An upper asymptote for the three-parameter logistic item-response model. Research Bulletin 81-20. Princeton, NJ: Educational Testing Service.

Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258-276. doi: 10.1016/0022-2496(69)90005-4

Bock, R. D., and Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444. doi: 10.1177/014662168200600405

Haley, D.C. (1952). Estimation of the dosage mortality relationship when the dose is subject to error. Technical report no 15. Palo Alto, CA: Applied Mathematics and Statistics Laboratory, Stanford University.

Jeffreys, H. (1939). Theory of probability. Oxford, UK: Oxford University Press.

Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 186, 453-461.

Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

Lord, F. M., and Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score equatings. Applied Psychological Measurement, 8, 453-461. doi: 10.1177/014662168400800409

Magis, D. and Barrada, J. R. (2017). Computerized Adaptive Testing with R: Recent Updates of the Package catR. Journal of Statistical Software, Code Snippets, 76(1), 1-18. doi: 10.18637/jss.v076.c01

Magis, D., and Raiche, G. (2012). Random Generation of Response Patterns under Computerized Adaptive Testing with the R Package catR. Journal of Statistical Software, 48 (8), 1-31. doi: 10.18637/jss.v048.i08

Mosteller, F., and Tukey, J. (1977). Exploratory data analysis and regression. Reading, MA: Addison-Wesley.

Schuester, C., and Yuan, K.-H. (2011). Robust estimation of latent ability in item response models. Journal of Educational and Behavioral Statistics, 36, 720)735.doi: 10.3102/1076998610396890

Warm, T.A. (1989). Weighted likelihood estimation of ability in item response models. Psychometrika, 54, 427-450. doi: 10.1007/BF02294627

See Also

semTheta, thetaEst,

Examples

## Dichotomous models ##

 # Generation of ten items under 1PL model
 it <- genDichoMatrix(10, model = "1PL")
 
 # Full distribution of ML estimator for ability level zero
 fullDist(0, it, method = "ML")

 # Idem with BM estimator (probabilities don't change, only estimated abilities)
 fullDist(0, it, method = "BM")

 # Idem with ability level 1 (only probabilities change)
 fullDist(1, it, method = "BM")

 # Distributions with two ability levels 1 and 0.5
 fullDist(c(1, 0.5), it, method = "BM")

 # Generation of ten items under 2PL model
 it2 <- genDichoMatrix(10, model = "2PL")
 
 # Full distribution of ML estimator for ability level zero
 fullDist(0, it2, method = "ML")
 

catR documentation built on June 24, 2022, 9:06 a.m.