reliability: Marginal reliability coefficient of IRT

In IRTest: Parameter Estimation of Item Response Theory with Estimation of Latent Distribution

Marginal reliability coefficient of IRT

Description

Marginal reliability coefficient of IRT

Usage

reliability(x)

Arguments

x	A model fit object from either IRTest_Dich, IRTest_Poly, IRTest_Cont, or IRTest_Mix.

Details

Reliability coefficient on summed-score scale

In accordance with the concept of reliability in classical test theory (CTT), this function calculates the IRT reliability coefficients.

The basic concept and formula of the reliability coefficient can be expressed as follows (Kim & Feldt, 2010):

An observed score of Item i, X_i, is decomposed as the sum of a true score T_i and an error e_i. Then, with the assumption of \sigma_{T_{i}e_{j}}=\sigma_{e_{i}e_{j}}=0, the reliability coefficient of a test is defined as;

\rho_{TX}=\rho_{XX^{'}}=\frac{\sigma_{T}^{2}}{\sigma_{X}^{2}}=\frac{\sigma_{T}^{2}}{\sigma_{T}^{2}+\sigma_{e}^{2}}=1-\frac{\sigma_{e}^{2}}{\sigma_{X}^{2}}

See May and Nicewander (1994) for the specific formula used in this function.

Reliability coefficient on \theta scale

For the coefficient on the \theta scale, this function calculates the parallel-forms reliability (Green et al., 1984; Kim, 2012):

\rho_{\hat{\theta} \hat{\theta}^{'}} =\frac{\sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}}{\sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}+E\left( \sigma_{\hat{\theta}|\theta}^{2} \right)} =\frac{1}{1+E\left(I\left(\hat{\theta}\right)^{-1}\right)}

This assumes that \sigma_{E\left(\hat{\theta}\mid \theta \right )}^{2}=\sigma_{\theta}^{2}=1. Although the formula is often employed in several IRT studies and applications, the underlying assumption may not be true.

Value

Estimated marginal reliability coefficients.

Author(s)

Seewoo Li [e-mail]

References

Green, B.F., Bock, R.D., Humphreys, L.G., Linn, R.L., & Reckase, M.D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347–360.

Kim, S. (2012). A note on the reliability coefficients for item response model-based ability estimates. Psychometrika, 77(1), 153-162.

Kim, S., Feldt, L.S. (2010). The estimation of the IRT reliability coefficient and its lower and upper bounds, with comparisons to CTT reliability statistics. Asia Pacific Education Review, 11, 179–188.

May, K., Nicewander, A.W. (1994). Reliability and information functions for percentile ranks. Journal of Educational Measurement, 31(4), 313-325.

Examples

data <- DataGeneration(N=500, nitem_D = 10)$data_D

# Analysis

M1 <- IRTest_Dich(data)


# Reliability coefficients
reliability(M1)

IRTest documentation built on Oct. 4, 2024, 5:11 p.m.