tam.modelfit | R Documentation |
The function tam.modelfit
computes several model fit statistics.
It includes the Q3 statistic (Yen, 1984) and an
adjusted variant of it (see Details). Effect sizes of model fit
(MADaQ3
, MADRESIDCOV,
SRMR) are also available.
The function IRT.modelfit
is a wrapper to tam.modelfit
,
but allows convenient model comparisons by using the
CDM::IRT.compareModels
function.
The tam.modelfit
function can also be used for fitted
models outside the TAM package by applying
tam.modelfit.IRT
or tam.modelfit.args
.
The function tam.Q3
computes the Q_3 statistic based on
weighted likelihood estimates (see tam.wle
).
tam.modelfit(tamobj, progress=TRUE) ## S3 method for class 'tam.modelfit' summary(object,...) ## S3 method for class 'tam.mml' IRT.modelfit(object, ...) ## S3 method for class 'tam.mml.3pl' IRT.modelfit(object, ...) ## S3 method for class 'tamaan' IRT.modelfit(object, ...) ## S3 method for class 'IRT.modelfit.tam.mml' summary(object, ...) ## S3 method for class 'IRT.modelfit.tam.mml.3pl' summary(object, ...) ## S3 method for class 'IRT.modelfit.tamaan' summary(object, ...) tam.modelfit.IRT( object, progress=TRUE ) tam.modelfit.args( resp, probs, theta, post, progress=TRUE ) tam.Q3(tamobj, ... ) ## S3 method for class 'tam.Q3' summary(object,...)
tamobj |
Object of class |
progress |
An optional logical indicating whether progress should be displayed |
object |
Object of class |
resp |
Dataset with item responses |
probs |
Array with item response functions evaluated at |
theta |
Matrix with used \bold{θ} grid |
post |
Individual posterior distribution |
... |
Further arguments to be passed |
For each item i and each person n, residuals e_{ni}=X_{ni}-E(X_{ni}) are computed. The expected value E(X_{ni}) is obtained by integrating the individual posterior distribution.
The Q3 statistic of item pairs i and j is defined as the
correlation Q3_{ij}=Cor( e_{ni}, e_{nj} ). The residuals in
tam.modelfit
are
calculated by integrating values of the individual posterior distribution.
Residuals in tam.Q3
are calculated by using weighted likelihood
estimates (WLEs) from tam.wle
.
It is known that under local independence, the expected value of Q_3
is slightly smaller than zero. Therefore,
an adjusted Q3 statistic (aQ3
; aQ3_{ij})
is computed by subtracting the average of all Q3 statistics from
Q3. To control for multiple testing,
a p value adjustment by the method of
Holm (p.holm
) is employed (see Chen, de la Torre & Zhang, 2013).
An effect size of model fit (MADaQ3
) is defined as
the average of absolute values of aQ3 statistics. An equivalent
statistic based on the Q_3 statistic is similar to the
standardized generalized dimensionality discrepancy measure (SGDDM; Levy,
Xu, Yel & Svetina, 2015).
The SRMSR (standardized root mean square root of squared residuals, Maydeu-Olivaras, 2013) is based on comparing residual correlations of item pairs
SRMSR=√{ \frac{1}{ J(J-1)/2 } ∑_{i < j} ( r_{ij} - \hat{r}_{ij} )^2 }
Additionally, the SRMR is computed as
SRMR=\frac{1}{ J(J-1)/2 } ∑_{i < j} | r_{ij} - \hat{r}_{ij} |
The MADRESIDCOV statistic (McDonald & Mok, 1995) is based on comparing residual covariances of item pairs
MADRESIDCOV=\frac{1}{ J(J-1)/2 } ∑_{i < j} | c_{ij} - \hat{c}_{ij} |
This statistic is just multiplied by 100 in the output of this function.
A list with following entries
stat.MADaQ3 |
Global fit statistic |
chi2.stat |
Data frame with chi square tests of conditional independence for every item pair (Chen & Thissen, 1997) |
fitstat |
Model fit statistics 100 \cdot MADRESIDCOV, SRMR and SRMSR |
modelfit.test |
Test statistic of global fit based on multiple testing correction of χ^2 statistics |
stat.itempair |
Q3 and adjusted Q3 statistic for all item pairs |
residuals |
Residuals |
Q3.matr |
Matrix of Q_3 statistics |
aQ3.matr |
Matrix of adjusted Q_3 statistics |
Q3_summary |
Summary of Q_3 statistics |
N_itempair |
Sample size for each item pair |
Chen, J., de la Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140. doi: 10.1111/j.1745-3984.2012.00185.x
Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265-289.
Levy, R., Xu, Y., Yel, N., & Svetina, D. (2015). A standardized generalized dimensionality discrepancy measure and a standardized model-based covariance for dimensionality assessment for multidimensional models. Journal of Educational Measurement, 52(2), 144–158. doi: 10.1111/jedm.12070
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137. doi: 10.1080/15366367.2013.831680
McDonald, R. P., & Mok, M. M.-C. (1995). Goodness of fit in item response models. Multivariate Behavioral Research, 30, 23-40. doi: 10.1207/s15327906mbr3001_2
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125-145. doi: 10.1177/014662168400800201
############################################################################# # EXAMPLE 1: data.cqc01 ############################################################################# data(data.cqc01) dat <- data.cqc01 #***************************************************** #*** Model 1: Rasch model mod1 <- TAM::tam.mml( dat ) # assess model fit res1 <- TAM::tam.modelfit( tamobj=mod1 ) summary(res1) # display item pairs with five largest adjusted Q3 statistics res1$stat.itempair[1:5,c("item1","item2","aQ3","p","p.holm")] ## Not run: # IRT.modelfit fmod1 <- IRT.modelfit(mod1) summary(fmod1) #***************************************************** #*** Model 2: 2PL model mod2 <- TAM::tam.mml.2pl( dat ) # IRT.modelfit fmod2 <- IRT.modelfit(mod2) summary(fmod2) # model comparison IRT.compareModels(fmod1, fmod2 ) ############################################################################# # SIMULATED EXAMPLE 2: Rasch model ############################################################################# set.seed(8766) N <- 1000 # number of persons I <- 20 # number of items # simulate responses library(sirt) dat <- sirt::sim.raschtype( stats::rnorm(N), b=seq(-1.5,1.5,len=I) ) #*** estimation mod1 <- TAM::tam.mml( dat ) summary(dat) #*** model fit res1 <- TAM::tam.modelfit( tamobj=mod1) summary(res1) ############################################################################# # EXAMPLE 3: Model fit data.gpcm | Partial credit model ############################################################################# data(data.gpcm) dat <- data.gpcm # estimate partial credit model mod1 <- TAM::tam.mml( dat) summary(mod1) # assess model fit tmod1 <- TAM::tam.modelfit( mod1 ) summary(tmod1) ############################################################################# # EXAMPLE 4: data.read | Comparison Q3 statistic ############################################################################# library(sirt) data(data.read, package="sirt") dat <- data.read #**** Model 1: 1PL model mod1 <- TAM::tam.mml( dat ) summary(mod1) #**** Model 2: 2PL model mod2 <- TAM::tam.mml.2pl( dat ) summary(mod2) #**** assess model fits # Q3 based on posterior fmod1 <- TAM::tam.modelfit(mod1) fmod2 <- TAM::tam.modelfit(mod2) # Q3 based on WLEs q3_mod1 <- TAM::tam.Q3(mod1) q3_mod2 <- TAM::tam.Q3(mod2) summary(fmod1) summary(fmod2) summary(q3_mod1) summary(q3_mod2) ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.