getResults: Collect all results from Conquest/TAM analysis into a common...

In weirichs/eatModel: Educational Assessment Tools for IRT Modeling using Conquest or TAM

Collect all results from Conquest/TAM analysis into a common data frame

Description

First the IRT model should be defined using defineModel. Afterwards, call runModel with the argument returned by defineModel to start the estimation. The last step then is to create a results frame using getResults.

Usage

getResults( runModelObj, overwrite = FALSE, Q3 = TRUE, q3theta = c("pv", "wle", "eap"), 
            q3MinObs = 0, q3MinType = c("singleObs", "marginalSum"), omitFit = FALSE, 
            omitRegr = FALSE, omitWle = FALSE, omitPV = FALSE, abs.dif.bound = 0.6, 
            sig.dif.bound = 0.3, p.value = 0.9, nplausible = NULL, ntheta = 2000,
            normal.approx = FALSE, samp.regr = FALSE, theta.model=FALSE, np.adj=8,
            group = NULL, beta_groups = TRUE, level = .95, n.iter = 1000,
            n.burnin = 500, adj_MH = .5, adj_change_MH = .05, refresh_MH = 50, 
            accrate_bound_MH = c(.45, .55),	sample_integers=FALSE, theta_init=NULL,
            print_iter = 20, verbose = TRUE, calc_ic=TRUE, omitUntil = 1, seed=NA) 

Arguments

runModelObj	The object returned by runModel.
overwrite	Logical. Should result files be overwritten if exist?
Q3	Logical. Estimate the Q3 statistic according to Yen (1984)? Note: this is only possible for uni-dimensional models. If software == "tam", Q3 statistic is estimated using the tam.modelfit function. If software == "Conquest", Q3 statistic is estimated using the Q3 function from the sirt package.
q3theta	Specify whether the Q3 statistic should be estimated using PVs, WLEs or EAPs as the theta variable.
q3MinObs	Q3 statistic might be untrustworthy if item covariance estimation is based on very few observations. Define the minimum number of observation which should be fulfilled for Q3 estimation.
q3MinType	If "singleObs", q3MinObs argument is based on the least number of observations in the 2\times 2 0/1 frequency table of item pairs. If "marginalSum", q3MinObs argument is based on the sum of marginals in the 2\times 2 0/1 frequency table of item pairs.
omitFit	Logical. Should item fit values be included into the results?
omitRegr	Logical. Should regression parameters and their standard errors be included into the results?
omitWle	Logical. Should WLE estimates be included into the results?
omitPV	Logical. Should plausible values be included into the results?
abs.dif.bound	Applies only if DIF analyses are performed before. When DIF-Parameter are evaluated, this specifies the critical value for absolute DIF. See the details section for further information.
sig.dif.bound	Applies only if DIF analyses are performed before. When DIF-Parameter are evaluated, this specifies the critical value for confidence interval DIF. See the details section for further information.
p.value	Applies only if DIF analyses are performed before. When DIF-Parameter are evaluated, this specifies the critical p-value for confidence interval DIF. See the details section for further information.
nplausible	Applies only if software = "tam": Number of plausible values to be drawn. Note: number of plausible values were already defined in defineModel, because Conquest needs to know the number of PVs prior to estimation. In TAM, it is possible to redefine the number of plausible values and overwrite the definition that was given in defineModel.
ntheta	Applies only if software = "tam". Following description is borrowed from the help file of tam.pv from the TAM package: Number of ability nodes for plausible value imputation. Note that in this function ability nodes are simulated for the whole sample, not for every person (contrary to the software Conquest).
normal.approx	Applies only if software = "tam". Following description is borrowed from the help file of tam.pv from the TAM package: An optional logical indicating whether the individual posterior distributions should be approximated by a normal distribution? The default is FALSE. In the case normal.approx=TRUE (normal distribution approximation), the number of ability nodes ntheta can be substantially smaller than 2000, say 200 or 500. The normal approximation is implemented for unidimensional and multidimensional models.
samp.regr	Applies only if software = "tam". Following description is borrowed from the help file of tam.pv from the TAM package: An optional logical indicating whether regression coefficients should be fixed in the plausible value imputation or also sampled from their posterior distribution? The default is FALSE. Sampled regression coefficients are obtained by nonparametric bootstrap.
theta.model	Applies only if software = "tam". Following description is borrowed from the help file of tam.pv from the TAM package: Logical indicating whether the theta grid from the tamobj object should be used for plausible value imputation. In case of normal.approx=TRUE, this should be sufficient in many applications.
np.adj	Applies only if software = "tam". Following description is borrowed from the help file of tam.pv from the TAM package: This parameter defines the “spread” of the random theta values for drawing plausible values when normal.approx=FALSE. If s_{EAP} denotes the standard deviation of the posterior distribution of theta (in the one-dimensional case), then theta is simulated from a normal distribution with standard deviation np.adj times s_{EAP}.
group	Applies only if software = "tam" and pvMethod = "bayesian". Optional vector of group identifiers. See the help page of tam.pv.mcmc for further details.
beta_groups	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
level	Applies only if software = "tam" and pvMethod = "bayesian". Confidence level in bayesian approach. See the help page oftam.pv.mcmc for further details.
n.iter	Applies only if software = "tam" and pvMethod = "bayesian". Number of iterations in the bayesian approach. See the help page of tam.pv.mcmc for further details.
n.burnin	Applies only if software = "tam" and pvMethod = "bayesian". Number of burn-in iterations in the bayesian approach. See the help page of tam.pv.mcmc for further details.
adj_MH	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
adj_change_MH	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
refresh_MH	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
accrate_bound_MH	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
sample_integers	Applies only if software = "tam" and pvMethod = "bayesian". Logical indicating whether weights for complete cases should be sampled in bootstrap. See the help page of tam.pv.mcmc for further details.
theta_init	Applies only if software = "tam" and pvMethod = "bayesian". Optional matrix with initial theta values. See the help page of tam.pv.mcmc for further details.
print_iter	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
verbose	Applies only if software = "tam" and pvMethod = "bayesian". See the help page of tam.pv.mcmc for further details.
calc_ic	Applies only if software = "tam" and pvMethod = "bayesian". Logical indicating whether information criteria should be computed. See the help page of tam.pv.mcmc for further details.
omitUntil	Argument is passed to plotDevianceConquest: An optional value indicating number of iterations to be omitted for plotting.
seed	Fixed simulation seed. This value is directly passed on to the tam.fit function.

Details

If defineModel was run with software Conquest, a path argument ('dir') is necessary. The path argument is optional for software TAM. If 'dir' was specified, getResults additionally writes its output into the specified folder, using the analysis.name argument for file naming. Otherwise, getResults only returnes the result data frame.

If DIF analyses were performed before, the user can specify the criteria according to which DIF should be interpreted or evaluated. By default, the ETS criteria (Zieky, 1993) are used which classify DIF into three distinct categories, "A", "B", or "C". Small DIF ("A") corresponds to absolute DIF values below .43 (no significance test is performed here); medium DIF ("B") corresponds to absolute DIF values between .43 and .64 which are significantly different from zero. High DIF ("C") corresponds to absolute DIF values above .64 which are significantly different from .43 (DeMars, 2011; Monahan et al. 2007). Alternatively, the three arguments abs.dif.bound, sig.dif.bound, and p.value allow to specify user-defined dichotomous criteria. If items should be flagged as DIF, if the absolute value increases 0.5 and significantly exceeds 0.1 at a alpha level of 0.05, use abs.dif.bound = 0.5 and sig.dif.bound = 0.1 and p.value = 0.95.

Value

A data frame in the long format with ten columns.

model	The name of the model (as specified by the user in analysis.name.
source	The estimation software (i.e, conquest or TAM)
var1	The variable name for which the corresponding value is given, i.e. its indicator.
var2	Additional variable information if necessary.
type	Type of coefficient (for example, random or fixed).
indicator.group	The type of the group the corresponding variable belongs to.
group	The group the corresponding variable belongs to. Note: group is nested within indicator.group.
par	The type of the parameter.
derived.par	Optionally: The derived parameter.
value	The value of the corresponding estimate.

References

DeMars, C. E. (2011). An analytic comparison of effect sizes for differential item functioning. Applied Measurement in Education, 24 (3), 189-209. https://doi.org/10.1080/08957347.2011.580255

Monahan, P. O., McHorney, C. A., Stump, T. E. & Perkins, A. J. (2007). Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. Journal of Educational and Behavioral Statistics, 32 (1), 92-109. https://doi.org/10.3102/1076998606298035

Zieky, M. (1993). Practical questions in the use of DIF statistics in item development. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum.

Examples

# see examples in the help file of defineModel()

weirichs/eatModel documentation built on Jan. 26, 2025, 4:01 p.m.