Firestar: Firestar - Computerized Adaptive Testing (CAT) simulation...

In choi-phd/Firestar: Computerized Adaptive Testing (CAT) Simulation Program

Firestar - Computerized Adaptive Testing (CAT) simulation program

Description

Firestar simulates CAT with dichotomous and polytomous IRT models and generates results in various tables and plots

Usage

Firestar(
  filename.ipar = "",
  item.pool = NULL,
  filename.resp = "",
  filename.content = "",
  ncc = 1,
  filename.theta = "",
  true.theta = NULL,
  min.score.0 = FALSE,
  simulate.theta = FALSE,
  pop.dist = "NORMAL",
  pop.par = c(0, 1),
  n.simulee = 1000,
  eap.full.length = TRUE,
  max.cat = 5,
  min.theta = -4,
  max.theta = 4,
  inc = 0.1,
  min.NI = 4,
  max.NI = 12,
  max.SE = 0.3,
  exposure.control = FALSE,
  exposure.control.method = "RD",
  top.N = 1,
  PAS = 1,
  r.max = 0.25,
  stop.SE = 0.01,
  continue.SE = 0.03,
  min.SE.change = 0,
  extreme.response.check = "N",
  max.extreme.response = 4,
  selection.method = "MPWI",
  info.AMC = "KL",
  stop.AMC = "SE",
  alpha.AMC = 0.05,
  BH = FALSE,
  interim.theta = "EAP",
  Fisher.scoring = TRUE,
  shrinkage.correction = FALSE,
  se.method = 1,
  first.item.selection = 1,
  first.at.theta = 0,
  first.item = 1,
  show.theta.audit.trail = FALSE,
  plot.usage = FALSE,
  plot.info = FALSE,
  plot.prob = FALSE,
  add.final.theta = FALSE,
  bank.diagnosis = FALSE,
  prior.dist = 1,
  prior.mean = 0,
  prior.sd = 1,
  file.items.used = "",
  file.theta.history = "",
  file.se.history = "",
  file.final.theta.se = "",
  file.other.thetas = "",
  file.likelihood.dist = "",
  file.posterior.dist = "",
  file.matrix.info = "",
  file.full.length.theta = "",
  file.selected.item.resp = "",
  output.previous = NULL
)

Arguments

filename.ipar	Name of a required item parameter file (comma separated, no headers, columns in the order of id, model, a, cb1, cb2,...,cbk, blank for NA; model: 1=1PL, 2=2PL, 3=3PL, 4=PC, 5=GPC, 6=GR)
item.pool	Object of item.pool class
filename.resp	Name of an optional item response file (comma separated, no headers, item responses, base 1, blank for missing)
filename.content	Name of an optional content specification file
ncc	Number of Content Categories, effective only if content balancing is invoked by providing filename.content
filename.theta	Name of an optional true or external theta file
true.theta	True theta values (default: NULL)
min.score.0	TRUE if the minimum item score is 0 not 1 (default: FALSE)
simulate.theta	TRUE to simulate item responses or FALSE to read in from an external file (filename.theta)
pop.dist	Population distribution type for simulated theta: NORMAL, UNIFORM, or GRID
pop.par	Population distribution parameters: For example, pop.par=c(M,SD) if pop.dist="NORMAL", pop.par=c(LL,UL) if pop.dist="UNIFORM", or pop.par=c(-3,-2,...3) if pop.dist="GRID"
n.simulee	Toral number of simulees to generate if pop.dist in c("NORMAL","UNIFORM") or the number per theta point if pop.dist="GRID"
eap.full.length	TRUE to generate EAP theta estimates based on all items or FALSE to supress
max.cat	Maximum number of response categories across items
min.theta	Minimum theta value
max.theta	Maximum theta value
inc	Theta increment value to generate a grid between min.theta and max.theta
min.NI	Minimum number of items to administer (default: 4)
max.NI	Maximum number of items to administer (default: 12)
max.SE	Maximum SE for stopping
exposure.control	TRUE to invoke exposure control or FALSE to supress (default: FALSE)
exposure.control.method	Exposure control method: RD, PR, SH (defaul: "RD")
top.N	Top N items from which a next item is selected randomly; effective when exposure.control.method == "Randomesque" (default: 1)
PAS	A vector of the Probability of Administration given Selection, P(A|S), for each item; effective when exposure.control.method == "SH" (default: 1)
r.max	Maximumum target exposure rate; effective when exposure.control.method == "SH" (default = 0.25)
stop.SE	Minimum reduction in predicted SE to override continuing and stop under PSER (default: 0.01)
continue.SE	Minimum reduction in predicted SE to override stopping and continue under PSER (default: 0.03)
min.SE.change	Minimum reduction in SE to continue beyond satisfying min.NI (default: 0.0); not effective under PSER
extreme.response.check	Check for repeated extreme responses: L for checking in the left side (low) only, R for right (high) only, E for either, or N for neither (default: N)
max.extreme.response	Maximum number of responses allowed before stopping (default: 4)
selection.method	Item selection method: MFI, MKL, MLWI, MPWI, MPWKL, MEI, MEPV, MEPWI, RND, KET, LOC, SEQ, TSB, PSER, MI, or AMC (default: MPWI)
info.AMC	Information method for AMC: KL, MI, PWKL, or FI (default: KL)
stop.AMC	Test statistic for AMC to determine whether to stop: SE, Z, LR, or ST (default: SE)
alpha.AMC	Type-I error rate for AMC test statistic (default: 0.05)
BH	TRUE to apply Benjamini-Hotchberg correction (default: FALSE)
interim.theta	Interim theta estimator: EAP or MLE
Fisher.scoring	TRUE to use Fisher's method of scoring for MLE
shrinkage.correction	TRUE to correct for the bias of EAP (default: FALSE)
se.method	SE estimation method: 1 = Posterior Standard Deviation or 2 = Inverse of Square Root of Information
first.item.selection	Alternative first item selection method: 1 = Prior Mean, 2 = At a fixed value specified by first.at.theta, 3 = Use a specific item identified by first.item, or 4 = At external or theta values specified by filename.theta
first.at.theta	Specific theta location at which the first item is optimized
first.item	Specific item number to be selected as the first item
show.theta.audit.trail	TRUE to generate CAT audit trail plots or FALSE to suppress
plot.usage	TRUE to generate item usage plot or FALSE to suppress
plot.info	TRUE to generate item intormation plots or FALSE to suppress
plot.prob	TRUE to generate item response probability plots or FALSE to suppress
add.final.theta	TRUE to append three additional final theta estimates (MLE, MAP, and WLE) to file.other.thetas or FALSE to supress
bank.diagnosis	TRUE to generate item bank diagnostic plots or FALSE to suppress
prior.dist	Type of prior distribution: 1 = Normal or 2 = Losgistic
prior.mean	Prior distribution mean (default: 0.0)
prior.sd	Prior distribution standard deviation (default: 1.0)
file.items.used	Name of the file to contain information on items administered
file.theta.history	Name of the file to contain information on history of theta estimates
file.se.history	Name of the file to contain information on history of SE estimates
file.final.theta.se	Name of the file to contain final theta and SE estimates
file.other.thetas	Name of the file to contain other theta estimates (MLE, MAP, and WLE)
file.likelihood.dist	Name of the file to contain likelihood functions
file.posterior.dist	Name of the file to contain posterior distributions
file.matrix.info	Name of the file to contain the item information matrix
file.full.length.theta	Name of the file to contain theta estimates based on all items in the bank
file.selected.item.resp	Name of the file to contain item responses for the selected items only
output.previous	List object from Firestar for the previous test

Details

Firestar is designed for simulating CAT with dichotomous and polytomous items. The item response theory models supported by the program include the dichotomous models (Birnbaum, 1968), Samejima's (1969) graded response model (GRM) and Muraki's (1992) generalized partial credit model (GPCM). Both Masters' (1982) partial credit model (PCM) and Andrich's (1978) rating scale model are also supported as special cases of the GPCM.

Value

List of summary statistics and output results:

• call Call with all of the specified arguments

• nia Total number of items administered

• mean.nia Mean of the number of items administered

• cor.theta Correlation between true theta and theta from CAT

• rmsd.theta RMSE based on true theta and theta from CAT

• true.theta True theta

• mean.SE Mean standard error

• item.pool Item pool object

• resp Item response matrixc

• items.used Items used by examinee

• theta.history Theta history by examinee

• se.history Standard error history by examinee

• selected.item.resp Selected item responses by examinee

• final.theta.se Final theta and standard error by examinee

• likelihood.dist Final likelihood distribution by examinee

• posterior.dist Final posterior distribution by examinee

• matrix.info Matrix of item information

• ni.administered Number of items administered by examinee

• Z Z-test statistic if selection.method == 'AMC'

• LR Likelihood-ratio test statistic if selection.method == 'AMC'

• ST Score test statistic if selection.method == 'AMC'

Author(s)

Seung W Choi, [e-mail]

References

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Reading, MA: Addison-Wesley. Choi, S. W. (2009). Computerized Adaptive Testing Simulation Program for Polytomous IRT Models. Applied Psychological Measurement. 33, 644-645. Choi, S. W., & Swartz, J. R. (2009). Comparison of CAT Item Selection Criteria for Polytomous Items. Applied Psychological Measurement. 33, 419-440. Choi, S. W., Grady, M., & Dodd, B. G. (2011). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurement. 71, 37-53. Choi, S. W., Podrabsky, T., & McKinney, N. (2012). Firestar-D: Computerized Adaptive Testing Simulation Program for Dichotomous Item Response Theory Models. Applied Psychological Measurement, 36, 67-68. Choi, S. W. (2018). Firestar: Simulating Computerized Adaptive Testing. In W. J. van der Linden (Ed.), Handbook of Item Response Theory. Chapman and Hall/CRC. Finkelman, M. D., Weiss, D. J., Kim-Kang, G. (2010). Item selection and hypothesis testing for the adaptive measurement of change. Applied Psychological Measurement, 34, 238-254. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.

choi-phd/Firestar documentation built on Sept. 14, 2022, 1:47 a.m.