mirtCAT | R Documentation |
Provides tools to generate an HTML interface for creating adaptive and
non-adaptive educational and psychological tests using the shiny
package.
Suitable for applying unidimensional and multidimensional computerized adaptive tests
using item response theory methodology. Test scoring is performed using the mirt
package.
However, if no scoring is required (i.e., a standard survey) then defining a mirt
object may be omitted.
mirtCAT(
df = NULL,
mo = NULL,
method = "MAP",
criteria = "seq",
start_item = 1,
local_pattern = NULL,
AnswerFuns = list(),
design_elements = FALSE,
cl = NULL,
progress = FALSE,
primeCluster = TRUE,
customTypes = list(),
design = list(),
shinyGUI = list(),
preCAT = list(),
...
)
## S3 method for class 'mirtCAT'
print(x, ...)
## S3 method for class 'mirtCAT'
summary(object, sort = TRUE, ...)
## S3 method for class 'mirtCAT'
plot(
x,
pick_theta = NULL,
true_thetas = TRUE,
SE = 1,
main = NULL,
par.strip.text = list(cex = 0.7),
par.settings = list(strip.background = list(col = "#9ECAE1"), strip.border = list(col =
"black")),
scales = list(x = list(rot = 90)),
...
)
df |
a
|
mo |
single group object defined by the |
method |
argument passed to |
criteria |
adaptive criteria used, default is to administer each item sequentially
using Possible inputs for unidimensional adaptive tests include: Possible inputs for multidimensional adaptive tests include: Applicable to both unidimensional and multidimensional tests are the
Non-adaptive methods applicable even when no |
start_item |
two possible inputs to determine the starting item are available.
Passing a number will indicate the specific item to be used as the start item;
default is 1, which selects the first item in the defined test/survey.
If a character string is passed then the item will be selected from one of
the item selections criteria available (see the |
local_pattern |
a character/numeric matrix of response patterns
used to run the CAT application without generating the GUI interface.
This option requires complete response pattern(s) to be supplied. |
AnswerFuns |
a list with the length equal to the number of items in the item bank consisting
of user-defined functions. These functions are used to determine whether a given
response obtained from the GUI is 'correct' or 'incorrect' by returning a logical scalar value,
while For example, the following provides a customized response function for the first item. AnswerFuns <- as.list(rep(NA, nrow(df))) AnswerFuns[[1]] <- function(input) input == '10' || to.lower(input) == 'ten' |
design_elements |
logical; return an object containing the test, person, and design
elements? Primarily this is to be used with the |
cl |
an object definition to be passed to the parallel package
(see |
progress |
logical; print a progress bar to the console
with the |
primeCluster |
logical; when a |
customTypes |
an optional list input containing functions for Designing Original Graphical Stimuli (DOGS).
DOGS elements in the input list must contain a unique name, and the item with which it is associated must be
declared in the a myDOGS <- function(inputId, df_row) ... and must return, at the very minimum, an associated The following is a simple example of DOGS for a true-false question and how it is passed: good_dogs <- function(inputId, df_row){ return(list(h2('This statement is false'), radioButtons(inputId = inputId, label='', choices = c('True', 'False'), selected = '') )) } df <- data.frame(Question = '', ..., Type = 'Doug') results <- mirtCAT(df=df, customTypes = list(Doug = good_dogs)) IMPORTANT: When using the custom inputs the select defined |
design |
a list of design based control parameters for adaptive and non-adaptive tests. These can be
|
shinyGUI |
a list of GUI based parameters to be over-written. These can be
|
preCAT |
a list object which can be used to specify a pre-CAT block in which
different test properties may be applied prior to beginning the CAT session. If the
list is empty, no preCAT block will be used. All of the following elements are required
to use the
|
... |
additional arguments to be passed to |
x |
object of class |
object |
object of class |
sort |
logical; sort the response patterns based on the order they were administered? If FALSE, the raw response patterns containing NAs will be returned for items that were not administered |
pick_theta |
a number indicating which theta to plot (only applicable for multidimensional
tests). The default is to facet each theta on one plot, but to plot only the first factor pass
|
true_thetas |
logical; include a horizontal line indicating where the population-level theta values are? Only applicable to Monte Carlo simulations because this value would not be known otherwise |
SE |
size of the standard errors to plot. The default is 1, and therefore plots the
standard error. To obtain the 95% interval use |
main |
title of the plot. Will default to |
par.strip.text |
plotting argument passed to |
par.settings |
plotting argument passed to |
scales |
plotting argument passed to |
All tests will stop once the 'min_SEM'
criteria has been reached or classification
above or below the specified cutoffs can be made. If all questions should
be answered, users should specify an extremely small 'min_SEM'
or, equivalently,
a large 'min_items'
criteria to the design
list input.
Returns a list object of class 'Person'
containing the following elements:
raw_responses
A character vector indicating the raws responses to the respective items, where NA indicates the item was not answered
scored_responses
An integer vector of scored responses if the item_answers
input
was used for each respective item
items_answered
An integer vector indicating the order in which the items were answered
thetas
A numeric vector indicating the final theta estimates
SE_thetas
A numeric vector indicating the standard errors of the final theta estimates
thetas_history
A matrix indicating the progression of updating the theta values during the test
thetas_SE_history
A matrix indicating the standard errors for theta after each successive item was answered
item_time
A numeric vector indicating how long the respondent took to answer each question (in seconds)
demographics
A data.frame object containing the information collected on the first page of the shiny GUI. This is used to store the demographic information for each participant
classification
A character vector indicating whether the traits could be classified as 'above' or 'below' the desired cutoffs
To access examples, vignettes, and exercise files that have been generated with knitr
please
visit https://github.com/philchalmers/mirtCAT/wiki.
design
object directly through customNextItem()
(advanced)In addition to providing a completely defined item-selection map via the customNextItem()
function,
users may also wish to control some of the more fine-grained elements of the design
object to adjust
the general control parameters of the CAT (e.g., modifying the maximum number of items to administer, stopping
the CAT if something peculiar has been detected in the response patterns, etc). Note that
this feature is rarely required for most applications, though more advanced users may wish to
modify these various low-level elements of the design
object directly to change the flow of the CAT
to suit their specific needs.
While the person
object is defined as a Reference Class
(see setRefClass
)
the design object is generally considered a fixed S4 class, meaning that, unlike the person
object,
it's elements are not mutable. Therefore, in order to make changes directly to the
design
object the users should follow these steps:
Within the defined customNextItem
function, the design
object slots are first modified (e.g.,
design@max_items <- 20L
).
Along with the desired next item scalar value from customNextItem()
, the scalar object should also
contain an attribute with the name 'design'
which holds the newly defined design
object
(e.g., attr(ret, 'design') <- design; return(ret)
).
Following the above process the work-flow in mirtCAT
will use the new design
object in place of the
old one, even in Monte Carlo simulations.
Phil Chalmers rphilip.chalmers@gmail.com
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v048.i06")}
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5), 1-39. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v071.i05")}
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v048.i06")}
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5), 1-39. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v071.i05")}
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v048.i06")}
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5), 1-39. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v071.i05")}
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v048.i06")}
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5), 1-39. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.18637/jss.v071.i05")}
generate_pattern
, generate.mirt_object
,
extract.mirtCAT
, findNextItem
, computeCriteria
## Not run:
### unidimensional scored example with generated items
# create mo from estimated parameters
set.seed(1234)
nitems <- 50
itemnames <- paste0('Item.', 1:nitems)
a <- matrix(rlnorm(nitems, .2, .3))
d <- matrix(rnorm(nitems))
dat <- simdata(a, d, 1000, itemtype = 'dich')
mod <- mirt(dat, 1)
coef(mod, simplify=TRUE)
# alternatively, define mo from population values (not run)
pars <- data.frame(a1=a, d=d)
mod2 <- generate.mirt_object(pars, itemtype='2PL')
coef(mod2, simplify=TRUE)
# simple math items
questions <- answers <- character(nitems)
choices <- matrix(NA, nitems, 5)
spacing <- floor(d - min(d)) + 1 #easier items have more variation in the options
for(i in 1:nitems){
n1 <- sample(1:50, 1)
n2 <- sample(51:100, 1)
ans <- n1 + n2
questions[i] <- paste0(n1, ' + ', n2, ' = ?')
answers[i] <- as.character(ans)
ch <- ans + sample(c(-5:-1, 1:5) * spacing[i,], 5)
ch[sample(1:5, 1)] <- ans
choices[i, ] <- as.character(ch)
}
df <- data.frame(Question=questions, Option=choices,
Type = 'radio', stringsAsFactors = FALSE)
head(df)
(res <- mirtCAT(df)) #collect response only (no scoring or estimating thetas)
summary(res)
# include scoring by providing Answer key
df$Answer <- answers
(res_seq <- mirtCAT(df, mod)) #sequential scoring
(res_random <- mirtCAT(df, mod, criteria = 'random')) #random
(res_MI <- mirtCAT(df, mod, criteria = 'MI', start_item = 'MI')) #adaptive, MI starting item
summary(res_seq)
summary(res_random)
summary(res_MI)
#-----------------------------------------
# HTML tags for better customization, coerced to characters for compatibility
# help(tags, package='shiny')
options <- matrix(c("Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"),
nrow = 3, ncol = 5, byrow = TRUE)
shinyStems <- list(HTML('Building CATs with mirtCAT is difficult.'),
div(HTML('mirtCAT requires a'), br(), HTML('substantial amount of coding.')),
div(strong('I would use'), HTML('mirtCAT in my research.')))
questions <- sapply(shinyStems, as.character)
df <- data.frame(Question=questions,
Option = options,
Type = "radio",
stringsAsFactors=FALSE)
res <- mirtCAT(df)
res
#-----------------------------------------
# run locally, random response pattern given Theta
set.seed(1)
pat <- generate_pattern(mod, Theta = 0, df=df)
head(pat)
# seq scoring with character pattern for the entire test (adjust min_items)
res <- mirtCAT(df, mod, local_pattern=pat, design = list(min_items = 50))
summary(res)
# same as above, but using special input vector that doesn't require df input
set.seed(1)
pat2 <- generate_pattern(mod, Theta = 0)
head(pat2)
print(mirtCAT(mo=mod, local_pattern=pat2))
# run CAT, and save results to object called person (start at 10th item)
person <- mirtCAT(df, mod, item_answers = answers, criteria = 'MI',
start_item = 10, local_pattern = pat)
print(person)
summary(person)
# plot the session
plot(person) #standard errors
plot(person, SE=1.96) #95 percent confidence intervals
#-----------------------------------------
### save response object to temp directory in case session ends early
wdf <- paste0(getwd(), '/temp_file.rds')
res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))
# resume test this way if test was stopped early (and temp files were saved)
res <- mirtCAT(df, mod, shinyGUI = list(temp_file = wdf))
print(res)
## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.