FI: Calculate Expected and Observed Fisher Information for IRT...
In swnydick/catIrt: Simulate IRT-Based Computerized Adaptive Tests
FI R Documentation
Calculate Expected and Observed Fisher Information for IRT Models
Description
FI calculates expected and/or observed Fisher information for various
IRT models given a vector of ability values, a vector/matrix of item
parameters, and an IRT model. It also calculates test
information and expected/observed standard error of measurement.
Usage
FI( params, theta, type = c("expected", "observed"), resp = NULL )
## S3 method for class 'brm'
FI( params, theta, type = c("expected", "observed"), resp = NULL )
## S3 method for class 'grm'
FI( params, theta, type = c("expected", "observed"), resp = NULL )
Arguments
params
numeric: a vector or matrix of item parameters. If specified
as a matrix, the rows must index the items, and the columns
must designate the item parameters. Furthermore, if calculating
expected information, the number of rows must match the number
of columns of resp. The class of params must match
the model: either "brm" or "grm". For the binary response
model, params must either be a 3-dimensional vector or a 3-column
matrix. See Details for more information.
theta
numeric: a vector of ability values, one for each simulee. If calculating
expected information, the length of theta must match
the number of rows of resp, unless theta is a scalar,
in which case resp could also be a vector of length
nrow(params).
type
character: a character string indicating the type of information, either
"expected" or "observed". For the 1-parameter and
2-parameter binary response model (of class "brm" with the
third column of params set to 0), both "expected"
and "observed" information are identical. See Details
for more information.
resp
numeric: either a N \times J matrix (where N indicates the
number of simulees and J indicates the number of items), a
N length vector (if there is only one item) or a J length
vector (if there is only one simulee). For the binary response model
("brm"), resp must solely contain 0s and 1s. For the
graded response model ("grm"), resp must solely contain
integers 1, …, K, where K is the number of categories, as
indicated by the dimension of params.
Details
The function FI returns item information, test information, and standard error
of measurement for the binary response model ("brm") or the graded response
model ("grm"). If the log likelihood is twice differentiable, expected Fisher
information is the negative, expected, second derivative of the log likelihood with respect
to the parameter. For the binary response model, expected item information simplifies to the
following:
I(θ_i | a_j, b_j, c_j) = (p_{ij}^{\prime})^2/(p_{ij}(1 - p_{ij}))
where p_{ij}^{\prime} is the partial derivative
of p_{ij} with respect to θ, and p_{ij} is the probability of response, as
indicated in the help page for simIrt.
For the graded response model, expected item information simplifies to the following:
I(θ_i | a_j, b_{j1}, …, b_{j(k - 1)}) = ∑_k[(P_{ijk}^{\prime})^2/P_{ijk}]
where P_{ijk}^{\prime} is the partial derivative
of P_{ijk} with respect to θ, and P_{ijk} is the probability of responding
in category k as indicated in the help page for simIrt. See van der Linden and Pashley
(2010).
Observed information is the negative second derivative of the log-likelihood. For the binary
response model ("brm") with 2-parameters (such that the third column of the
parameter matrix is set to 0), observed and expected information are identical because the second
derivative of their log-likelihoods do not contain observed data. See Baker and Kim (2004),
pp. 66 – 69.
For all models, test information is defined as the following:
T(θ_i) = ∑_jI_j(θ_i)
where I(θ_i)_j is shorthand for Fisher information of simulee i on item j.
Finally, the standard error of measurement (SEM) is the inverse, square-root of test information.
FI is frequently used to select items in a CAT and to estimate the precision
of \hat{θ}_i after test termination.
Value
FI, FI.brm, and FI.grm return a list of the following elements:
item
either: (1) a N \times J matrix of item information for each simulee to
each item; (2) a J-length vector of item information for one simulee to
each item; or (3) an N-length vector of item information for all simulees
to one item, depending on the dimensions of params and theta.
test
an N-length vector of test information, one for each simulee. Test
information is the sum of item information across items. See Details for
more information.
sem
an N-length vector of expected or observed standard error of measurement
for each simulee, which is the inverse-square-root of test information.
See Details for more information.
type
either "observed" or "expected", indicating the type
of information calculated.
Author(s)
Steven W. Nydick swnydick@gmail.com
References
Baker, F. B., & Kim, S.-H. (2004). Item Response Theory: Parameter Estimation Techniques, Second Edition. New York, NY: Marcel Dekker, Inc.
Dodd, B. G., De Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with polytomous items. Applied Psychological Measurement, 19, 5 – 22.
Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Mahway, NJ: Lawrence Erlbaum Associates.
Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79 – 86.
van der Linden, W. J. & Pashley, P. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing. New York, NY: Springer.
See Also
catIrt, KL, simIrt
Examples
#########################
# Binary Response Model #
#########################
## 1 ##
set.seed(888)
# generating random theta:
theta <- rnorm(20)
# generating an item bank under a 2-parameter binary response model:
b.params <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = 0)
# simulating responses using random theta:
b.mod <- simIrt(params = b.params, theta = theta, mod = "brm")
# you can indicate class of params or extract it from simIrt object:
class(b.params) <- "brm"
# calculating expected and observed information:
e.info <- FI(params = b.params, theta = theta, type = "expected")
o.info <- FI(params = b.params, theta = theta, type = "observed", resp = b.mod$resp)
# 2-parameter model, so e.info will be equal to o.info:
all(signif(e.info$item) == signif(o.info$item))
## 2 ##
# generating an item bank under a 3-parameter binary response model:
b.params2 <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2)
# simulating responses using pre-specified thetas:
b.mod2 <- simIrt(params = b.params2, mod = "brm")
# calculating expected and observed information:
# (if you don't indicate class, you can extract from simIrt object)
e.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "expected")
o.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "observed",
resp = b.mod2$resp)
# 3-parameter model, so e.info will not be equal to o.info:
all(signif(e.info2$item) == signif(o.info2$item))
## 3 ##
# if theta is a scalar, item will be a vector and test will be a scalar:
e.info3 <- FI(params = b.params2, theta = 0, type = "expected")
dim(e.info3$item) # no dimension because it's a vector
length(e.info3$item) # of length equal to the number of items
# if params is a vector, item will be a matrix with one row:
e.info4 <- FI(params = c(1, 2, 0), theta = c(1, 2), type = "expected")
dim(e.info4$item)
# if you don't class params, FI will assume a binary response model.
#########################
# Graded Response Model #
#########################
set.seed(999)
# generating random theta
theta <- rnorm(10)
# generating an item bank under a graded response model:
g.params <- cbind(a = runif(30, .5, 1.5), b1 = rnorm(30), b2 = rnorm(30),
b3 = rnorm(30), b4 = rnorm(30))
# you can sort the parameters yourself:
g.params <- cbind(g.params[ , 1],
t(apply(g.params[ ,2:dim(g.params)[2]], MARGIN = 1,
FUN = sort)))
# simulating responses using random theta:
g.mod <- simIrt(params = g.params, theta = theta, mod = "grm")
# calculating expected and observed information:
class(g.params) <- "grm" # always indicate model or extract from simulation.
e.info5 <- FI(params = g.params, theta = theta, type = "expected")
o.info5 <- FI(params = g.params, theta = theta, type = "observed", resp = g.mod$resp)
# grm, so e.info will not be equal to o.info:
all(signif(e.info5$item) == signif(o.info5$item))
# if thet is a vector and params is a vector, item will be a J x N matrix:
dim(e.info5$item)
# if you don't want to sort the parameters, you can extract from simIrt object:
e.info6 <- FI(params = g.mod$params[ , -1], theta = g.mod$theta, type = "expected")
# but you first need to remove column 1 (the item number column).
swnydick/catIrt documentation built on June 13, 2022, 9:30 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| FI | R Documentation |
Calculate Expected and Observed Fisher Information for IRT Models
Description
FI calculates expected and/or observed Fisher information for various
IRT models given a vector of ability values, a vector/matrix of item
parameters, and an IRT model. It also calculates test
information and expected/observed standard error of measurement.
Usage
FI( params, theta, type = c("expected", "observed"), resp = NULL )
## S3 method for class 'brm'
FI( params, theta, type = c("expected", "observed"), resp = NULL )
## S3 method for class 'grm'
FI( params, theta, type = c("expected", "observed"), resp = NULL )
Arguments
params |
numeric: a vector or matrix of item parameters. If specified
as a matrix, the rows must index the items, and the columns
must designate the item parameters. Furthermore, if calculating
expected information, the number of rows must match the number
of columns of |
theta |
numeric: a vector of ability values, one for each simulee. If calculating
expected information, the length of |
type |
character: a character string indicating the type of information, either
"expected" or "observed". For the 1-parameter and
2-parameter binary response model (of class "brm" with the
third column of |
resp |
numeric: either a N \times J matrix (where N indicates the
number of simulees and J indicates the number of items), a
N length vector (if there is only one item) or a J length
vector (if there is only one simulee). For the binary response model
("brm"), |
Details
The function FI returns item information, test information, and standard error
of measurement for the binary response model ("brm") or the graded response
model ("grm"). If the log likelihood is twice differentiable, expected Fisher
information is the negative, expected, second derivative of the log likelihood with respect
to the parameter. For the binary response model, expected item information simplifies to the
following:
I(θ_i | a_j, b_j, c_j) = (p_{ij}^{\prime})^2/(p_{ij}(1 - p_{ij}))
where p_{ij}^{\prime} is the partial derivative
of p_{ij} with respect to θ, and p_{ij} is the probability of response, as
indicated in the help page for simIrt.
For the graded response model, expected item information simplifies to the following:
I(θ_i | a_j, b_{j1}, …, b_{j(k - 1)}) = ∑_k[(P_{ijk}^{\prime})^2/P_{ijk}]
where P_{ijk}^{\prime} is the partial derivative
of P_{ijk} with respect to θ, and P_{ijk} is the probability of responding
in category k as indicated in the help page for simIrt. See van der Linden and Pashley
(2010).
Observed information is the negative second derivative of the log-likelihood. For the binary response model ("brm") with 2-parameters (such that the third column of the parameter matrix is set to 0), observed and expected information are identical because the second derivative of their log-likelihoods do not contain observed data. See Baker and Kim (2004), pp. 66 – 69.
For all models, test information is defined as the following:
T(θ_i) = ∑_jI_j(θ_i)
where I(θ_i)_j is shorthand for Fisher information of simulee i on item j.
Finally, the standard error of measurement (SEM) is the inverse, square-root of test information.
FI is frequently used to select items in a CAT and to estimate the precision
of \hat{θ}_i after test termination.
Value
FI, FI.brm, and FI.grm return a list of the following elements:
item |
either: (1) a N \times J matrix of item information for each simulee to
each item; (2) a J-length vector of item information for one simulee to
each item; or (3) an N-length vector of item information for all simulees
to one item, depending on the dimensions of |
test |
an N-length vector of test information, one for each simulee. Test information is the sum of item information across items. See Details for more information. |
sem |
an N-length vector of expected or observed standard error of measurement for each simulee, which is the inverse-square-root of test information. See Details for more information. |
type |
either "observed" or "expected", indicating the type of information calculated. |
Author(s)
Steven W. Nydick swnydick@gmail.com
References
Baker, F. B., & Kim, S.-H. (2004). Item Response Theory: Parameter Estimation Techniques, Second Edition. New York, NY: Marcel Dekker, Inc.
Dodd, B. G., De Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with polytomous items. Applied Psychological Measurement, 19, 5 – 22.
Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. Mahway, NJ: Lawrence Erlbaum Associates.
Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79 – 86.
van der Linden, W. J. & Pashley, P. J. (2010). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of Adaptive Testing. New York, NY: Springer.
See Also
catIrt, KL, simIrt
Examples
#########################
# Binary Response Model #
#########################
## 1 ##
set.seed(888)
# generating random theta:
theta <- rnorm(20)
# generating an item bank under a 2-parameter binary response model:
b.params <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = 0)
# simulating responses using random theta:
b.mod <- simIrt(params = b.params, theta = theta, mod = "brm")
# you can indicate class of params or extract it from simIrt object:
class(b.params) <- "brm"
# calculating expected and observed information:
e.info <- FI(params = b.params, theta = theta, type = "expected")
o.info <- FI(params = b.params, theta = theta, type = "observed", resp = b.mod$resp)
# 2-parameter model, so e.info will be equal to o.info:
all(signif(e.info$item) == signif(o.info$item))
## 2 ##
# generating an item bank under a 3-parameter binary response model:
b.params2 <- cbind(a = runif(100, .5, 1.5), b = rnorm(100, 0, 2), c = .2)
# simulating responses using pre-specified thetas:
b.mod2 <- simIrt(params = b.params2, mod = "brm")
# calculating expected and observed information:
# (if you don't indicate class, you can extract from simIrt object)
e.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "expected")
o.info2 <- FI(params = b.params2, theta = b.mod2$theta, type = "observed",
resp = b.mod2$resp)
# 3-parameter model, so e.info will not be equal to o.info:
all(signif(e.info2$item) == signif(o.info2$item))
## 3 ##
# if theta is a scalar, item will be a vector and test will be a scalar:
e.info3 <- FI(params = b.params2, theta = 0, type = "expected")
dim(e.info3$item) # no dimension because it's a vector
length(e.info3$item) # of length equal to the number of items
# if params is a vector, item will be a matrix with one row:
e.info4 <- FI(params = c(1, 2, 0), theta = c(1, 2), type = "expected")
dim(e.info4$item)
# if you don't class params, FI will assume a binary response model.
#########################
# Graded Response Model #
#########################
set.seed(999)
# generating random theta
theta <- rnorm(10)
# generating an item bank under a graded response model:
g.params <- cbind(a = runif(30, .5, 1.5), b1 = rnorm(30), b2 = rnorm(30),
b3 = rnorm(30), b4 = rnorm(30))
# you can sort the parameters yourself:
g.params <- cbind(g.params[ , 1],
t(apply(g.params[ ,2:dim(g.params)[2]], MARGIN = 1,
FUN = sort)))
# simulating responses using random theta:
g.mod <- simIrt(params = g.params, theta = theta, mod = "grm")
# calculating expected and observed information:
class(g.params) <- "grm" # always indicate model or extract from simulation.
e.info5 <- FI(params = g.params, theta = theta, type = "expected")
o.info5 <- FI(params = g.params, theta = theta, type = "observed", resp = g.mod$resp)
# grm, so e.info will not be equal to o.info:
all(signif(e.info5$item) == signif(o.info5$item))
# if thet is a vector and params is a vector, item will be a J x N matrix:
dim(e.info5$item)
# if you don't want to sort the parameters, you can extract from simIrt object:
e.info6 <- FI(params = g.mod$params[ , -1], theta = g.mod$theta, type = "expected")
# but you first need to remove column 1 (the item number column).
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.