powerChange: Power analysis of tests in context of measurement of change...
In tcl: Testing in Conditional Likelihood Context
powerChange R Documentation
Power analysis of tests in context of measurement of change using LLTM
Description
Returns power of Wald (W), likelihood ratio (LR), Rao score (RS)
and gradient (GR) test given probability of error of first kind \alpha, sample size, and
a deviation from the hypothesis to be tested. The latter states that the shift parameter
quantifying the constant change for all items between time points 1 and 2 equals 0.
The alternative states that the shift parameter is not equal to 0.
It is assumed that the same items are presented at both time points. See function change_test.
Usage
powerChange(alpha = 0.05, n_total, eta, persons = rnorm(10^6))
Arguments
alpha
Probability of the error of first kind.
n_total
Total sample size for which power shall be determined.
eta
A vector of eta parameters of the LLTM. The last element represents the constant change or shift for all items
between time points 1 and 2. The other elements of the vector are the item parameters at time point 1. A choice of the eta
parameters constitutes a scenario of deviation from the hypothesis of no change.
persons
A vector of person parameters (drawn from a specified distribution). By default 10^6 parameters are drawn at
random from the standard normal distribution. The larger this number the more accurate are the computations. See Details.
Details
In general, the power of the tests is determined from the assumption that the approximate distributions of
the four test statistics are from the family of noncentral \chi^2 distributions with df = 1 and noncentrality
parameter \lambda. The latter depends on a scenario of deviation from the hypothesis to be tested and a specified sample size.
Given the probability of the error of the first kind \alpha the power of the tests can be determined from \lambda.
More details about the distributions of the test statistics and the relationship between \lambda, power, and sample size can be found
in Draxler and Alexandrowicz (2015).
As regards the concept of sample size a distinction between informative and total sample size has to be made since the power
of the tests depends only on the informative sample size. In the conditional maximum likelihood context, the responses of
persons with minimum or maximum person score are completely uninformative. They do not contribute to the value of the test
statistic. Thus, the informative sample size does not include these persons. The total sample size is composed of all persons.
In particular, the determination of \lambda and the power of the tests, respectively, is based on a simple Monte Carlo approach.
Data (responses of a large number of persons to a number of items presented at two time points) are generated given a
user-specified scenario of a deviation from the hypothesis to be tested. The hypothesis to be tested assumes no change
between time points 1 and 2. A scenario of a deviation is given by a choice of the item parameters at time point 1 and
the shift parameter, i.e., the LLTM eta parameters, as well as the person parameters (to be drawn randomly from a specified
distribution). The shift parameter represents a constant change of all item parameters from time point 1 to time point 2.
A test statistic T (Wald, LR, score, or gradient) is computed from the simulated data. The observed value t of the test
statistic is then divided by the informative sample size n_{infsim} observed in the simulated data. This yields the so-called
global deviation e = t / n_{infsim}, i.e., the chosen scenario of a deviation from the hypothesis to be tested being represented
by a single number. The power of the tests can be determined given a user-specified total sample size denoted by n_{total}.
The noncentrality parameter \lambda can then be expressed by \lambda = n_{total}* (n_{infsim} / n_{totalsim}) * e,
where n_{totalsim} denotes the total number of persons in the simulated data and n_{infsim} / n_{totalsim} is the proportion of
informative persons in the sim. data. Let q_{\alpha} be the 1 - \alpha quantile of the central \chi^2 distribution with df = 1.
Then,
power = 1 - F_{df, \lambda} (q_{\alpha}),
where F_{df, \lambda} is the cumulative distribution function of the noncentral \chi^2 distribution with df = 1 and
\lambda = n_{total} * (n_{infsim} / n_{totalsim}) * e. Thereby, it is assumed that n_{total} is composed of a frequency distribution
of person scores that is proportional to the observed distribution of person scores in the simulated data.
Note that in this approach the data have to be generated only once. There are no replications needed. Thus, the procedure is
computationally not very time-consuming.
Since e is determined from the value of the test statistic observed in the simulated data it has to be treated as a realized
value of a random variable E. The same holds true for \lambda as well as the power of the tests. Thus, the power is a realized
value of a random variable that shall be denoted by P. Consequently, the (realized) value of the power of the tests need
not be equal to the exact power that follows from the user-specified n_{total}, \alpha, and the chosen item parameters and shift
parameter used for the simulation of the data. If the CML estimates of these parameters computed from the simulated data are
close to the predetermined parameters the power of the tests will be close to the exact value. This will generally be the
case if the number of person parameters used for simulating the data is large, e.g., 10^5 or even 10^6 persons. In such
cases, the possible random error of the computation procedure based on the sim. data may not be of practical relevance
any more. That is why a large number (of persons for the simulation process) is generally recommended.
For theoretical reasons, the random error involved in computing the power of the tests can be pretty well approximated.
A suitable approach is the well-known delta method. Basically, it is a Taylor polynomial of first order, i.e., a linear
approximation of a function. According to it the variance of a function of a random variable can be linearly approximated
by multiplying the variance of this random variable with the square of the first derivative of the respective function.
In the present problem, the variance of the test statistic T is (approximately) given by the variance of a noncentral
\chi^2 distribution. Thus, Var(T) = 2 (df + 2 \lambda),
with df = 1 and \lambda = t.
Since the global deviation e = (1 / n_{infsim})* t it follows for the variance of the corresponding random variable E
that Var(E) = (1 / n_{infsim})^2 * Var(T). The power of the tests is a function of e which is given by
F_{df, \lambda} (q_{\alpha}), where \lambda = n_{total} * (n_{infsim} / n_{totalsim}) * e and df = 1.
Then, by the delta method one obtains (for the variance of P)
Var(P) = Var(E) * (F'_{df, \lambda} (q_{\alpha}))^2,
where F'_{df, \lambda} is the derivative of F_{df, \lambda} with respect to e. This derivative is determined
numerically and evaluated at e using the package numDeriv. The square root of Var(P) is then used to quantify the random
error of the suggested Monte Carlo computation procedure. It is called Monte Carlo error of power.
Value
A list of results.
power
Power value for each test.
MC error of power
Monte Carlo error of power computation for each test.
deviation
Shift parameter estimated from the simulated data representing the constant shift of item parameters between time points 1 and 2.
person score distribution
Relative frequencies of person scores observed in simulated data. Uninformative scores,
i.e., minimum and maximum score, are omitted. Note that the person score distribution does also have an influence on the power of the tests.
degrees of freedom
Degrees of freedom df.
noncentrality parameter
Noncentrality parameter \lambda of \chi^2 distribution from which power is determined.
call
The matched call.
References
Draxler, C., & Alexandrowicz, R. W. (2015). Sample size determination within the scope of conditional
maximum likelihood estimation with special focus on testing the Rasch model. Psychometrika, 80(4), 897-919.
Fischer, G. H. (1995). The Linear Logistic Test Model. In G. H. Fischer & I. W. Molenaar (Eds.),
Rasch models: Foundations, Recent Developments, and Applications (pp. 131-155). New York: Springer.
Fischer, G. H. (1983). Logistic Latent Trait Models with Linear Constraints. Psychometrika, 48(1), 3-26.
See Also
sa_sizeChange, and post_hocChange.
Examples
## Not run:
# Numerical example: 4 items presented twice, thus 8 virtual items
# eta Parameter, first 4 are nuisance
# (easiness parameters of the 4 items at time point 1),
# last one is the shift parameter
eta <- c(-2,-1,1,2,0.5)
res <- powerChange(alpha = 0.05, n_total=150, eta=eta, persons=rnorm(10^6))
# > res
# $power
# W LR RS GR
# 0.905 0.910 0.908 0.911
#
# $`MC error of power`
# W LR RS GR
# 0.002 0.002 0.002 0.002
#
# $`deviation (estimate of shift parameter)`
# [1] 0.499
#
# $`person score distribution`
#
# 1 2 3 4 5 6 7
# 0.034 0.093 0.181 0.249 0.228 0.147 0.068
#
# $`degrees of freedom`
# [1] 1
#
# $`noncentrality parameter`
# W LR RS GR
# 10.692 10.877 10.815 10.939
#
# $call
# powerChange(alpha = 0.05, n_total = 150, eta = eta, persons = rnorm(10^6))
#
## End(Not run)
tcl documentation built on May 3, 2023, 1:17 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| powerChange | R Documentation |
Power analysis of tests in context of measurement of change using LLTM
Description
Returns power of Wald (W), likelihood ratio (LR), Rao score (RS)
and gradient (GR) test given probability of error of first kind \alpha, sample size, and
a deviation from the hypothesis to be tested. The latter states that the shift parameter
quantifying the constant change for all items between time points 1 and 2 equals 0.
The alternative states that the shift parameter is not equal to 0.
It is assumed that the same items are presented at both time points. See function change_test.
Usage
powerChange(alpha = 0.05, n_total, eta, persons = rnorm(10^6))
Arguments
alpha |
Probability of the error of first kind. |
n_total |
Total sample size for which power shall be determined. |
eta |
A vector of eta parameters of the LLTM. The last element represents the constant change or shift for all items between time points 1 and 2. The other elements of the vector are the item parameters at time point 1. A choice of the eta parameters constitutes a scenario of deviation from the hypothesis of no change. |
persons |
A vector of person parameters (drawn from a specified distribution). By default |
Details
In general, the power of the tests is determined from the assumption that the approximate distributions of
the four test statistics are from the family of noncentral \chi^2 distributions with df = 1 and noncentrality
parameter \lambda. The latter depends on a scenario of deviation from the hypothesis to be tested and a specified sample size.
Given the probability of the error of the first kind \alpha the power of the tests can be determined from \lambda.
More details about the distributions of the test statistics and the relationship between \lambda, power, and sample size can be found
in Draxler and Alexandrowicz (2015).
As regards the concept of sample size a distinction between informative and total sample size has to be made since the power of the tests depends only on the informative sample size. In the conditional maximum likelihood context, the responses of persons with minimum or maximum person score are completely uninformative. They do not contribute to the value of the test statistic. Thus, the informative sample size does not include these persons. The total sample size is composed of all persons.
In particular, the determination of \lambda and the power of the tests, respectively, is based on a simple Monte Carlo approach.
Data (responses of a large number of persons to a number of items presented at two time points) are generated given a
user-specified scenario of a deviation from the hypothesis to be tested. The hypothesis to be tested assumes no change
between time points 1 and 2. A scenario of a deviation is given by a choice of the item parameters at time point 1 and
the shift parameter, i.e., the LLTM eta parameters, as well as the person parameters (to be drawn randomly from a specified
distribution). The shift parameter represents a constant change of all item parameters from time point 1 to time point 2.
A test statistic T (Wald, LR, score, or gradient) is computed from the simulated data. The observed value t of the test
statistic is then divided by the informative sample size n_{infsim} observed in the simulated data. This yields the so-called
global deviation e = t / n_{infsim}, i.e., the chosen scenario of a deviation from the hypothesis to be tested being represented
by a single number. The power of the tests can be determined given a user-specified total sample size denoted by n_{total}.
The noncentrality parameter \lambda can then be expressed by \lambda = n_{total}* (n_{infsim} / n_{totalsim}) * e,
where n_{totalsim} denotes the total number of persons in the simulated data and n_{infsim} / n_{totalsim} is the proportion of
informative persons in the sim. data. Let q_{\alpha} be the 1 - \alpha quantile of the central \chi^2 distribution with df = 1.
Then,
power = 1 - F_{df, \lambda} (q_{\alpha}),
where F_{df, \lambda} is the cumulative distribution function of the noncentral \chi^2 distribution with df = 1 and
\lambda = n_{total} * (n_{infsim} / n_{totalsim}) * e. Thereby, it is assumed that n_{total} is composed of a frequency distribution
of person scores that is proportional to the observed distribution of person scores in the simulated data.
Note that in this approach the data have to be generated only once. There are no replications needed. Thus, the procedure is computationally not very time-consuming.
Since e is determined from the value of the test statistic observed in the simulated data it has to be treated as a realized
value of a random variable E. The same holds true for \lambda as well as the power of the tests. Thus, the power is a realized
value of a random variable that shall be denoted by P. Consequently, the (realized) value of the power of the tests need
not be equal to the exact power that follows from the user-specified n_{total}, \alpha, and the chosen item parameters and shift
parameter used for the simulation of the data. If the CML estimates of these parameters computed from the simulated data are
close to the predetermined parameters the power of the tests will be close to the exact value. This will generally be the
case if the number of person parameters used for simulating the data is large, e.g., 10^5 or even 10^6 persons. In such
cases, the possible random error of the computation procedure based on the sim. data may not be of practical relevance
any more. That is why a large number (of persons for the simulation process) is generally recommended.
For theoretical reasons, the random error involved in computing the power of the tests can be pretty well approximated.
A suitable approach is the well-known delta method. Basically, it is a Taylor polynomial of first order, i.e., a linear
approximation of a function. According to it the variance of a function of a random variable can be linearly approximated
by multiplying the variance of this random variable with the square of the first derivative of the respective function.
In the present problem, the variance of the test statistic T is (approximately) given by the variance of a noncentral
\chi^2 distribution. Thus, Var(T) = 2 (df + 2 \lambda),
with df = 1 and \lambda = t.
Since the global deviation e = (1 / n_{infsim})* t it follows for the variance of the corresponding random variable E
that Var(E) = (1 / n_{infsim})^2 * Var(T). The power of the tests is a function of e which is given by
F_{df, \lambda} (q_{\alpha}), where \lambda = n_{total} * (n_{infsim} / n_{totalsim}) * e and df = 1.
Then, by the delta method one obtains (for the variance of P)
Var(P) = Var(E) * (F'_{df, \lambda} (q_{\alpha}))^2,
where F'_{df, \lambda} is the derivative of F_{df, \lambda} with respect to e. This derivative is determined
numerically and evaluated at e using the package numDeriv. The square root of Var(P) is then used to quantify the random
error of the suggested Monte Carlo computation procedure. It is called Monte Carlo error of power.
Value
A list of results.
power |
Power value for each test. |
MC error of power |
Monte Carlo error of power computation for each test. |
deviation |
Shift parameter estimated from the simulated data representing the constant shift of item parameters between time points 1 and 2. |
person score distribution |
Relative frequencies of person scores observed in simulated data. Uninformative scores, i.e., minimum and maximum score, are omitted. Note that the person score distribution does also have an influence on the power of the tests. |
degrees of freedom |
Degrees of freedom |
noncentrality parameter |
Noncentrality parameter |
call |
The matched call. |
References
Draxler, C., & Alexandrowicz, R. W. (2015). Sample size determination within the scope of conditional maximum likelihood estimation with special focus on testing the Rasch model. Psychometrika, 80(4), 897-919.
Fischer, G. H. (1995). The Linear Logistic Test Model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, Recent Developments, and Applications (pp. 131-155). New York: Springer.
Fischer, G. H. (1983). Logistic Latent Trait Models with Linear Constraints. Psychometrika, 48(1), 3-26.
See Also
sa_sizeChange, and post_hocChange.
Examples
## Not run:
# Numerical example: 4 items presented twice, thus 8 virtual items
# eta Parameter, first 4 are nuisance
# (easiness parameters of the 4 items at time point 1),
# last one is the shift parameter
eta <- c(-2,-1,1,2,0.5)
res <- powerChange(alpha = 0.05, n_total=150, eta=eta, persons=rnorm(10^6))
# > res
# $power
# W LR RS GR
# 0.905 0.910 0.908 0.911
#
# $`MC error of power`
# W LR RS GR
# 0.002 0.002 0.002 0.002
#
# $`deviation (estimate of shift parameter)`
# [1] 0.499
#
# $`person score distribution`
#
# 1 2 3 4 5 6 7
# 0.034 0.093 0.181 0.249 0.228 0.147 0.068
#
# $`degrees of freedom`
# [1] 1
#
# $`noncentrality parameter`
# W LR RS GR
# 10.692 10.877 10.815 10.939
#
# $call
# powerChange(alpha = 0.05, n_total = 150, eta = eta, persons = rnorm(10^6))
#
## End(Not run)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.