kmassess: Perform a probabilistic knowledge assessment

In kstMatrix: Basic Functions in Knowledge Space Theory Using Matrix Representation

Perform a probabilistic knowledge assessment

Description

kmassess performs a probabilistic knowledge assessment for a given response vector, knowledge structure, and BLIM parameters.

kmsassess performs a simplified probabilistic knowledge assessment for a given response vector, knowledge structure, and BLIM parameters. It assumes an equal probability distribution over the knowledge structure as starting point and identical beta and eta values for all items.

Usage

kmassess(
  r,
  pks,
  questioning,
  update,
  beta,
  eta,
  zeta0,
  zeta1,
  threshold,
  probdev = FALSE
)

kmsassess(
  r,
  ks,
  questioning,
  update,
  beta,
  eta,
  zeta0,
  zeta1,
  threshold,
  probdev = FALSE
)

Arguments

r	Response pattern (binary vector)
pks	Probabilistic knowledge structure: a data frame with a probability distribution in the first columns and the structure matrix in the subsequent columns.
questioning	Questioning rule ("halfsplit" o "informative")
update	Update rule ("Bayesian" or "multiplicative")
beta	Careless error probability
eta	Lucky guess probability
zeta0	Update parameter for wrong responses
zeta1	Update parameter for correct responses
threshold	Probability threshold for stopping criterion
probdev	Provide information on the probability development including Hasse diagrams stored in tempdir(). Defaults to FALSE.
ks	Knowledge structure: a binary matrix

Details

kmassess implements the stochastic assessment procedures according to Doignon & Falmagne, 1999, chapter 10.

kmassess stops if the number of questions has reached twice the number of items.

Value

A list with the following elements:

state

Diagnosed knowledge state (binary vector)

probs

Resulting probability distribution. If probdev is set to TRUE, a list of probability distributions for each step is given instead.

queried

Sequence of items used in the assessment (list)

qtime

Average time for finding a question

utime

Average time for updating the probabilities

A list with the following elements:

state

Diagnosed knowledge state (binary vector)

probs

Resulting probability distribution. If probdev is set to TRUE, a list of probability distributions for each step is given instead.

queried

Sequence of items used in the assessment (list)

qtime

Average time for finding a question

utime

Average time for updating the probabilities

Background

Doignon & Falmagne (1985, 1999) proposed knowledge space theory originally with adaptive knowledge assessment in mind. The basic idea is to apply prerequisite relationships between items for reducing the number of problems to be posed to a learner in knowledge assessment.

Falmagne & Doignon (1988; Doignon & Falmange, 1999, chapte 10) proposed a class of stochastic procdures for such adaptive assessment which take into account that careless errors and lucky guesses may happen during the assessment by estimating a probability distribution over the knowledge structure. Such an assessment consists of three important parts

• Question rule

• Update rule

• Stopping criterion

For the question rule, they propose the halfsplit and the infomrative rules, implemented in kmassesshalfslit and kmassessinfomrative.

For the update rule, they again propose two possibilities there the multiplicative rule is a generalisation of the (classical) Bayesian update rule implemented here in kmassessmultiplicative and kmassessbayesian, respectively.

As stopping criterion, usually a threshold for the maximal probability for one knowledge state is used. It is strongly recommended to keep this larger than 0.5 in order to have one unequivocal resulting state (see also Hockemeyer, 2002).

Framework of assessment functions within the kstMatrix package:

The founding stones are the four aforementioned functions for finding suitable questions and for updating the probability estimates, respectively. They could also be used in an interactive system, e.g. a Shiny app, for "real" adaptive assessment.

The remaining thee assessment functions serve for mere simulation of adaptive assessment. kmassess takes, among others, a full response pattern as parameter and takes the responses for the selected questions from this vector. kmsassess is a simplified version where the update parameters (beta and eta for Bayesian or zeta0 and zeta1 for multiplicative update, respectively) are identical for all items whereas they are item-specific in kmassess. Finally, kmassesssimulation takes a whole data set, i.e. a collection of response patterns, and does an assessment for each of these patterns. Its result is a data frame which should be suitable for further statistical evaluation, especially if it is called several times with variant parameters (e.g., structures, update parameters, update and question rules).

Both, kmsassess and kmassesssimulation call kmassess.

Problems

In rare cases kmassess may flip forth and back between probability distributions resulting in an endless loop. Therefore, it stops after twice the number of items delivering a NULL result.

References

Doignon, J.-P. & Falmagne, J.-C. (1985). Spaces for the assessment of knowledge. International Journal of Man-Machne-Studies, 23, 175-196. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/S0020-7373(85)80031-6")}.

Doignon, J.-P. & Falmagne, J.-C. (1999). Knowledge Spaces. Springer Verlag, Berlin. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1007/978-3-642-58625-5")}.

Falmagne, J.-C. & Doignon, J.-P. (1988). A class of stochastic procedures for the assessment of knowledge. British Journal of Mathematical and Statistical Psychology, 41, 1-23. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1111/j.2044-8317.1988.tb00884.x")}.

Hoxkemeyer, C. (2002). A comparison of non-deterministic procedures for the adaptive assessment of knowledge. Psychlogische Beiträge, 44(4), 495-503.

See Also

Other Knowledge assessment: kmassessbayesian(), kmassesshalfsplit(), kmassessinformative(), kmassessmentsimulation(), kmassessmultiplicative()

Other Knowledge assessment: kmassessbayesian(), kmassesshalfsplit(), kmassessinformative(), kmassessmentsimulation(), kmassessmultiplicative()

Examples

kmassess(c(1, 1, 0, 0),
         cbind(as.data.frame(as.matrix(rep(1/9.0, 9), ncol=1)), xpl$space),
         "halfsplit",
         "Bayesian",
         rep(0.12, 4),
         rep(0.1, 4),
         NULL,
         NULL,
         0.55
        )

kmsassess(c(1,1,0,0), xpl$space, "halfsplit", "Bayesian", 0.1, 0.1, NULL, NULL, 0.55)

kstMatrix documentation built on Dec. 18, 2025, 5:07 p.m.