LanguageData: Accuracy and Expected Accuracy Matrixes for Langauge Test.

In ralmond/CPTtools: Tools for Creating Conditional Probability Tables

Accuracy and Expected Accuracy Matrixes for Langauge Test.

Description

This is an example used in Chapter 7 of Almond, et. al. (2015), based on a conceptual assessment described in Mislevy (1995). The assessment has four reporting variables: _Reading_, _Writing_, _Speaking_, and _Listening_, each of which can take on states 'Novice', 'Intermediate', or 'Advanced'. To estimate the reliability 1000 students were simulated from the network (see NetworkTester and accuracy, using Maximum A Prior estimates, ('Langauge_modal') and expected accuracy ('Language_exp') estimates were calculated.

Usage

data("Language_modal")
data("Language_exp")
data("language16")
data("language24")

Format

The format for both 'Language_modal' and 'Language_exp' is: List of 4 $ Reading A 3x3 matrix giving the accuracy of the Reading measure. $ Writing A 3x3 matrix giving the accuracy of the Writing measure. $ Speaking A 3x3 matrix giving the accuracy of the Speaking measure. $ Listening A 3x3 matrix giving the accuracy of the Listening measure.

All cases, the table has been normalized so that all entries sum to 1.

The data sets 'language16' and 'language24' are simulated data (1000 cases each) from both the 16 task test described above, and a 24 task variant which has 6 copies of each task. Both data frames have the following columns (not necessarily in this order).

IDnum

A case number.

Listening,Speaking,Writing,Reading

The "true" (simulated) ability.

TaskD,TaskD1,TaskD2,TaskD3,TaskD4,TaskD5

The simulated responses for the type D (Listening) tasks. 'TaskD5' is only in 'language24'.

TaskC,TaskC1,TaskC2,TaskC3,TaskC4,TaskC5

The simulated reponses for the type C (Speaking) tasks. 'TaskC3', 'TaskC4', and 'TaskC5' are only in the 'langauge24' data set.

TaskB,TaskB1,TaskB2,TaskB3,TaskB4,TaskB5

The simulated responses for the type B (Writing) tasks. 'TaskB3', 'TaskB4', and 'TaskB5' are only in the 'language24' data set.

TaskA,TaskA1,TaskA2,TaskA3,TaskA4,TaskA5

The simulated responses for the type A (Reading) tasks. 'TaskA5' is only available in the 'langauge24' data set.

Listening.Novice,Listening.Intermediate,Listening.Advanced

The estimated probability that this subject is in the 'Novice', 'Intermediate' and 'Advanced' categories respectively of the Listening skill.

Speaking.Novice,Speaking.Intermediate,Speaking.Advanced

The estimated probability that this subject is in the 'Novice', 'Intermediate' and 'Advanced' categories respectively of the Speaking skill.

Writing.Novice,Writing.Intermediate,Writing.Advanced

The estimated probability that this subject is in the 'Novice', 'Intermediate' and 'Advanced' categories respectively of the Writing skill.

Reading.Novice,Reading.Intermediate,Reading.Advanced

The estimated probability that this subject is in the 'Novice', 'Intermediate' and 'Advanced' categories respectively of the Reading skill.

mode.Listening,mode.Speaking,mode.Writing,mode.Reading

The most likely state of the skill variables given the relevant data.

The simulation code used to build 'language16' and 'language24' is in 'vignette("SimulationStudies",package="RNetica")'.

Details

The sample language assessment contains:

A

5 Reading tasks which only tap the Reading attribute.

B

3 Reading/Writing integrated tasks which require a written response to a textual prompt.

C

3 Reading/Speaking/Listening Integrated tasks which require a spoken response to an aural stimulus, with textual instructions.

D

5 Listening tasks which have both aural stimulus and instructions.

Because different numbers of tasks (which have different strengths of evidence) are available, different amounts of information are available about each of the four target variables. A number of different measures of reliability can be calculated from the (expected) accuracy matrixes.

Source

Almond, R.G., Mislevy, R.J., Steinberg, L.S., Williamson, D.M. and Yan, D. (2015) Bayesian Networks in Educational Assessment. Springer. Chapter 7.

References

Mislevy, R. J. (1995) Test theory and language learning in assessment. Language Testing, 12, 341–369.

Examples

data(Language_modal)
fcKappa(Language_modal$Reading)
gkLambda(Language_modal$Reading)

ralmond/CPTtools documentation built on Dec. 27, 2024, 7:15 a.m.