This package contains functions for criterion-referenced test and item analysis.
The functions below can be used to examine the dependability of single administration CRT assessments.
The subkoviak
function follows Subkoviak's (1988) single administration procedures for estimating $\kappa$ and p~o~. To use the subkoviak
function the data can come in one of two forms:
In both situations, the analyst/researcher needs to know the raw cut score of the test.
The procedure uses two pieces of information to estimate $\kappa$ and p~o~. They are a reliability coefficient (estimated in this function using Kuder-Richardson 20 or 21) and a standardized cut-score, which is estimated following Subkoviak (1988). The estimated $\alpha$ coefficient and standardized cut-score are used to look up the values for $\kappa$ and p~o~ in look up tables, which the function will return if look_up = TRUE
$$ z = \frac{c - 0.5 - M}{S}$$
where:
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", echo=TRUE, warning=FALSE, message=FALSE, error=FALSE)
# Load the library library(rcrtan) # Some data test_data <- tibble::tribble( ~ID, ~Q1, ~Q2, ~Q3, ~Q4, ~Q5, ~Q6, ~Q7, ~Q8, ~Q9, ~Q10, ~Total, 1441L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 10L, 1387L, 1L, 1L, 1L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 8L, 1994L, 1L, 1L, 1L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 8L, 1453L, 1L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L, 8L, 1679L, 1L, 0L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 1L, 7L, 1899L, 1L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L, 8L, 1631L, 1L, 1L, 1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 9L, 1894L, 1L, 0L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 9L, 1206L, 0L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 0L, 1L, 5L, 1163L, 1L, 1L, 1L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 8L, 1964L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 10L, 1050L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 10L, 1034L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 10L, 1826L, 1L, 0L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 9L, 1973L, 1L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L, 8L, 1936L, 0L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 0L, 8L, 1114L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 5L, 1181L, 0L, 1L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 7L, 1917L, 1L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L, 8L ) # Analyze the dichomous data. Use look_up = TRUE if you want to see the look up tables from Subkoviak (1988). sub_ex_one <- subkoviak(data = test_data, items = 2:11, raw_cut_score = 8)
The output from the analysis contains five variables.
sub_ex_one
When item-level information is not available the items argument takes the number of dichotomous items on the test, and the total argument takes the name of the column in the dataframe with the total score.
sub_ex_two <- subkoviak(data = test_data, items = 10, raw_cut_score = 8, total = "Total") sub_ex_two
The phi_domain
function calculates Brown (1988) short-cut estimate of the generalizability coefficient ($\phi$ dependability) for absolute decisions.
# When item level information is known phi_d_one <- phi_domain(data = test_data, items = 2:11) # When only total scores and number of items on the test are known phi_d_two <- phi_domain(data = test_data, items = 10, total = "Total") phi_d_one phi_d_two
The phi_lambda
function uses Brennan (1984) formula to estimate the dependability of a test. One difference to note between this function and subkoviak
is that the cut_score argument is expressed as a proportion.
phi_l_one <- rcrtan::phi_lambda(test_data, 2:11, cut_score = 0.80) phi_l_two <- phi_lambda(test_data, 10, cut_score = 0.80, total = 'Total') phi_l_one phi_l_two
Brennan, R. L. (1984). Estimating the dependability of the scores. In A guide to criterion-referenced test construction, (Ed.) Ronald A Berk. 231–66. Baltimore, MD: The Johns Hopkins University Press.
Brown, J. D. (1988). Short cut estimates of criterion-referenced test reliability. Language Testing, 7, 77–97.
Subkoviak, M. J. (1988). A practitioner’s guide to computation and interpretation of reliability indices for mastery tests. Journal of Educational Measurement, 25, 47–55.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.