In peterhalpin/cirt: Psychometric models of small group collaborations

knitr::opts_chunk$set(echo = TRUE)

Set up sesssion

devtools::install_github("peterhalpin/BearShare")
library("BearShare")
library("ltm")
library("ggplot2")
library("stats4")

Calibrate NAEP items using 2PL and save item parameters

calib_ltm <- ltm(calibration ~ z1)
beta <- coef(calib_ltm)[,1]
alpha <- coef(calib_ltm)[,2]

The friends dataset

Item responses from 7 pairs of friends working on two sets of items. Items names are formatted as follows: first 3 characters are the item number (1:100). The fourth character is the item difficulty, according to NAEP (E = easy, M = medium, H = hard). The fifth character is the item content area, according to NAEP (A = algebra, D = data, G = graphs, M = measurement, N = numbers). The _I and _C denote whether items were responded to by individuals working alone or in collaboration with their partner, respectively. Within the individual and collaborative forms, the items are ordered according to their difficulty in the calibration sample.

head(friends[,c(1:3,26, 30, 37, 50)])

Let's get the individual and collaborative forms formatted for factor scoring using ltm.

ind_form <- col_form <- friends[,-1]
ind_form[, grep("C", names(ind_form))] <- NA
col_form[, grep("I", names(col_form))] <- NA

Apply the conjunctive scoring rule to the collaborative form

odd <- seq(1, nrow(col_form), by = 2)
col_form[odd,] <- col_form[odd+1,] <- col_form[odd,]*col_form[odd+1,]

Estimate and plot abilities on individual and collaborative test forms. Any guesses about which pairs exemplify which models of collaboration??

ind_theta <- factor.scores(calib_ltm, ind_form, type = "EB", prior = F)$score.dat$z1
col_theta <- factor.scores(calib_ltm, col_form, type = "EB", prior = F)$score.dat$z1
barbell_plot(ind_theta, col_theta)

Set up and run LR tests for models. Note that nrow(resp) and length(col_theta) should correspond to the number of pairs; and that ncol(resp), alpha, and betashould only include items on the collaborative form (not the both forms, as used with ltm above). Use help(lr_test) to see a full list of arguments, or write lr_test to see the source code.

Do the conclusions based on the LR tests line up with the barbell plot? Any other considerations about this output?

parms <- coef(calib_ltm)
beta_C <- parms[grep("C", row.names(parms)), 1]
alpha_C <- parms[grep("C", row.names(parms)), 2]

resp <- col_form[odd, grep("C", names(col_form))]
models <- c("Ind", "Min", "Max", "AI")

lr <- lr_test(resp, models, alpha_C, beta_C, ind_theta, col_theta[odd], n_boot = 500)

lr

Here is the sanity check from the slides. The basic idea is to use the individual form and apply the conjunctive scoring rule. In this case We know the individual model must be correct -- but what does the LR test tell us?

beta_I <- parms[grep("I", row.names(parms)), 1]
alpha_I <- parms[grep("I", row.names(parms)), 2]

sanity_form  <- ind_form[odd,]*ind_form[odd+1,]

sanity_theta <- factor.scores(calib_ltm, 
                      sanity_form, 
                      type = "EB", 
                      prior = F)$score.dat$z1

sanity_resp <- sanity_form[, grep("I", names(sanity_form))]

sanity_lr <- lr_test(sanity_resp, 
                    models, 
                    alpha_I, 
                    beta_I, 
                    ind_theta, 
                    sanity_theta, 
                    n_boot = 500)

sanity_lr

Suggestions for further analyses: Use sim_data to generate data from a model (help(sim_data)). Use the pilot dataset to run an analysis for some of the data that I'll talk about this afternoon and tomorrow.

peterhalpin/cirt documentation built on May 25, 2019, 1:50 a.m.