Nothing
#' Response-refit simulations for a fitted GPvam model
#'
#' Quantify fixed-effect estimation error under a specified conditional generating
#' model, re-estimating covariance and persistence parameters for every response.
#'
#' @param model A converged GPvam fit created with version 3.4-0 or later.
#' @param data The original input data frame, including missing-score rows.
#' Student, teacher and year keys, row names, observed-score pattern, fitted
#' responses and fixed-effect design must match the supplied fit.
#' @param nsim Number of independent simulations, an integer at least 2.
#' @param beta Generating fixed-effect vector, defaulting to the fitted vector.
#' Named vectors must have exactly the fitted names and order.
#' @param k_list Named list of contrast vectors in the fitted coefficient order.
#' A single numeric vector is also accepted. NULL examines every coefficient.
#' @param generator Optional function taking model and replicate, returning
#' finite numeric vectors random_effects and residual_errors. Their orders are
#' those of model$eta.hat and model$observation, respectively. The default
#' holds fitted random effects fixed and draws centered Gaussian residuals
#' with the original fitted residual covariance on observed scores.
#' @param centered_errors Logical declaration that generating residuals have
#' mean zero, default TRUE for the default generator and FALSE otherwise.
#' @param refit_control Named list with optional max.iter.EM and tol1 overrides.
#' Defaults are the original fit's resolved values. Model structure, formula,
#' and ML/REML choice cannot be changed. Hessians are not recomputed.
#' @param verbose Print one progress line per replicate.
#'
#' @details
#' All R-side persistence types (GP, rGP, VP, CP, ZP) and the supported G-side GP
#' model are handled. The original design, score-missingness pattern, and known
#' teacher assignments on missing-score rows are fixed. Missing scores are not
#' imputed; their known assignments can still contribute to later scores. For
#' R-side models the original within-student residual correlations are retained.
#' G-side residuals have the fitted year-specific variances; the default also
#' holds the fitted student random effects fixed, along with teacher effects.
#' These fixed predictions are shrunken; refitted covariance estimates can differ
#' systematically from the original fit as part of this conditional target.
#' R-side defaults redraw within-student dependence through residuals, whereas
#' G-side defaults hold student effects fixed, so their targets differ.
#'
#' Responses are generated as y_s = X beta + h_s + e_s, where h_s = Z_0 eta_s
#' uses the ORIGINAL fitted random-effect design. Every refit uses the same
#' formula, persistence type, student side and ML/REML setting. For VP, the
#' persistence multipliers and hence the refitted design Z_s may change.
#' With fitted GLS map W_s, Alignment is k'W_s h_s, Residual is k'W_s e_s,
#' and Total is their sum. Estimation_Error records k'(beta_hat_s - beta), and
#' the pointwise decomposition is checked numerically. Internal_Diagnostic is
#' k'W_s Z_s eta_hat_s, using the REFITTED design and effects; it is not the
#' generated latent contribution. Alignment_Teacher and Alignment_Student split
#' Alignment for G-side fits. The student term is zero for R-side fits.
#'
#' If centered_errors is TRUE, Residual_CV is k'(W_s - W_0)e_s, where W_0 uses
#' the original fitted covariance. Its expectation equals that of Residual
#' because W_0 is fixed and the declared residual mean is zero. Total_CV is
#' Alignment + Residual_CV. These paired control variates often reduce Monte
#' Carlo uncertainty, but variance reduction is not guaranteed. Raw contributions
#' are always retained. MCSE is the empirical standard deviation of the actual
#' replicate values divided by the square root of the successful replicate count.
#'
#' Simulations are sequential and use R's RNG; use set.seed(). Custom generators
#' must produce independent replicates with finite variances for this MCSE
#' interpretation. A generator error or invalid return aborts the entire call;
#' completed replicates are not returned after this contract violation. Refit
#' error and nonconvergence rows remain explicit with missing
#' contributions. After failures, raw summaries describe successful refits only;
#' ALL control-variate summary means and MCSEs are withheld, because selection
#' of successful refits can invalidate their equal-expectation argument. An
#' Ill_Conditioned flag records a covariance block with eigenvalue ratio below
#' 1e-8 or a nonpositive eigenvalue; those converged refits are retained. This flag does not detect every
#' variance boundary (in particular, a tiny scalar variance has ratio one).
#' GLS and decomposition checks use a tolerance of 1e-6 times the larger of
#' one and the absolute values being compared; discrepancies are retained in
#' fit_status, and values beyond tolerance make the replicate a failure.
#'
#' This is a conditional sensitivity experiment under chosen generating values,
#' not an estimate of unconditional population bias or a calibrated bias test.
#' It does not generate an endogenous classroom-assignment or stochastic-covariate
#' mechanism. No automatic coefficient adjustment is supplied. A custom generator
#' changes latent/error draws at a fixed design; stochastic designs require an
#' external joint generating and refitting workflow.
#'
#' The original data are required explicitly and are never recovered by evaluating
#' the stored call. Older fitted objects must be refitted once with the current
#' GPvam to obtain validated simulation metadata. Formula dependence on y is
#' rejected, as are formula variables absent from data. Use the same collation
#' locale as for the original fit; differing locales are rejected before refits.
#' Output contains coefficient-level summaries and statuses, not input
#' rows, generated response vectors, fitted objects, or original identifiers.
#'
#' @return A list of class GPvam_simulation with summary, replicates, fit_status,
#' counts, beta, original_diagnostic, centered_errors, generator_type, and
#' model_spec. Use qualified function names
#' GPvam::simulate_bias and mixedbiastest::simulate_bias when both are attached.
#' @seealso \code{\link{GPvam}}, \code{\link{bias.test.custom}}
#' @examples
#' \donttest{
#' data(vam_data)
#' fit <- GPvam(vam_data, ~as.factor(year) + cont_var + 0,
#' persistence = "CP", verbose = FALSE)
#' set.seed(20261022)
#' # Two draws illustrate the API; use more to obtain useful Monte Carlo precision.
#' sim <- simulate_bias(fit, vam_data, nsim = 2)
#' sim$summary
#' }
#' @export
simulate_bias <- function(model, data, nsim = 100, beta = NULL,
k_list = NULL, generator = NULL,
centered_errors = is.null(generator),
refit_control = list(), verbose = FALSE) {
valid_vector <- function(x, size) {
is.numeric(x) && !is.complex(x) && is.null(dim(x)) &&
length(x) == size && all(is.finite(x))
}
if (!inherits(model, "GPvam") || !isTRUE(model$converged))
stop("model must be a converged GPvam fit.", call. = FALSE)
if (is.null(model$simulation_spec))
stop("Refit this model with GPvam 3.4-0 or later to obtain verified simulation metadata.", call. = FALSE)
if (!model$persistence %in% c("GP", "rGP", "VP", "CP", "ZP") ||
!model$student.side %in% c("R", "G") ||
(model$student.side == "G" && model$persistence != "GP"))
stop("Unsupported persistence/student-side combination.", call. = FALSE)
if (!valid_vector(nsim, 1L) || nsim < 2 || nsim != floor(nsim) ||
nsim > .Machine$integer.max) stop("nsim must be an integer at least 2.", call. = FALSE)
nsim <- as.integer(nsim)
if (!is.null(generator) && !is.function(generator)) stop("generator must be NULL or a function.")
for (nm in c("centered_errors", "verbose")) {
z <- get(nm)
if (!is.logical(z) || length(z) != 1L || is.na(z)) stop(nm, " must be TRUE or FALSE.")
}
.gpvam_simulation_validate_data(model, data)
if (!is.list(refit_control) || (length(refit_control) &&
(is.null(names(refit_control)) || anyDuplicated(names(refit_control)) ||
any(!names(refit_control) %in% c("max.iter.EM", "tol1")))))
stop("refit_control may contain only named max.iter.EM and tol1 values.")
control <- model$simulation_spec[c("max.iter.EM", "tol1")]
control[names(refit_control)] <- refit_control
if (!valid_vector(control$max.iter.EM, 1L) || control$max.iter.EM < 1 ||
control$max.iter.EM != floor(control$max.iter.EM) || control$max.iter.EM > .Machine$integer.max)
stop("max.iter.EM must be a positive integer.")
if (!valid_vector(control$tol1, 1L) || control$tol1 <= 0) stop("tol1 must be positive and finite.")
p <- ncol(model$X); n <- nrow(model$X); q <- ncol(model$Z)
fixed_names <- colnames(model$X)
if (is.null(beta)) beta <- stats::coef(model)
if (!valid_vector(beta, p) || (!is.null(names(beta)) && !identical(names(beta), fixed_names)))
stop("beta must follow the fitted coefficient names and order.")
if (is.null(k_list)) {
k_list <- lapply(seq_len(p), function(i) { z <- numeric(p); z[i] <- 1; z })
names(k_list) <- fixed_names
} else if (is.numeric(k_list)) k_list <- list(Contrast = k_list)
if (!is.list(k_list) || !length(k_list) || !all(vapply(k_list, valid_vector, logical(1), size = p)))
stop("k_list must contain finite contrast vectors in the fitted coefficient order.")
for (k in k_list) if (!is.null(names(k)) && !identical(names(k), fixed_names))
stop("Named contrasts must follow the fitted coefficient names and order.")
if (is.null(names(k_list))) names(k_list) <- paste0("Contrast", seq_along(k_list))
if (anyNA(names(k_list)) || any(!nzchar(names(k_list))) || anyDuplicated(names(k_list)))
stop("Contrast labels must be nonempty and unique.")
K <- do.call(rbind, k_list); labels <- names(k_list)
W0 <- K %*% .gpvam_simulation_weights(model)
# pivot=FALSE is essential: U'U = R.inv in the original observation order.
U <- .gpvam_simulation_residual_factor(model)
fixed_mean <- as.numeric(model$X %*% beta)
student_columns <- if (model$student.side == "G") seq_len(model$num.student) else integer()
teacher_columns <- setdiff(seq_len(q), student_columns)
original_diagnostic <- data.frame(Contrast = labels,
Internal_Diagnostic = as.numeric(W0 %*% (model$Z %*% model$eta.hat)))
components <- c("Alignment", "Residual", "Total", "Estimation_Error", "Internal_Diagnostic",
"Alignment_Teacher", "Alignment_Student", "Residual_CV", "Total_CV")
rows <- statuses <- vector("list", nsim)
for (i in seq_len(nsim)) {
draws <- if (is.null(generator)) list(random_effects = as.numeric(model$eta.hat),
residual_errors = as.numeric(solve(U, stats::rnorm(n)))) else generator(model, i)
if (!is.list(draws) || !valid_vector(draws$random_effects, q) ||
!valid_vector(draws$residual_errors, n))
stop("Generator replicate ", i, " must supply finite random_effects and residual_errors in fitted order.")
latent_teacher <- as.numeric(model$Z[, teacher_columns, drop = FALSE] %*% draws$random_effects[teacher_columns])
latent_student <- if (length(student_columns)) as.numeric(model$Z[, student_columns, drop = FALSE] %*%
draws$random_effects[student_columns]) else numeric(n)
latent <- latent_teacher + latent_student
simulated <- data
simulated$y[model$observation] <- fixed_mean + latent + draws$residual_errors
row <- data.frame(Replicate = i, Contrast = labels, stringsAsFactors = FALSE)
for (component in components) row[[component]] <- NA_real_
messages <- character(); started <- proc.time()[3]
fit <- tryCatch(withCallingHandlers(GPvam(simulated, fixed_effects = model$fixed_effects,
student.side = model$student.side, persistence = model$persistence,
max.iter.EM = control$max.iter.EM, tol1 = control$tol1,
hessian = FALSE, REML = model$REML, verbose = FALSE),
warning = function(w) { messages <<- c(messages, conditionMessage(w)); invokeRestart("muffleWarning") },
message = function(m) { messages <<- c(messages, conditionMessage(m)); invokeRestart("muffleMessage") }),
error = function(e) e)
status <- "ok"; ill_conditioned <- NA; identity_error <- NA_real_; gls_error <- NA_real_; iterations <- NA_integer_
if (inherits(fit, "error")) { status <- "error"; messages <- c(messages, conditionMessage(fit))
} else {
iterations <- fit$iter
if (!isTRUE(fit$converged)) status <- "nonconverged" else {
calculated <- tryCatch({
if (!identical(fit$observation, model$observation) ||
!identical(colnames(fit$X), fixed_names) ||
!isTRUE(all.equal(as.matrix(fit$X), as.matrix(model$X), tolerance = 1e-12)) ||
!identical(fit$teach.effects[c("teacher_year", "teacher", "effect_year")],
model$teach.effects[c("teacher_year", "teacher", "effect_year")]) ||
!identical(dim(fit$Z), dim(model$Z)))
stop("The refit changed the observed-row, fixed-effect or teacher-effect mapping.")
if (model$persistence != "VP" && max(abs(fit$Z - model$Z)) > 1e-12)
stop("The refit changed a fixed random-effect design.")
weights <- .gpvam_simulation_weights(fit)
gls_beta <- as.numeric(weights %*% fit$y)
gls_error <- max(abs(gls_beta - stats::coef(fit)))
if (!is.finite(gls_error) || gls_error > 1e-6 * max(1, abs(gls_beta), abs(stats::coef(fit))))
stop("Returned coefficients do not match GLS at the final returned covariance.")
W <- K %*% weights
row$Alignment_Teacher <- as.numeric(W %*% latent_teacher)
row$Alignment_Student <- as.numeric(W %*% latent_student)
row$Alignment <- row$Alignment_Teacher + row$Alignment_Student
row$Residual <- as.numeric(W %*% draws$residual_errors)
row$Total <- row$Alignment + row$Residual
row$Estimation_Error <- as.numeric(K %*% (stats::coef(fit) - beta))
row$Internal_Diagnostic <- as.numeric(W %*% (fit$Z %*% fit$eta.hat))
if (centered_errors) {
row$Residual_CV <- as.numeric((W - W0) %*% draws$residual_errors)
row$Total_CV <- row$Alignment + row$Residual_CV
}
active_components <- if (centered_errors) components else setdiff(components, c("Residual_CV", "Total_CV"))
if (any(!is.finite(as.matrix(row[, active_components, drop = FALSE]))))
stop("Nonfinite simulated component.")
identity_error <- max(abs(row$Total - row$Estimation_Error))
if (!is.finite(identity_error) || identity_error > 1e-6 * max(1, abs(row$Total), abs(row$Estimation_Error)))
stop("The per-replicate GLS decomposition failed its numerical identity check.")
ill_conditioned <- .gpvam_simulation_boundary(fit)
row
}, error = function(e) e)
if (inherits(calculated, "error")) {
status <- "error"; messages <- c(messages, conditionMessage(calculated))
for (component in components) row[[component]] <- NA_real_
} else row <- calculated
}
}
row$Status <- status; rows[[i]] <- row
statuses[[i]] <- data.frame(Replicate = i, Status = status, Ill_Conditioned = ill_conditioned,
Iterations = iterations, Seconds = unname(proc.time()[3] - started),
Decomposition_Error = identity_error, GLS_Coefficient_Error = gls_error,
Message = paste(unique(trimws(messages)), collapse = " | "), stringsAsFactors = FALSE)
if (verbose) { cat("Simulation", i, "of", nsim, ":", status, "\n"); flush.console() }
}
replicates <- do.call(rbind, rows); fit_status <- do.call(rbind, statuses)
success <- sum(fit_status$Status == "ok")
summary <- do.call(rbind, lapply(labels, function(label) {
d <- replicates[replicates$Contrast == label & replicates$Status == "ok", ]
do.call(rbind, lapply(components, function(component) {
values <- d[[component]]
available <- length(values) > 0 && all(is.finite(values)) &&
!(component %in% c("Residual_CV", "Total_CV") && success < nsim)
data.frame(Contrast = label, Component = component,
Mean = if (available) mean(values) else NA_real_,
MCSE = if (available && length(values) > 1) stats::sd(values)/sqrt(length(values)) else NA_real_,
N_Success = success, N_Failed = nsim - success, stringsAsFactors = FALSE)
}))
}))
counts <- data.frame(Requested = nsim, Success = success, Failed = nsim-success,
Ill_Conditioned = sum(fit_status$Ill_Conditioned, na.rm = TRUE),
With_Message = sum(nzchar(fit_status$Message)), CV_Summary_Available = centered_errors && success == nsim)
if (success < nsim) warning(nsim-success, " of ", nsim,
" refits failed or did not converge; raw summaries describe successful refits only.",
if (centered_errors) " CV summaries withheld." else "", call. = FALSE)
structure(list(summary = summary, replicates = replicates, fit_status = fit_status,
counts = counts, beta = beta, original_diagnostic = original_diagnostic,
centered_errors = centered_errors,
generator_type = if (is.null(generator)) "fixed predicted random effects; fitted Gaussian residuals" else "user-specified fixed-design scenario",
model_spec = list(persistence = model$persistence, student.side = model$student.side,
REML = model$REML, n_observations = n, n_random_effects = q,
n_student_effects = length(student_columns), refit_control = control)), class = "GPvam_simulation")
}
.gpvam_simulation_weights <- function(model) {
XtV <- crossprod(model$X, model$vinv)
as.matrix(solve(forceSymmetric(XtV %*% model$X), XtV))
}
.gpvam_simulation_validate_data <- function(model, data) {
spec <- model$simulation_spec
if (is.null(spec$collation) || !identical(spec$collation, Sys.getlocale("LC_COLLATE")))
stop("Use the same collation locale as the original fit, or refit with this GPvam version first.", call. = FALSE)
if (!is.data.frame(data) || anyDuplicated(names(data)) ||
!all(c("y", "student", "teacher", "year") %in% names(data)))
stop("data must be the original GPvam input data frame.")
if (!identical(data[, c("student", "teacher", "year"), drop = FALSE], spec$data_keys) ||
!identical(rownames(data), spec$row_names) || !identical(!is.na(data$y), spec$observed))
stop("Input student/year/teacher keys, row names or missing-score pattern differ from the fitted data.")
if (!is.numeric(data$y) || any(!is.finite(data$y[spec$observed])) ||
!isTRUE(all.equal(as.numeric(data$y[model$observation]), as.numeric(model$y), tolerance = 0)))
stop("Observed responses differ from the fitted data.")
if ("y" %in% all.vars(model$fixed_effects)) stop("Response-dependent fixed-effect formulas using y are not supported.")
if (length(setdiff(all.vars(model$fixed_effects), names(data))))
stop("All fixed-effect formula variables must be columns in data; external vectors are not supported.")
d <- as.data.frame(data)
d$year <- match(d$year, sort(unique(d$year)))
d$teacher <- as.character(d$teacher)
known <- !is.na(d$teacher)
d$teacher[known] <- paste0(d$teacher[known], "(year", d$year[known], ")")
d <- d[spec$observed, , drop = FALSE]
# Match each engine's formula-evaluation order, including order-dependent
# transforms. GP/rGP sort by year/teacher before forming X; G-side first
# sorts students, which determines tied year/teacher order.
if (model$student.side == "G") d <- d[order(d$student), , drop = FALSE]
if (model$persistence %in% c("GP", "rGP")) {
d <- d[order(d$year, d$teacher), , drop = FALSE]
} else d <- d[order(d$student, d$year, d$teacher), , drop = FALSE]
X <- sparse.model.matrix(model$fixed_effects, data = d, drop.unused.levels = TRUE)
X <- X[, !(colSums(abs(X)) == 0), drop = FALSE]
X <- as.matrix(X[match(rownames(model$X), rownames(X)), , drop = FALSE])
if (!identical(colnames(X), colnames(model$X)) ||
!identical(rownames(X), rownames(model$X)) ||
!isTRUE(all.equal(X, as.matrix(model$X), tolerance = 0, check.attributes = FALSE)))
stop("The fixed-effect design differs from the fitted data.")
invisible(TRUE)
}
.gpvam_simulation_boundary <- function(model) {
blocks <- model$teach.cov
if (model$student.side == "R") blocks <- c(blocks, list(model$R_i)) else
blocks <- c(blocks, list(model$stu.cov))
any(vapply(blocks, function(b) {
v <- eigen(as.matrix(b), symmetric = TRUE, only.values = TRUE)$values
min(v) <= 0 || min(v) < 1e-8 * max(v)
}, logical(1)))
}
.gpvam_simulation_residual_factor <- function(model) {
chol(forceSymmetric(model$R.inv), pivot = FALSE)
}
#' Print a GPvam response-refit simulation
#'
#' @param x An object returned by \code{\link{simulate_bias}}.
#' @param ... Additional arguments passed to \code{print}.
#' @return The object, invisibly.
#' @method print GPvam_simulation
#' @export
print.GPvam_simulation <- function(x, ...) {
cat("GPvam conditional response-refit simulation\n")
cat(x$model_spec$persistence, "persistence;", x$model_spec$student.side,
"student side;", if (x$model_spec$REML) "REML" else "ML", "\n")
print(x$counts, row.names = FALSE, ...)
cat("Component means and Monte Carlo SEs: $summary; individual draws: $replicates.\n")
cat("This conditional experiment does not identify unconditional population bias.\n")
invisible(x)
}
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.