This function allows users to rely on the powerful
caret package for
cross-validating and tuning a rma analysis. Methods for rma are not included
in the caret package, because the interface of caret is not entirely
compatible with rma's model call. Specifically, rma is not compatible with
train methods for classes 'formula' or 'recipe'. The variance of
the effect sizes can be passed to the 'weights' parameter of
When using clustered data (effect sizes within studies), make sure to use 'index = groupKFold(your_study_id_variable, k = 10))' in traincontrol, to sample by study ID when creating cross-validation partitions; otherwise the testing error will be positively biased.
ModelInfo list of length 13.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
## Not run: # Prepare data dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg) dat$yi <- as.numeric(dat$yi) dat$alloc <- factor(dat$alloc) # Run rma rma.model <- rma(y = dat$yi, mods = dat[, c("ablat", "year")], vi = dat$vi) # R^2 is estimated to be .64 rma.model$R2 # Now, use cross-validation to see how well this model generalizes # Leave-one-out cross-validation is more appropriate than 10-fold cv because # the sample size is very small fit_control <- trainControl(method = "LOOCV") # Train the model without tuning, because rma has no tuning parameters cv.mf.cluster <- train(y = dat$yi, x = dat[, c("ablat", "year")], weights = dat$vi, method = ModelInfo_rma(), trControl = fit_control) # Cross-validated R^2 is .08, suggesting substantial overfitting of the # original rma model cv.mf.cluster$results$Rsquared ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.