cv_summary: Summarise the quality of a set of cross-validation folds

In blockCV: Spatial and Environmental Blocking for Cross-Validation

Summarise the quality of a set of cross-validation folds

Description

A one-call diagnostic that gathers the fold-quality signals scattered across blockCV into a single object: fold sizes and class prevalence (always), the test-to-nearest-train distance and environmental-novelty diagnostics (when a raster or prediction domain is supplied), and a list of automatically detected warnings about degenerate folds.

Usage

cv_summary(
  cv,
  x = NULL,
  r = NULL,
  pred_points = NULL,
  model_domain = NULL,
  method = "MESS",
  num_sample = 10000L,
  min_test = 5L,
  seed = NULL,
  progress = FALSE
)

Arguments

cv	a blockCV cross-validation object, i.e. the output of cv_spatial, cv_cluster, cv_group, cv_buffer, cv_nndm or cv_knndm.
x	a simple features (sf) object of the sample points used to create cv. Required only for the distance/novelty diagnostics (when r, pred_points or model_domain is supplied).
r	a terra SpatRaster of the environmental covariates. When supplied, both the distance and the environmental-novelty diagnostics are computed. Optional.
pred_points, model_domain	optional prediction points or a model-domain polygon used by the distance diagnostic; see cv_distance. Supplying either (without r) adds the distance diagnostic but not the novelty diagnostic.
method	the similarity method passed to cv_similarity: "MESS", "L1" or "L2".
num_sample	number of random raster samples used by the distance and novelty diagnostics.
min_test	integer; folds with fewer than this many test points are flagged as tiny_test.
seed	integer; an optional random seed for the raster-sampling baselines.
progress	logical; whether to show a progress bar for the novelty diagnostic.

Details

The structural part (sizes, class prevalence and the structural warnings) is read from the cv object itself and needs no raster. When r (or pred_points/model_domain) is supplied, the per-fold distance diagnostic of cv_distance is added; when the covariate raster r is supplied, the per-fold environmental-novelty diagnostic of cv_similarity is added as well. These reuse the same computations as those functions.

The warnings are returned as data (a data.frame), not raised as warning() conditions, so they can be inspected programmatically. The structural warnings are a k-fold concept and are skipped for leave-one-out objects (cv_buffer, cv_nndm), whose folds hold a single test point by design. The flagged issues are:

• empty_test - a fold with no test points.

• single_class_test - a fold whose test set contains a single class (breaks AUC and other class-wise metrics).

• class_missing_train - a class entirely absent from a fold's training set.

• tiny_test - a fold with fewer than min_test test points (unstable per-fold estimate).

• imbalance - a class severely under-represented in a fold's test set relative to an even split.

• high_leakage - a fold where at least 90% of test points sit nearer to training than the median prediction distance (only available when the distance diagnostic is computed).

Value

an object of class cv_summary: a list with

• n_folds - the number of folds.

• is_loo - whether cv is a leave-one-out object.

• records - the per-fold train/test counts per class (fold sizes and class prevalence).

• distances - the per-fold test-to-nearest-train distance summary, or NULL.

• novelty - the per-fold environmental-novelty (extrapolation) summary, or NULL.

• warnings - a data.frame of flagged degenerate folds (fold, type, message); zero rows when nothing is flagged.

See Also

cv_distance and cv_similarity for the individual diagnostics, and cv_plot to visualise the folds

Examples

library(blockCV)

points <- read.csv(system.file("extdata/", "species.csv", package = "blockCV"))
pa_data <- sf::st_as_sf(points, coords = c("x", "y"), crs = 7845)
covar <- terra::rast(system.file("extdata/au/bio_5.tif", package = "blockCV"))

sb <- cv_spatial(x = pa_data, column = "occ", size = 450000, k = 5, iteration = 1)

# structural summary + warnings only (no raster)
cv_summary(sb)

# add the distance and novelty diagnostics
cv_summary(sb, x = pa_data, r = covar, num_sample = 5000)

blockCV documentation built on Aug. 20, 2026, 5:10 p.m.