cv_distance: Compare a cross-validation design to the prediction domain...
In blockCV: Spatial and Environmental Blocking for Cross-Validation

cv_distance

R Documentation

Compare a cross-validation design to the prediction domain via nearest-neighbour distances

Description

A diagnostic that works on any blockCV cross-validation object (the output of cv_spatial, cv_cluster, cv_group, cv_buffer, cv_nndm or cv_knndm). It draws the same nearest-neighbour distance distribution plot produced internally by cv_nndm and cv_knndm, but for a fold configuration that has already been generated. This lets you check whether a given fold-generation strategy actually pushes the test-to-train distances towards the distances the model will face when predicting over the target area. It does not compute pairwise distances for general use; it is a fold-quality diagnostic.

Usage

cv_distance(
  cv,
  x,
  r = NULL,
  pred_points = NULL,
  model_domain = NULL,
  transfer = NULL,
  space = "geographical",
  add_random = TRUE,
  num_random = 10L,
  num_sample = 10000L,
  sampling = "regular",
  scale = TRUE,
  seed = NULL,
  plot = TRUE
)

Arguments

`cv`	a `blockCV` cross-validation object, i.e. the output of `cv_spatial`, `cv_cluster`, `cv_group`, `cv_buffer`, `cv_nndm` or `cv_knndm`.
`x`	a simple features (sf) object of the spatial sample points used to create the `cv` object.
`r`	a terra SpatRaster object. It defines the area the model predicts over; when neither `pred_points` nor `model_domain` is supplied, prediction points are sampled from it. It is also required (for the covariates) when `space = "feature"`. One of `r`, `pred_points`, or `model_domain` is required.
`pred_points`	a simple features (sf) object of prediction points (optional). If provided, these are used directly as the prediction locations instead of sampling from `r` or `model_domain`.
`model_domain`	an sf polygon of the prediction area (optional). If provided (and `pred_points` is not), prediction points are sampled from it.
`transfer`	a terra SpatRaster object (optional) defining a second, transfer prediction domain – for example a future climate scenario or a different region – to compare the fold design against. It must carry the same predictor layers as `r` (identical names) and is only supported when `space = "feature"`, where its covariates are standardised with the same centring/scaling as the training data. See details.
`space`	character; `"geographical"` (default) matches distances in geographical space, or `"feature"` matches distances in the (scaled) covariate space of `r`.
`add_random`	logical; when `TRUE` (default), overlay the CV curve of a random `k`-fold split with the same number of folds as `cv` (ignored for leave-one-out objects, see details).
`num_random`	integer; the number of random `k`-fold splits used to estimate the random baseline when `add_random = TRUE`. The plotted random curve is the mean across these splits, with a 10-90 band.
`num_sample`	integer; the number of prediction points to sample from `r` or `model_domain` when `pred_points` is not supplied.
`sampling`	either `"regular"` (default) or `"random"` for sampling prediction points.
`scale`	logical; whether to scale the covariates when `space = "feature"`.
`seed`	integer; a random seed for reproducibility.
`plot`	logical; whether to draw the distance-distribution plot (default `TRUE`). The plot object is always built and returned in `$plot` either way (so it can be customised or arranged with e.g. `cowplot`); `plot = FALSE` only suppresses drawing it now.

Details

Three to five empirical cumulative distribution functions of nearest-neighbour distances are compared over a distance r:

Prediction (\hat{G}_{ij}) - from each prediction point to its nearest sample point. This is the distance regime the model meets at prediction time and is the target to match.
LOO (\hat{G}_{j}) - from each sample point to its nearest other sample point. This is the nearest-neighbour (leave-one-out) limit, i.e. the most optimistic, random-like case.
CV (\hat{G}_{j}^{*}) - from each test point to its nearest training point within the same fold of the supplied cv object. The closer this curve sits to the Prediction curve, the more the fold design mimics the prediction task.
Random - the CV curve of a random k-fold split with the same number of folds as cv, drawn only when add_random = TRUE. It shows where a naive random assignment lands, so the improvement of the supplied design is visible. For leave-one-out objects (cv_buffer, cv_nndm) a random split coincides with the LOO curve, so it is omitted.
Transfer - from each point of a transfer domain to its nearest sample point, drawn only when transfer is supplied (see below). This is the distance regime the model meets when transferred to that domain.

The gap between each curve and the Prediction curve is summarised by the Wasserstein-1 distance (the area between the two empirical distribution functions; lower means a closer match). These values are shown in the subtitle and attached to the returned plot as attr(p, "W").

When space = "feature" the distances are computed in the (optionally scaled) covariate space of r instead of the geographical space, mirroring cv_knndm.

Transfer domain. When a transfer raster is supplied, the diagnostic adds the Transfer curve above and, in $W_transfer, the Wasserstein-1 distance of the CV, LOO and Prediction curves to it. W_transfer["CV"] is the headline transferability read: how faithfully the fold design rehearses the transfer task (lower means the cross-validation is a fairer rehearsal of the transfer). A Transfer curve that sits to the right of the LOO curve flags a domain that no fold design built from these samples can emulate, i.e. extrapolation beyond the support of the training data, which cross-validation cannot validate. Because the transfer domain is compared in the (shared, training-scaled) covariate space, transfer is only supported when space = "feature"; it must carry the same predictor layers as r (matched by name and standardised the same way), and it may cover a different time (e.g. a future climate scenario) or a different region entirely.

When the supplied cv object was built with presence_bg = TRUE (in cv_spatial, cv_cluster, cv_knndm, cv_buffer or cv_nndm), all three distance distributions are computed on the presences only: the prediction, LOO and CV curves are expressed relative to the presence-only training data rather than the (often random) background points (locations sampled across the study area to represent the available conditions rather than confirmed absences), matching the nearest neighbour distance matching framing. This is read automatically from cv.

Value

an object of class cv_distance: a list with

distances - a per-fold data.frame of the test-to-nearest-train distances (the leakage signal): the number of test points (n_test), the min, quartiles (q1, median, q3) and max of those distances, and pct_below_pred, the percentage of test points nearer to training than the median prediction distance (higher means a more optimistic, leakier fold). For leave-one-out objects each fold holds a single test point, so there is one row per held-out point.
W - a named numeric vector of the Wasserstein-1 distance of each curve to the Prediction curve (lower means a closer match to the prediction domain).
W_transfer - only present when transfer is supplied: a named numeric vector of the Wasserstein-1 distance of the CV, LOO and Prediction curves to the Transfer curve (lower means a closer match to the transfer domain). W_transfer["CV"] is the transferability read.
plot - the ggplot of the nearest-neighbour distance distributions (always built, whether or not it is drawn).

The distributions are drawn when plot = TRUE (default). Printing the object shows a compact text summary rather than redrawing the plot; call plot() on the returned object to redraw it.

References

Milà, C., Mateu, J., Pebesma, E., & Meyer, H. (2022). Nearest neighbour distance matching leave-one-out cross-validation for map validation. Methods in Ecology and Evolution, 13(6), 1304-1316.

Linnenbrink, J., Milà, C., Ludwig, M., & Meyer, H. (2024). kNNDM CV: k-fold nearest neighbour distance matching cross-validation for map accuracy estimation. Geoscientific Model Development, 17(15), 5897-5912.

Examples


library(blockCV)

# import presence-absence species data
points <- read.csv(system.file("extdata/", "species.csv", package = "blockCV"))
# make an sf object from data.frame
pa_data <- sf::st_as_sf(points, coords = c("x", "y"), crs = 7845)

# load raster data
path <- system.file("extdata/au/bio_5.tif", package = "blockCV")
covar <- terra::rast(path)

# generate spatial folds
sb <- cv_spatial(x = pa_data, column = "occ", size = 450000, k = 5, iteration = 1)

# assess how close the folds are to the prediction domain
cv_distance(cv = sb, x = pa_data, r = covar, num_sample = 5000)

blockCV documentation built on Aug. 20, 2026, 5:10 p.m.