murmuration_plot: Visualise Linkage Weight Distribution with Threshold Overlay

In starling: Record Linkage for Public Health Surveillance

Visualise Linkage Weight Distribution with Threshold Overlay

Description

**Bird note**: When a murmuration peaks, a watcher on the ground sees the flock split momentarily into two layers — the dense, high-altitude core of confirmed companions and the looser, lower-flying fringe of uncertain travellers — before the whole shape resolves again. murmuration_plot() captures exactly that split: the linkage weight distribution separates into a high-score cluster of likely true matches at the top, a low-score cluster of likely non-matches at the bottom, and — critically — a gap or overlap zone in between that tells you whether your threshold is well-placed or needs adjustment.

Generates a jittered scatter plot of Fellegi-Sunter linkage weights from a scored pairs object (the output of predict() in the reclin2 workflow, before select_threshold() is called). Weights are plotted on the y-axis with jitter on the x-axis for readability. A horizontal line marks the chosen threshold, with regions above and below shaded to distinguish likely matches from likely non-matches. Optionally overlays a density curve to highlight the bimodal (match/non-match) distribution.

Use this plot before finalising threshold_value in murmuration to confirm that the threshold sits in the natural valley between the two score clusters. If the valley is not visible (scores form a unimodal distribution), this is a signal that the linkage variables or blocking strategy needs revision.

Usage

murmuration_plot(
  pairs_pred,
  threshold = 17,
  weight_col = "weights",
  n_sample = 5000L,
  jitter_width = 0.3,
  point_alpha = 0.35,
  point_size = 1.2,
  show_density = TRUE,
  show_counts = TRUE,
  palette = "sch",
  title = NULL,
  interactive = FALSE
)

Arguments

pairs_pred	A pairs object with a weights column — the output of reclin2::predict.problink_em(m, pairs = pairs, add = TRUE). May also be a plain data frame with a numeric weights column.
threshold	Numeric. The threshold value to display as a horizontal reference line. Default 17 (reasonable starting point for Australian surveillance linkage with Medicare + 2 names + DOB; see Details for guidance).
weight_col	Name of the column containing linkage weights. Default "weights".
n_sample	Integer. Maximum number of points to plot (random sample drawn if the pairs object is larger than this). Plotting millions of points is slow and uninformative; the distribution shape is well-represented by 5 000–10 000 points. Default 5000.
jitter_width	Numeric. Horizontal jitter width. Default 0.3.
point_alpha	Numeric (0–1). Point transparency. Default 0.35.
point_size	Numeric. Point size. Default 1.2.
show_density	Logical. Overlay a right-margin density curve of the weight distribution? Default TRUE.
show_counts	Logical. Annotate the plot with the count of pairs above and below the threshold? Default TRUE.
palette	Character. Colour palette for match/non-match zones: "sch" (Sunshine Coast HHS greens — default), "default" (teal/coral), or "grey" (greyscale, publication-safe).
title	Optional plot title string. NULL uses a default title.
interactive	Logical. Return a plotly object (TRUE) or a ggplot2 object (FALSE, default).

Details

## Threshold guidance for Australian public health linkage

The Fellegi-Sunter weights produced by the EM algorithm are log-likelihood ratios, not fixed absolute scores, so the "right" threshold is dataset-specific. That said, common reference points for Australian population health work (Medicare, 2 names, DOB) are:

| Threshold | Practical meaning | |—|—| | 8–12 | High sensitivity, lower specificity — suitable when recall matters more than precision (e.g. small datasets, rare disease surveillance) | | 12–17 | Balanced — typical starting point for surveillance linkage with a complete variable set | | 17–22 | High specificity — suitable when false matches are especially costly (e.g. VE studies where false vaccination attribution biases the estimate) | | > 22 | Very conservative — consider adding clerical review for records in the 17–22 range |

The most principled approach is to look at the weight distribution (this plot) and choose the threshold at the **lowest density point** between the two modes. If the two modes overlap substantially, your variable set may lack discriminating power for this dataset — see preflight for diagnostic guidance.

Value

A ggplot2 object (interactive = FALSE) or a plotly htmlwidget (interactive = TRUE).

See Also

murmuration, preflight

Examples

## Not run: 
# Standard reclin2 workflow, with plot inserted before threshold selection
pairs <- reclin2::pair_blocking(df1, df2, blocking_var)
reclin2::compare_pairs(pairs, on = compare_vars,
  default_comparator = reclin2::jaro_winkler(0.9), inplace = TRUE)
m          <- reclin2::problink_em(reformulate(compare_vars), data = pairs)
pairs_pred <- predict(m, pairs = pairs, add = TRUE)

# Inspect the weight distribution before committing to a threshold
murmuration_plot(pairs_pred, threshold = 17)

# Try a different threshold, interactively
murmuration_plot(pairs_pred, threshold = 14, interactive = TRUE)

# Greyscale for a journal figure
murmuration_plot(pairs_pred, threshold = 17, palette = "grey")

## End(Not run)

starling documentation built on July 10, 2026, 9:07 a.m.