Pre-Linkage Data Quality: preflight(), check_medicare(), and flock()

In starling: Record Linkage for Public Health Surveillance

knitr::opts_chunk$set(
  collapse = TRUE,
  comment  = "#>",
  message  = FALSE,
  warning  = FALSE
)

Why pre-linkage quality matters

The Fellegi-Sunter EM algorithm scores pairs by comparing field values. Poor data quality in any of those fields degrades the score distribution in ways that no threshold choice can fully recover:

• High missingness in a comparison variable reduces its discriminating power, flattening the valley between match and non-match clusters.
• Invalid Medicare numbers that happen to match a wrong AIR record add spurious positive weight to false pairs.
• Name typos are expected (the algorithm handles them via fuzzy string comparison) but systematic truncation or encoding differences are not.
• Duplicate IDs in the primary dataset produce multiple output rows per original record, inflating counts and biasing VE estimates.

starling provides three functions specifically for catching these issues before murmuration() is called.

---

preflight() — structured pre-linkage audit

preflight() runs a battery of checks across both datasets and returns a structured report. It is the recommended first step before any linkage call.

library(starling)
data(cases_notifiable)
data(vax_air)

audit <- preflight(
  data1        = cases_notifiable,
  data2        = vax_air,
  linkage_vars = c("lettername1", "lettername2", "dob", "medicare10"),
  id_col1      = "id_var",
  id_col2      = "id_var",
  date_cols    = c("dob", "onset_date"),
  medicare_col = "medicare10",
  verbose      = TRUE
)

The audit list contains structured results for programmatic access:

# Completeness table
head(audit$completeness)

# Duplicate ID counts
audit$duplicates

# Flags raised (empty = all clear)
audit$flags

---

check_medicare() — Modulus 10 checksum validation

Australian Medicare numbers contain a check digit at position 9 calculated from positions 1–8 using the weights 1, 3, 7, 9, 1, 3, 7, 9. check_medicare() verifies this and returns a three-state flag:

| Value | Meaning | |---|---| | 1L | Checksum passes — number is internally consistent | | 0L | Checksum fails — at least one digit is wrong | | NA | Missing, blank, or non-numeric — not verifiable |

cases_checked <- check_medicare(
  cases_notifiable,
  medicare_col = "medicare10",
  output_col   = "medicare_valid",
  verbose      = TRUE
)

# Distribution of flags
table(cases_checked$medicare_valid, useNA = "always")

What to do with invalid Medicare numbers

Do not discard the record — it may still link correctly on name and DOB. Instead, replace the invalid number with NA before linkage so it does not contribute negatively to the score:

cases_checked$medicare10 <- ifelse(
  cases_checked$medicare_valid == 1L,
  cases_checked$medicare10,
  NA_character_
)

---

flock() — blocking variable construction

flock() creates one to three blocking keys from demographic fields. Blocking restricts murmuration() to comparing only pairs that share a blocking key, making the search tractable for large datasets without sacrificing recall on the key variables.

# Single-field block (gender only — broadest, lowest specificity)
cases_blocked <- flock(
  cases_checked,
  block1_vars    = "gender",         # block1: gender only
  block2_vars    = "gender",         # block2: same here; use composite in production
  block3_vars    = "postcode",       # block3: postcode (finer)
  birth_year_col = "dob"             # derives birth_year column for composite use
)

vax_blocked <- flock(
  vax_air,
  block1_vars    = "gender",
  block3_vars    = "postcode",
  birth_year_col = "dob"
)

# Inspect blocking key distributions
table(cases_blocked$block1)
head(sort(table(cases_blocked$block3), decreasing = TRUE))

Choosing blocking variables for Australian datasets

| Dataset size | Recommended block | Rationale | |---|---|---| | < 10 000 records | gender | Broadest; halves candidate space immediately | | 10 000–500 000 | gender + birth_year composite | Balances sensitivity and specificity | | > 500 000 | postcode + birth_year | Fine-grained; requires high postcode completeness |

For maximum recall, run murmuration() twice — once with a broad block, once with a fine block — and union the results:

linked_broad <- murmuration(cases_blocked, vax_blocked,
  blocking_var = "block1", ...)
linked_fine  <- murmuration(cases_blocked, vax_blocked,
  blocking_var = "block3", ...)

# Union, keeping the highest-scoring link per case
linked_all <- dplyr::bind_rows(linked_broad, linked_fine) |>
  dplyr::group_by(id_var.x) |>
  dplyr::slice_max(weights, n = 1, with_ties = FALSE) |>
  dplyr::ungroup()

---

Putting it together

library(starling)
data(cases_notifiable); data(vax_air)

# 1. Audit
preflight(cases_notifiable, vax_air,
          linkage_vars = c("lettername1", "lettername2", "dob", "medicare10"),
          medicare_col = "medicare10")

# 2. Fix Medicare
cases <- check_medicare(cases_notifiable)
cases$medicare10 <- ifelse(cases$medicare_valid == 1L,
                            cases$medicare10, NA_character_)

# 3. Block
cases <- flock(cases, block1_vars = "gender", birth_year_col = "dob")
vax   <- flock(vax_air, block1_vars = "gender", birth_year_col = "dob")

# 4. Link
linked <- murmuration(cases, vax,
  linkage_type    = "v2c",
  event_date      = "onset_date",
  id_var          = "id_var",
  blocking_var    = "block1",
  compare_vars    = c("lettername1", "lettername2", "dob", "medicare10"),
  threshold_value = 17)

---

Session information

sessionInfo()

starling documentation built on July 10, 2026, 9:07 a.m.