Calibration curves and figures of merit

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 7,
  fig.height = 4,
  out.width = "100%",
  dpi = 96
)

A calibration relates a signal to the concentration of an element, from samples of known composition, and predicts the concentration of new samples. This vignette builds univariate calibration curves from line intensities, with their limits of detection and quantification, and compares them with a multivariate model of the whole spectrum, whose figures of merit come from the net analyte signal. It ends with routine analysis: whether new spectra lie within the calibration space, how to transfer the calibration to another instrument, and whether the instrument drifts.

library(specProc)
library(dplyr)
library(tidyr)
library(ggplot2)

data("forageLIBS")
spectra_id <- forageLIBS |> select(1:2) |> names()
minerals <- forageLIBS |> select(3:14) |> names()
spectra <- forageLIBS |> select(-all_of(c(spectra_id, minerals)))

Line intensities

We remove the baseline and measure, in every spectrum, the intensities of a potassium line, an iron line and the carbon line C I 247.86 nm. Carbon is the main element of the forage matrix, and dividing by its line compensates for the changes of ablated mass and plasma from spectrum to spectrum (the internal standard of vignette("preprocessing")):

corrected <- baseline_arpls(spectra, lambda = 1e5, max_iter = 20)$correction

lines <- c(C = 247.856, K = 769.896, Fe = 259.940)
intensities <- line_intensities(corrected, lines) |>
  select(spectrum, line, intensity) |>
  pivot_wider(names_from = line, values_from = intensity)

data <- forageLIBS |>
  select(all_of(spectra_id), K, Fe) |>
  mutate(K_signal = intensities$K / intensities$C,
         Fe_signal = intensities$Fe / intensities$C)

We set aside a quarter of the samples to test the calibrations:

set.seed(1)
test_samples <- sample(unique(data$Sample), round(0.25 * n_distinct(data$Sample)))
calibration <- filter(data, !Sample %in% test_samples)
test <- filter(data, Sample %in% test_samples)
c(calibration = nrow(calibration), test = nrow(test))

A calibration curve for potassium

calibration_curve() fits the signal as a function of the concentration, and returns the coefficients, the figures of merit and tests of linearity:

k_curve <- calibration_curve(calibration, K_signal, K)
k_curve
k_fom <- k_curve$figures_of_merit
mandel <- k_curve$linearity |> filter(test == "Mandel")

plot_calibration() draws the curve with its confidence band (where the mean signal lies) and its prediction band (where a single new measurement is expected). The replicate measurements of a sample overlap, so they are drawn semi-transparent (point_alpha), and the axes get their units:

plot_calibration(k_curve, point_alpha = 0.4, xlab = "K (%)",
                 ylab = "K I 769.90 nm / C I 247.86 nm")

predict() inverts the curve: it gives the concentration of new samples from their signals, with a prediction interval, and flags the values below the limits of detection and quantification:

k_predicted <- predict(k_curve, test$K_signal)
head(k_predicted)
sqrt(mean((k_predicted$concentration - test$K)^2))   # RMSEP, % K

The limit of quantification of this curve, r signif(k_fom$loq, 2)% K, is above the potassium content of almost every sample, so r round(100 * mean(k_predicted$below_loq))% of the predictions are flagged below_loq: a single line ranks the samples, but does not quantify them individually with the usual (10\,\sigma) margin.

The limits above follow the ICH convention. ISO 11843-2 and IUPAC define them by the risks of a false positive ((\alpha)) and of a false negative ((\beta)), and include the uncertainty of the curve itself: the critical value is the content above which potassium is declared present, and the limit of detection, the content detected with probability (1 - \beta):

k_iso <- calibration_curve(calibration, K_signal, K, lod_convention = "iso11843")
k_iso$figures_of_merit |> select(critical_value, lod, loq)

With risks of 5%, the limit of detection, r signif(k_iso$figures_of_merit$lod, 2)% K, is close to that of the ICH convention: with r nrow(calibration) standards, the uncertainty of the curve is small next to the scatter of the samples. predict() then marks the samples above the critical value as detected.

Uncertain reference values

Least squares treat the laboratory values as exact. Their errors bias the slope toward zero (regression dilution) and narrow the intervals. York's regression weights each standard by the uncertainties of both its concentration and its signal. The uncertainty of the laboratory values is not given with these data, so we take, for illustration, a relative standard uncertainty of 5%; that of the signal follows from the noise of the spectra (intensity_se of line_intensities()):

noise <- line_intensities(corrected, lines[c("C", "K")]) |>
  mutate(relative_se = intensity_se / intensity) |>
  select(spectrum, line, relative_se) |>
  pivot_wider(names_from = line, values_from = relative_se)
data <- data |>
  mutate(K_se = 0.05 * K, K_signal_se = K_signal * sqrt(noise$K^2 + noise$C^2))
calibration <- filter(data, !Sample %in% test_samples)
test <- filter(data, Sample %in% test_samples)

k_york <- calibration_curve(calibration, K_signal, K, regression = "york",
                            concentration_se = K_se, signal_se = K_signal_se)
k_york
york_predicted <- predict(k_york, test$K_signal)
rmsep <- \(p) sqrt(mean((p$concentration - test$K)^2))
slope <- \(curve) signif(curve$coefficients$estimate[2], 3)

The slope rises from r slope(k_curve) to r slope(k_york), and the prediction error on the test samples falls from r sprintf("%.2f", rmsep(k_predicted)) to r sprintf("%.2f", rmsep(york_predicted))% K. But the reduced chi-squared, r round(k_york$eiv$chi_squared), shows that the standards scatter far more than these uncertainties explain: matrix effects, not the noise of the spectra or the assumed errors of the laboratory values, dominate. York's weights then mostly lower the weight of the richest standards, whose 5% uncertainty is the largest, much like weights "1/x2", rather than correct the dilution of the slope; its standard errors are scaled up accordingly. York's regression is at its best with reference materials whose certified uncertainties, and the replicate measurements of their signals, describe the scatter (a reduced chi-squared near 1). When every standard has the same uncertainties, regression = "deming", with the ratio of their variances, is the simpler alternative.

Trace elements: weights and limits of detection

Iron ranges from r min(data$Fe, na.rm = TRUE) to r max(data$Fe, na.rm = TRUE) ppm, and its signal is noisier at high concentration. An unweighted fit is then dominated by the few rich samples, and so is its residual standard deviation, which inflates the limit of detection. Weights "1/x" give each sample a weight inversely proportional to its concentration. With weights, the residual standard deviation depends on their scale, so the blank standard deviation is taken from the standard error of the intercept (lod_method = "intercept"):

fe_calibration <- filter(calibration, !is.na(Fe))
fe_unweighted <- calibration_curve(fe_calibration, Fe_signal, Fe)
fe_weighted <- calibration_curve(fe_calibration, Fe_signal, Fe, weights = "1/x",
                                 lod_method = "intercept")

bind_rows(unweighted = fe_unweighted$figures_of_merit,
          weighted = fe_weighted$figures_of_merit, .id = "fit") |>
  select(fit, n, r_squared, lod, loq)
plot_calibration(fe_weighted, interval = "prediction", point_alpha = 0.4, xlab = "Fe (ppm)",
                 ylab = "Fe I 259.94 nm / C I 247.86 nm")

The weighted fit gives a limit of detection of r round(fe_weighted$figures_of_merit$lod) ppm, instead of r round(fe_unweighted$figures_of_merit$lod) ppm. Its R² is lower, because it no longer favors the rich samples, which dominate R².

fe_test <- filter(test, !is.na(Fe))
fe_predicted <- predict(fe_weighted, fe_test$Fe_signal)
count(fe_predicted, below_loq)
median(abs(fe_predicted$concentration - fe_test$Fe) / fe_test$Fe)   # relative error

A multivariate model and its figures of merit

A single line uses a small part of the spectrum. Partial least squares (PLS) regression uses all of it: we divide the baseline-corrected spectra by the carbon line, and choose the number of components by cross-validation on the calibration samples, with the pls package:

x <- as.matrix(corrected) / intensities$C
is_calibration <- !data$Sample %in% test_samples
x_calibration <- x[is_calibration, ]

cv <- pls::plsr(K ~ x, data = data.frame(K = data$K[is_calibration], x = I(x_calibration)),
                ncomp = 15, validation = "CV", segments = 10)
ncomp <- pls::selectNcomp(cv, method = "onesigma")
ncomp

nas() fits the same model and computes its net analyte signal (NAS): the part of each spectrum that is unique to potassium, orthogonal to the contributions of the other constituents. The figures of merit follow from it:

We estimate the noise from the three samples measured twice, as the standard deviation of their difference spectra divided by (\sqrt{2}):

repeated <- data$Sample[duplicated(data$Sample)]
differences <- sapply(repeated, \(s) {
  rows <- which(data$Sample == s)
  x[rows[1], ] - x[rows[2], ]
})
noise <- sd(differences) / sqrt(2)

k_nas <- nas(x_calibration, data$K[is_calibration], ncomp = ncomp, noise = noise)
k_nas$figures_of_merit
nas_predicted <- predict(k_nas, x[!is_calibration, ])
sqrt(mean((nas_predicted$predicted - test$K)^2))   # RMSEP, % K
summary(nas_predicted$selectivity)
fom <- k_nas$figures_of_merit
rmsep_pls <- sqrt(mean((nas_predicted$predicted - test$K)^2))
rmsep_line <- sqrt(mean((k_predicted$concentration - test$K)^2))
bind_rows(
  `K I 769.90 nm curve` = tibble(laboratory = test$K, predicted = k_predicted$concentration),
  `PLS (NAS)` = tibble(laboratory = test$K, predicted = nas_predicted$predicted),
  .id = "model"
) |>
  ggplot(aes(laboratory, predicted)) +
  geom_abline(colour = "grey50", linetype = "dashed") +
  geom_point(alpha = 0.6) +
  facet_wrap(~ model) +
  coord_equal() +
  labs(x = "Laboratory K (%)", y = "Predicted K (%)") +
  theme_bw()

The PLS model predicts the test samples with an error of r sprintf("%.2f", rmsep_pls)% K, against r sprintf("%.2f", rmsep_line)% for the calibration curve. Its limit of detection, r signif(fom[["lod"]], 2)% K, accounts for the noise of the spectra only, so it is a lower bound, far below the limit of the calibration curve, which includes all the scatter of the samples. Its mean selectivity, r round(fom[["selectivity"]], 2), means that only about a tenth of each spectrum is specific to potassium: the rest is shared with the other constituents and the plasma.

Spectra outside the calibration space

A model is reliable only for spectra like those it was calibrated on. applicability_domain() describes the calibration spectra by a principal component analysis, here with as many components as the PLS model, and predict() flags the test spectra whose Hotelling (T^2) (more extreme than the calibration samples) or Q residual (unlike them) exceeds its 99% limit:

domain <- applicability_domain(x_calibration, ncomp = ncomp)
domain
outside <- predict(domain, x[!is_calibration, ])
count(outside, outlier)
plot_influence(outside)
errors <- abs(nas_predicted$predicted - test$K)

r sum(!outside$inside) of the r nrow(outside) test spectra lie outside the domain. Their predictions are about as accurate as the others (mean absolute error r sprintf("%.2f", mean(errors[!outside$inside])) against r sprintf("%.2f", mean(errors[outside$inside]))% K): the flag says that the model extrapolates, not that the prediction is wrong, but such samples deserve a check in routine analysis. For a calibration curve, predict() flags the concentrations outside the range of the standards (outside_range).

Transferring the calibration to another instrument

A calibration is tied to the instrument that recorded its spectra, the primary instrument. On a secondary instrument, or on the same one after a change of its optics, the spectra differ, and so do the predictions. With no second instrument for these samples, we simulate one: its response tilts by ±15% across the spectral range, its wavelength axis is shifted by one channel, and it adds an offset:

wl_x <- as.numeric(colnames(x))
response <- 1 + 0.3 * (wl_x - mean(wl_x)) / diff(range(wl_x))
secondary <- sweep(cbind(x[, -1], x[, ncol(x)]), 2, response, "*") + 0.002

The PLS model of potassium predicts the test samples measured on this instrument badly. Twenty calibration samples, measured on both instruments, are enough to transfer their spectra: by direct standardization (ds()), a linear map of the whole spectrum, or by spectral space transformation (sst()), which moves the secondary spectra by the difference between the instruments in the principal components they share. Instead of the spectra, slope_bias() corrects the predictions by a straight line:

predict_k <- \(spectra) drop(predict(cv, newdata = data.frame(x = I(spectra)), ncomp = ncomp))
set.seed(2)
transfer_samples <- sample(which(is_calibration), 20)
secondary_test <- secondary[!is_calibration, ]

by_ds <- ds(x[transfer_samples, ], secondary[transfer_samples, ])
by_sst <- sst(x[transfer_samples, ], secondary[transfer_samples, ])
by_slope_bias <- slope_bias(predict_k(secondary[transfer_samples, ]), data$K[transfer_samples])

prediction_error <- \(predicted) sqrt(mean((predicted - test$K)^2))
transfer_errors <- c(
  primary = prediction_error(predict_k(x[!is_calibration, ])),
  secondary = prediction_error(predict_k(secondary_test)),
  ds = prediction_error(predict_k(predict(by_ds, secondary_test))),
  sst = prediction_error(predict_k(predict(by_sst, secondary_test))),
  slope_bias = prediction_error(predict(by_slope_bias, predict_k(secondary_test)))
)
round(transfer_errors, 2)   # RMSEP, % K

Direct standardization and SST bring the error of the test samples back to that of the primary instrument, r sprintf("%.2f", transfer_errors[["primary"]])% K, from r sprintf("%.2f", transfer_errors[["secondary"]])% K. The slope and bias correction removes most of the bias, but not the distortion of the spectra, which changes the ranking of the samples. pds(), piecewise direct standardization, suits local differences such as shifts of the axis, but its transfer matrix has one row and one column per channel (r ncol(x) here), heavy for whole spectra.

Monitoring drift

The measurements are numbered in the order in which they were acquired. The carbon line, the internal standard of these calibrations, follows the ablated mass and the plasma, so a control chart of its intensity shows the stability of the instrument over the acquisition. The first 100 measurements set the center and the standard deviation, and an exponentially weighted moving average (EWMA), sensitive to slow drifts, follows the others:

carbon <- control_chart(intensities$C, order = data$Measurement, method = "ewma",
                        baseline = 100)
plot_control_chart(carbon, xlab = "Measurement (acquisition order)",
                   ylab = "EWMA of the C I 247.86 nm intensity")
runs <- rle(carbon$signal)
longest <- which.max(ifelse(runs$values, runs$lengths, 0))
run_end <- cumsum(runs$lengths)[longest]
run_start <- run_end - runs$lengths[longest] + 1

The intensity changes by blocks between the sessions of the acquisition (the gaps in the numbering), and the chart signals r sum(carbon$signal) measurements; the longest run of signals, from measurement r carbon$order[run_start] to r carbon$order[run_end], is a drop of the intensity. Dividing by the carbon line compensates for such changes in the calibrations above. In routine analysis, a check sample measured at regular intervals, charted the same way (method = "shewhart", "ewma" or "cusum"), separates a drift of the instrument from a change of the samples.

Summary

The last vignette, vignette("plasma-diagnostics"), measures the plasma itself: electron density, temperature and self-absorption.



Try the specProc package in your browser

Any scripts or data that you put into this service are public.

specProc documentation built on Nov. 20, 2026, 5:11 p.m.