knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 4, out.width = "100%", dpi = 96 )
A calibration relates a signal to the concentration of an element, from samples of known composition, and predicts the concentration of new samples. This vignette builds univariate calibration curves from line intensities, with their limits of detection and quantification, and compares them with a multivariate model of the whole spectrum, whose figures of merit come from the net analyte signal. It ends with routine analysis: whether new spectra lie within the calibration space, how to transfer the calibration to another instrument, and whether the instrument drifts.
library(specProc) library(dplyr) library(tidyr) library(ggplot2) data("forageLIBS") spectra_id <- forageLIBS |> select(1:2) |> names() minerals <- forageLIBS |> select(3:14) |> names() spectra <- forageLIBS |> select(-all_of(c(spectra_id, minerals)))
We remove the baseline and measure, in every spectrum, the intensities of
a potassium line, an iron line and the carbon line C I 247.86 nm. Carbon
is the main element of the forage matrix, and dividing by its line
compensates for the changes of ablated mass and plasma from spectrum to
spectrum (the internal standard of vignette("preprocessing")):
corrected <- baseline_arpls(spectra, lambda = 1e5, max_iter = 20)$correction lines <- c(C = 247.856, K = 769.896, Fe = 259.940) intensities <- line_intensities(corrected, lines) |> select(spectrum, line, intensity) |> pivot_wider(names_from = line, values_from = intensity) data <- forageLIBS |> select(all_of(spectra_id), K, Fe) |> mutate(K_signal = intensities$K / intensities$C, Fe_signal = intensities$Fe / intensities$C)
We set aside a quarter of the samples to test the calibrations:
set.seed(1) test_samples <- sample(unique(data$Sample), round(0.25 * n_distinct(data$Sample))) calibration <- filter(data, !Sample %in% test_samples) test <- filter(data, Sample %in% test_samples) c(calibration = nrow(calibration), test = nrow(test))
calibration_curve() fits the signal as a function of the concentration,
and returns the coefficients, the figures of merit and tests of linearity:
k_curve <- calibration_curve(calibration, K_signal, K) k_curve
k_fom <- k_curve$figures_of_merit mandel <- k_curve$linearity |> filter(test == "Mandel")
r signif(k_fom$lod, 2)% and
r signif(k_fom$loq, 2)% K, describe this calibration rather than the
instrument.r signif(mandel$p_value, 2), points to a slight
curvature, as expected for a resonance line, which the plasma partly
reabsorbs at high concentration.plot_calibration() draws the curve with its confidence band (where the
mean signal lies) and its prediction band (where a single new measurement
is expected). The replicate measurements of a sample overlap, so they are
drawn semi-transparent (point_alpha), and the axes get their units:
plot_calibration(k_curve, point_alpha = 0.4, xlab = "K (%)", ylab = "K I 769.90 nm / C I 247.86 nm")
predict() inverts the curve: it gives the concentration of new samples
from their signals, with a prediction interval, and flags the values below
the limits of detection and quantification:
k_predicted <- predict(k_curve, test$K_signal) head(k_predicted) sqrt(mean((k_predicted$concentration - test$K)^2)) # RMSEP, % K
The limit of quantification of this curve, r signif(k_fom$loq, 2)% K,
is above the potassium content of almost every sample, so
r round(100 * mean(k_predicted$below_loq))% of the predictions are
flagged below_loq: a single line ranks the samples, but does not
quantify them individually with the usual (10\,\sigma) margin.
The limits above follow the ICH convention. ISO 11843-2 and IUPAC define them by the risks of a false positive ((\alpha)) and of a false negative ((\beta)), and include the uncertainty of the curve itself: the critical value is the content above which potassium is declared present, and the limit of detection, the content detected with probability (1 - \beta):
k_iso <- calibration_curve(calibration, K_signal, K, lod_convention = "iso11843") k_iso$figures_of_merit |> select(critical_value, lod, loq)
With risks of 5%, the limit of detection,
r signif(k_iso$figures_of_merit$lod, 2)% K, is close to that of the ICH
convention: with r nrow(calibration) standards, the uncertainty of the
curve is small next to the scatter of the samples. predict() then marks
the samples above the critical value as detected.
Least squares treat the laboratory values as exact. Their errors bias the
slope toward zero (regression dilution) and narrow the intervals. York's
regression weights each standard by the uncertainties of both its
concentration and its signal. The uncertainty of the laboratory values is
not given with these data, so we take, for illustration, a relative
standard uncertainty of 5%; that of the signal follows from the noise of
the spectra (intensity_se of line_intensities()):
noise <- line_intensities(corrected, lines[c("C", "K")]) |> mutate(relative_se = intensity_se / intensity) |> select(spectrum, line, relative_se) |> pivot_wider(names_from = line, values_from = relative_se) data <- data |> mutate(K_se = 0.05 * K, K_signal_se = K_signal * sqrt(noise$K^2 + noise$C^2)) calibration <- filter(data, !Sample %in% test_samples) test <- filter(data, Sample %in% test_samples) k_york <- calibration_curve(calibration, K_signal, K, regression = "york", concentration_se = K_se, signal_se = K_signal_se) k_york
york_predicted <- predict(k_york, test$K_signal) rmsep <- \(p) sqrt(mean((p$concentration - test$K)^2)) slope <- \(curve) signif(curve$coefficients$estimate[2], 3)
The slope rises from r slope(k_curve) to r slope(k_york), and the
prediction error on the test samples falls from
r sprintf("%.2f", rmsep(k_predicted)) to
r sprintf("%.2f", rmsep(york_predicted))% K. But the reduced
chi-squared, r round(k_york$eiv$chi_squared), shows that the standards
scatter far more than these uncertainties explain: matrix effects, not the
noise of the spectra or the assumed errors of the laboratory values,
dominate. York's weights then mostly lower the weight of the richest
standards, whose 5% uncertainty is the largest, much like weights
"1/x2", rather than correct the dilution of the slope; its standard
errors are scaled up accordingly. York's regression is at its best with
reference materials whose certified uncertainties, and the replicate
measurements of their signals, describe the scatter (a reduced
chi-squared near 1). When every standard has the same uncertainties,
regression = "deming", with the ratio of their variances, is the simpler
alternative.
Iron ranges from r min(data$Fe, na.rm = TRUE) to
r max(data$Fe, na.rm = TRUE) ppm, and its signal is noisier at high
concentration. An unweighted fit is then dominated by the few rich samples,
and so is its residual standard deviation, which inflates the limit of
detection. Weights "1/x" give each sample a weight inversely proportional
to its concentration. With weights, the residual standard deviation depends
on their scale, so the blank standard deviation is taken from the standard
error of the intercept (lod_method = "intercept"):
fe_calibration <- filter(calibration, !is.na(Fe)) fe_unweighted <- calibration_curve(fe_calibration, Fe_signal, Fe) fe_weighted <- calibration_curve(fe_calibration, Fe_signal, Fe, weights = "1/x", lod_method = "intercept") bind_rows(unweighted = fe_unweighted$figures_of_merit, weighted = fe_weighted$figures_of_merit, .id = "fit") |> select(fit, n, r_squared, lod, loq)
plot_calibration(fe_weighted, interval = "prediction", point_alpha = 0.4, xlab = "Fe (ppm)", ylab = "Fe I 259.94 nm / C I 247.86 nm")
The weighted fit gives a limit of detection of
r round(fe_weighted$figures_of_merit$lod) ppm, instead of
r round(fe_unweighted$figures_of_merit$lod) ppm. Its R² is lower,
because it no longer favors the rich samples, which dominate R².
fe_test <- filter(test, !is.na(Fe)) fe_predicted <- predict(fe_weighted, fe_test$Fe_signal) count(fe_predicted, below_loq) median(abs(fe_predicted$concentration - fe_test$Fe) / fe_test$Fe) # relative error
A single line uses a small part of the spectrum. Partial least squares (PLS) regression uses all of it: we divide the baseline-corrected spectra by the carbon line, and choose the number of components by cross-validation on the calibration samples, with the pls package:
x <- as.matrix(corrected) / intensities$C is_calibration <- !data$Sample %in% test_samples x_calibration <- x[is_calibration, ] cv <- pls::plsr(K ~ x, data = data.frame(K = data$K[is_calibration], x = I(x_calibration)), ncomp = 15, validation = "CV", segments = 10) ncomp <- pls::selectNcomp(cv, method = "onesigma") ncomp
nas() fits the same model and computes its net analyte signal (NAS): the
part of each spectrum that is unique to potassium, orthogonal to the
contributions of the other constituents. The figures of merit follow from
it:
noise).We estimate the noise from the three samples measured twice, as the standard deviation of their difference spectra divided by (\sqrt{2}):
repeated <- data$Sample[duplicated(data$Sample)] differences <- sapply(repeated, \(s) { rows <- which(data$Sample == s) x[rows[1], ] - x[rows[2], ] }) noise <- sd(differences) / sqrt(2) k_nas <- nas(x_calibration, data$K[is_calibration], ncomp = ncomp, noise = noise) k_nas$figures_of_merit
nas_predicted <- predict(k_nas, x[!is_calibration, ]) sqrt(mean((nas_predicted$predicted - test$K)^2)) # RMSEP, % K summary(nas_predicted$selectivity)
fom <- k_nas$figures_of_merit rmsep_pls <- sqrt(mean((nas_predicted$predicted - test$K)^2)) rmsep_line <- sqrt(mean((k_predicted$concentration - test$K)^2))
bind_rows( `K I 769.90 nm curve` = tibble(laboratory = test$K, predicted = k_predicted$concentration), `PLS (NAS)` = tibble(laboratory = test$K, predicted = nas_predicted$predicted), .id = "model" ) |> ggplot(aes(laboratory, predicted)) + geom_abline(colour = "grey50", linetype = "dashed") + geom_point(alpha = 0.6) + facet_wrap(~ model) + coord_equal() + labs(x = "Laboratory K (%)", y = "Predicted K (%)") + theme_bw()
The PLS model predicts the test samples with an error of
r sprintf("%.2f", rmsep_pls)% K, against r sprintf("%.2f", rmsep_line)%
for the calibration curve. Its limit of detection, r signif(fom[["lod"]], 2)% K,
accounts for the noise of the spectra only, so it is a lower bound, far
below the limit of the calibration curve, which includes all the scatter of
the samples. Its mean selectivity, r round(fom[["selectivity"]], 2),
means that only about a tenth of each spectrum is specific to potassium:
the rest is shared with the other constituents and the plasma.
A model is reliable only for spectra like those it was calibrated on.
applicability_domain() describes the calibration spectra by a principal
component analysis, here with as many components as the PLS model, and
predict() flags the test spectra whose Hotelling (T^2) (more extreme
than the calibration samples) or Q residual (unlike them) exceeds its 99%
limit:
domain <- applicability_domain(x_calibration, ncomp = ncomp) domain outside <- predict(domain, x[!is_calibration, ]) count(outside, outlier)
plot_influence(outside)
errors <- abs(nas_predicted$predicted - test$K)
r sum(!outside$inside) of the r nrow(outside) test spectra lie outside
the domain. Their predictions are about as accurate as the others (mean
absolute error r sprintf("%.2f", mean(errors[!outside$inside])) against
r sprintf("%.2f", mean(errors[outside$inside]))% K): the flag says that the
model extrapolates, not that the prediction is wrong, but such samples
deserve a check in routine analysis. For a calibration curve, predict()
flags the concentrations outside the range of the standards
(outside_range).
A calibration is tied to the instrument that recorded its spectra, the primary instrument. On a secondary instrument, or on the same one after a change of its optics, the spectra differ, and so do the predictions. With no second instrument for these samples, we simulate one: its response tilts by ±15% across the spectral range, its wavelength axis is shifted by one channel, and it adds an offset:
wl_x <- as.numeric(colnames(x)) response <- 1 + 0.3 * (wl_x - mean(wl_x)) / diff(range(wl_x)) secondary <- sweep(cbind(x[, -1], x[, ncol(x)]), 2, response, "*") + 0.002
The PLS model of potassium predicts the test samples measured on this
instrument badly. Twenty calibration samples, measured on both
instruments, are enough to transfer their spectra: by direct
standardization (ds()), a linear map of the whole spectrum, or by
spectral space transformation (sst()), which moves the secondary spectra by
the difference between the instruments in the principal components they
share. Instead of the spectra, slope_bias() corrects the predictions by a
straight line:
predict_k <- \(spectra) drop(predict(cv, newdata = data.frame(x = I(spectra)), ncomp = ncomp)) set.seed(2) transfer_samples <- sample(which(is_calibration), 20) secondary_test <- secondary[!is_calibration, ] by_ds <- ds(x[transfer_samples, ], secondary[transfer_samples, ]) by_sst <- sst(x[transfer_samples, ], secondary[transfer_samples, ]) by_slope_bias <- slope_bias(predict_k(secondary[transfer_samples, ]), data$K[transfer_samples]) prediction_error <- \(predicted) sqrt(mean((predicted - test$K)^2)) transfer_errors <- c( primary = prediction_error(predict_k(x[!is_calibration, ])), secondary = prediction_error(predict_k(secondary_test)), ds = prediction_error(predict_k(predict(by_ds, secondary_test))), sst = prediction_error(predict_k(predict(by_sst, secondary_test))), slope_bias = prediction_error(predict(by_slope_bias, predict_k(secondary_test))) ) round(transfer_errors, 2) # RMSEP, % K
Direct standardization and SST bring the error of the test samples back
to that of the primary instrument, r sprintf("%.2f", transfer_errors[["primary"]])% K,
from r sprintf("%.2f", transfer_errors[["secondary"]])% K. The slope and bias
correction removes most of the bias, but not the distortion of the
spectra, which changes the ranking of the samples. pds(), piecewise
direct standardization, suits local differences such as shifts of the
axis, but its transfer matrix has one row and one column per channel
(r ncol(x) here), heavy for whole spectra.
The measurements are numbered in the order in which they were acquired. The carbon line, the internal standard of these calibrations, follows the ablated mass and the plasma, so a control chart of its intensity shows the stability of the instrument over the acquisition. The first 100 measurements set the center and the standard deviation, and an exponentially weighted moving average (EWMA), sensitive to slow drifts, follows the others:
carbon <- control_chart(intensities$C, order = data$Measurement, method = "ewma", baseline = 100) plot_control_chart(carbon, xlab = "Measurement (acquisition order)", ylab = "EWMA of the C I 247.86 nm intensity")
runs <- rle(carbon$signal) longest <- which.max(ifelse(runs$values, runs$lengths, 0)) run_end <- cumsum(runs$lengths)[longest] run_start <- run_end - runs$lengths[longest] + 1
The intensity changes by blocks between the sessions of the acquisition
(the gaps in the numbering), and the chart signals r sum(carbon$signal)
measurements; the longest run of signals, from measurement
r carbon$order[run_start] to r carbon$order[run_end], is a drop of the
intensity. Dividing by the carbon line compensates for such changes in the
calibrations above. In routine analysis, a check sample measured at regular
intervals, charted the same way (method = "shewhart", "ewma" or
"cusum"), separates a drift of the instrument from a change of the
samples.
calibration_curve(); resonance lines often curve at high
concentration.nas()), they are lower bounds.ds(), sst()
or pds(), or correct its predictions with slope_bias().The last vignette, vignette("plasma-diagnostics"), measures the plasma
itself: electron density, temperature and self-absorption.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.