reject_shots: Rejection of Outlying Laser Shots

View source: R/reject_shots.R

reject_shotsR Documentation

Rejection of Outlying Laser Shots

Description

Flags the outlying shots (spectra) of each sample of a LIBS dataset, such as shots with a weak or missed plasma, or shots on an inclusion or a contaminated spot, before the shots are averaged.

Usage

reject_shots(
  data,
  sample,
  method = c("intensity", "correlation"),
  cutoff = 3.5,
  wavelength = NULL,
  drop = FALSE,
  shot = NULL,
  scale = c("floor", "sample", "pooled")
)

Arguments

data

A data frame with one shot per row: a sample column and spectral columns named by their wavelengths (other columns are kept but not used).

sample

The column identifying the sample of each shot, unquoted or as a string.

method

The criteria: one or more of "intensity", "correlation" (both by default) and "distance".

cutoff

The robust z-score above which a shot is rejected. Default is 3.5.

wavelength

An optional wavelength range (nm) in which the criteria are computed, for example to avoid saturated lines.

drop

A logical: return only the kept shots, without the added columns (FALSE, default).

shot

An optional column (unquoted or as a string) giving the order of the shots in each sample, such as the shot number. Default is the order of the rows.

scale

The robust scale of the z-scores: "floor" (default), "sample" or "pooled" (see Details).

Details

Each shot is compared with the other shots of its sample, by robust z-scores of one or more criteria:

  • "intensity": the total intensity of the spectrum, on a log scale (its deviation is relative to the median shot of the sample). Both weak and unusually strong shots are flagged.

  • "correlation": the Pearson correlation r of the spectrum with the median spectrum of the sample, which detects changes of the shape of the spectrum (other lines, other line ratios) whatever its intensity. The z-scores are those of Fisher's z = \tanh^{-1}(r): correlations near 1 are bounded and skewed, so that small differences such as 0.998 and 0.9995 would otherwise look extreme. Only shots with a low correlation are flagged.

  • "distance": the Euclidean distance of the spectrum to the median spectrum of the sample, relative to the norm of the median spectrum, on a log scale, which combines both. Only far shots are flagged.

Scale. A z-score is the deviation of a shot from the median of its sample divided by a robust scale (the MAD). With the few shots of a sample (often 5 to 10), the MAD of the sample is unstable: when its shots happen to be very similar, a shot that differs by little gets a large z-score. With scale = "floor" (default), the scale of a sample is its own MAD, but not less than the pooled MAD of the deviations of all the samples (the typical shot-to-shot variability of the data set); "sample" uses the MAD of each sample alone, and "pooled" the pooled MAD for all samples, which suits samples of similar variability. In simulations of 8 shots per sample with noise varying between samples, "floor" with Fisher's z rejected 0.1% to 0.5% of the regular shots (against 2.5% to 5% with the MAD of each sample and the raw correlations), and still detected the shots of a plasma weakened by a quarter or more, or with an extra line of 10% of the strongest line. Lower cutoff to detect milder deviations.

A shot is rejected when any criterion exceeds cutoff (3.5 by default, following Iglewicz and Hoaglin, 1993). Samples with fewer than 3 shots are not checked. When a MAD is zero, the mean absolute deviation is used.

The result keeps the raw criteria (.intensity, the total intensity relative to the median shot of the sample; .correlation; .distance), easier to read than the z-scores, and the settings, which plot_shots() uses to show the shots and the rejections. To average the kept shots per sample, use average() on the result with drop = TRUE. In a recipe, use step_reject_shots().

Value

A tibble: data with the shot number in its sample .shot, a logical column .rejected, the criteria that rejected each shot in .reason (NA for kept shots), the raw criteria .intensity, .correlation and .distance, and the robust z-score of each criterion used (.intensity_z, .correlation_z, .distance_z). Its attribute "reject_shots" holds the settings. With drop = TRUE, the kept rows of data.

Author(s)

Christian L. Goueguel

References

  • Iglewicz, B., Hoaglin, D.C. (1993). How to Detect and Handle Outliers. ASQC Quality Press, Milwaukee.

  • Fisher, R.A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4):507-521.

See Also

plot_shots(), step_reject_shots(), average(), saturation_summary(), forageShots

Examples

data(forageShots)
# 8 shots of each of 20 forage samples
res <- reject_shots(forageShots, Measurement, shot = shot)
res[res$.rejected, c("Measurement", ".shot", ".reason", ".intensity", ".correlation")]
# the kept shots only
nrow(reject_shots(forageShots, Measurement, drop = TRUE))
# the former behavior, with the MAD of each sample alone
sum(reject_shots(forageShots, Measurement, scale = "sample")$.rejected)

specProc documentation built on Nov. 20, 2026, 5:11 p.m.