extract_QC: Extract Quality Control Information

In lsigut/openeddy: Post-process eddy covariance data with ease

Extract Quality Control Information

Description

QC information stored in columns of data frame x is extracted and interpreted in respect to instruments or individual fluxes.

Usage

extract_QC(
  x,
  filters = c("spikesHF", "ampres", "dropout", "abslim", "skewkurt_hf", "skewkurt_sf",
    "discont_hf", "discont_sf", "timelag_hf", "timelag_sf", "attangle", "nonsteady",
    "missfrac", "scf", "wresid", "runs", "lowcov", "var", "humid", "LI7200", "LI7500"),
  rotation = c("double", "planar fit"),
  prefix = "[8]",
  split = "[/]",
  missfrac_thr = c(0.1, 0.1),
  scf_thr = c(2, 3),
  w_unrot_thr = c(0.35, 0.35),
  w_rot_thr = c(0.1, 0.15),
  lowcov_thr = c(-0.005, 0.005),
  ts_var_thr = c(2, 2),
  RH_thr = c(95, 95),
  LI7200_signal_thr = c(90, 80),
  LI7500_signal_thr = c(90, 80),
  simplify = TRUE
)

Arguments

x	A data frame with column names representing required variables.
filters	A character vector. A full set of supported filters to extract (default) or its subset.
rotation	A character string. Specifies the type of coordinate rotation applied. Allowed values are "double" and "planar fit". Can be abbreviated.
prefix	Character string containing a regular expression identifying the prefix of coded values. See 'extract_coded'.
split	Character string containing a regular expression identifying the separator of variable names in units attribute. See 'extract_coded'.
missfrac_thr	A numeric vector with 2 non-missing values. Represents thresholds for fraction of missing high frequency data within averaging period if filters include "missfrac". apply_thr flags the records higher than the given thresholds.
scf_thr	A numeric vector with 2 non-missing values. Represents thresholds for spectral correction factor if filters include "scf". apply_thr flags the records higher than the given thresholds.
w_unrot_thr	A numeric vector with 2 non-missing values. Represents thresholds for absolute value of unrotated w if filters include "wresid" and rotation = "double". apply_thr flags the records higher than the given thresholds.
w_rot_thr	A numeric vector with 2 non-missing values. Represents thresholds for absolute value of rotated w if filters include "wresid" and rotation = "planar fit". apply_thr flags the records higher than the given thresholds.
lowcov_thr	A numeric vector with 2 non-missing values. Represents thresholds for computed covariance if filters include "lowcov". apply_thr flags the records between given thresholds.
ts_var_thr	A numeric vector with 2 non-missing values. Represents thresholds for sonic temperature variance if filters include "var". apply_thr flags the records higher than the given thresholds.
RH_thr	A numeric vector with 2 non-missing values. Represents thresholds for relative humidity if filters include "humid". apply_thr flags the records higher than the given thresholds.
LI7200_signal_thr, LI7500_signal_thr	A numeric vector with 2 non-missing values. Represents thresholds for signal strength provided by LI-COR 7200 or 7500 if filters include "LI7200" or "LI7500". apply_thr flags the records lower than the given thresholds.
simplify	A logical value. Should be soft (suffix "_sf" in EddyPro column name) and hard (suffix "_hf" in EddyPro column name) flags extracted from EddyPro coded variables combined? See extract_coded.

Details

The data frame x is expected to have certain properties. It is required that it contains column names according to considered QC filters. In the case of coded variables, attribute units containing the format of coded variable is required to extract the coded QC information. See extract_coded.

Extracted QC information can be relevant to fluxes measured by given instrument(s), specific flux or is applicable to all fluxes. See 'Naming Strategy' below. Results are reported according to the QC scheme using QC flag range 0 - 2. In cases when extracted variable is checked against thresholds, apply_thr is used to assign flag values.

Check of missing data in averaging period (missfrac) takes into account number of valid records used for given averaging period. This number is further reduced by the sum of count of high frequency data spikes out of variables needed to compute covariance. Covariance pairs are w, u (Tau); w, ts (H); w, h2o (LE) and w, co2 (NEE).

Value

A data frame. Each column has attributes "varnames" and "units".

Extracted QC Checks

Filters from coded EddyPro columns (thresholds are set within EddyPro software). All of the filters extracted from coded variables are not additive.

• spikesHF: check of high frequency data spike percentage in averaging period against thresholds. HF in spikesHF means high frequency and distinguishes despiking done on raw data (spikesHF) from despiking done on low frequency data (spikesLF; see despikeLF).

• ampres: check of amplitude resolution in the recorded data.

• dropout: check of drop-outs, i.e. situations when the time series stays for “too long” on a value that is far from the mean.

• abslim: check of absolute limits when raw data are out of plausible range.

• skewkurt_sf and skewkurt_hf: check of skewness and kurtosis limits. If simplify = TRUE, skewkurt_sf and skewkurt_hf are combined into skewkurt column instead.

• discont_sf and discont_hf: check of discontinuities that lead to semi-permanent changes in the time series. If simplify = TRUE, discont_sf and discont_hf are combined into discont column instead.

• timelag_sf and timelag_hf: check of estimated timelags compared to the expected timelags. If simplify = TRUE, timelag_sf and timelag_hf are combined into timelag column instead.

• attangle: check of angle of attack.

• nonsteady: check of steadiness of horizontal wind.

Additional filters extracted from EddyPro output. The only additive test is "wresid".

• missfrac: check of missing data in averaging period against thresholds (default is 10%).

• scf: check of spectral correction factor against thresholds.

• wresid: check of mean unrotated w (double rotation) or w residual (planar fit) against thresholds.

• runs: check of runs with repeating values (see flag_runs). Flux values are rounded to 2 digits prior checking runs, except for Tau.

• lowcov: check of fluxes too close to zero (assuming issues during covariance computation)

• var: check of variances against thresholds.

• humid: check of relative humidity against thresholds.

• LI7200: check of CO2 and H2O signal strength against thresholds.

• LI7500: check of CO2 signal strength against thresholds (H2O signal not provided by the instrument).

Content and Format of Columns

• For details concerning coded variables see extract_coded.

• "file_records": number of valid records found in the raw file

• "used_records": number of valid records used at given averaging period. This number can also contain high frequency spikes that should be excluded.

• "u_spikes", "ts_spikes", "h2o_spikes" and "co2_spikes": number of high frequency spikes detected at given averaging period in respective variable. Values can be set to 0 if "used_records" already accounts for spikes.

• "Tau_scf", "H_scf", "LE_scf" and "co2_scf": spectral correction factor for given flux. Values are above 1.

• "w_unrot", "w_rot": unrotated and rotated w wind component, respectively (should be close to 0 m s-1).

Naming Strategy

QC prefixes (specifies which flux is affected by that QC output):

• qc_SA: applicable to fluxes relying only on SA (Tau, H)

• qc_GA: applicable to fluxes relying on GA (LE, NEE); only GA issues considered

• qc_SAGA: applicable to fluxes relying both on SA and GA (LE, NEE); SA and GA issues considered

• qc_Tau, qc_H, qc_LE, qc_NEE: only applicable for the respective flux

• qc_ALL: applicable to all fluxes

QC suffixes (specifies which QC check was performed to get this QC output):

• See 'Extracted QC Checks' above for the complete list.

Abbreviations

• QC: Quality Control

• SA: Sonic Anemometer

• GA: Gas Analyzer

• Tau: Momentum flux [kg m-1 s-2]

• H: Sensible heat flux [W m-2]

• LE: Latent heat flux [W m-2]

• NEE: Net ecosystem exchange [umol m-2 s-1]

• u: Longitudinal wind speed component [m s-1]

• w: Vertical wind speed component [m s-1]

• ts: Sonic temperature [degC]

• h2o: H2O concentration [mmol mol-1]

• co2: CO2 concentration [umol mol-1]

• RH: relative humidity

.

References

Foken, T., Wichura, B., 1996. Tools for quality assessment of surface-based flux measurements. Agric. For. Meteorol. 78, 83–105. https://doi.org/10.1016/0168-1923(95)02248-1

Vickers, D. and Mahrt, L., 1997. Quality Control and Flux Sampling Problems for Tower and Aircraft Data. Journal of Atmospheric and Oceanic Technology, 14(3), 512-526. https://doi.org/10.1175/1520-0426(1997)014<0512:QCAFSP>2.0.CO;2

Mauder, M., Cuntz, M., Drue, C., Graf, A., Rebmann, C., Schmid, H.P., Schmidt, M., Steinbrecher, R., 2013. A strategy for quality and uncertainty assessment of long-term eddy-covariance measurements. Agric. For. Meteorol. 169, 122-135. https://doi.org/10.1016/j.agrformet.2012.09.006

McGloin, R., Sigut, L., Havrankova, K., Dusek, J., Pavelka, M., Sedlak, P., 2018. Energy balance closure at a variety of ecosystems in Central Europe with contrasting topographies. Agric. For. Meteorol. 248, 418-431. https://doi.org/10.1016/j.agrformet.2017.10.003

See Also

extract_coded and apply_thr.

lsigut/openeddy documentation built on Jan. 15, 2025, 8:14 a.m.