cleanFlux: Cleaning eddy covariane flux measurements

Description Usage Arguments Details Value Author(s) References Examples

View source: R/cleanFlux.R

Description

This is the main function of the RFlux library. It performs the data cleaning procedure described by Vitale et al (2019).

Usage

1
2
cleanFlux(path_workset, path_ecmd, path_output=NULL, FileName=NULL,
          plotQC=FALSE, storage=FALSE)

Arguments

path_workset

path where the workset file generated from function ecworkset is stored.

path_ecmd

path where the eddy covariance metadata file (CC-xxx_ecmd.csv) is stored.

path_output

path where the output file will be stored. Default is NULL.

FileName

file name for the output file. Deafult is NULL.

plotQC

Logical. Should the details of data cleaning procedure be saved. If TRUE two .jpeg files (Details and Synthesis) for each flux variables will be stored in the path specified by path_output. Default is FALSE.

storage

Logical. Should the Net Ecosystem Exchange flux take into account the CO2 storage term. Default is FALSE.

Details

Comparing statistics estimated by qcStat with two threshold values, each test returns one of 3 possible statements:

SevEr:

if the test provides strong evidence about the presence of a specific source of systematic error.

ModEr:

if the test provides only weak evidence about the presence of a specific source of systematic error.

NoEr:

if the test does not provide evidence about the presence of a specific source of systematic error.

The data cleaning is based on a two-step procedure. In the first stage, fluxes that inherited at least one SevEr statement are rejected, while fluxes that inherited no SevEr statements and any number of ModEr statements are retained. In the second stage, flux data that inherited no SevEr statement are subject to an outlier detection procedure and only flux data that are both detected as outlier and inherited at least a ModEr statement are conclusively rejected. This implies that data points that inherited any number of ModEr statements but were not detected to be outliers, as well as outliers which showed no evidence of systematic errors, are retained in the dataset and can be used for any analysis or modeling purposes.

Value

Returns a dataframe containing:

TIMESTAMP_START

ISO timestamp start of averaging period (format: yyyymmddHHMM).

TIMESTAMP_END

ISO timestamp end of averaging period (format: yyyymmddHHMM).

H_UNCLEANED

Sensible heat turbulent flux (no storage correction, uncleaned).

H

Sensible heat turbulent flux (no storage correction, cleaned).

H_DATA_FLAG

Flag for H (0: observed flux for which any quality control (QC) tests provided negligible evidences of error; 1: outlying flux rejected because at least one of the QC tests provided a moderate evidence of error; 2: flux removed because at least one of the QC test provided a severe evidence of error).

LE_UNCLEANED

Latent heat turbulent flux (no storage correction, uncleaned).

LE

Latent heat turbulent flux (no storage correction, cleaned).

LE_DATA_FLAG

Flag for LE (0: observed flux for which any quality control (QC) tests provided negligible evidences of error; 1: outlying flux rejected because at least one of the QC tests provided a moderate evidence of error; 2: flux removed because at least one of the QC test provided a severe evidence of error).

FC

Carbon Dioxide (CO2) turbulent flux (no storage correction).

SC

Carbon Dioxide (CO2) storage flux.

NEE_UNCLEANED

Net Ecosystem Exchange (uncleaned).

NEE

Net Ecosystem Exchange (cleaned).

NEE_DATA_FLAG

Flag for NEE (i.e., 0: observed flux for which any quality control (QC) tests provided negligible evidences of error; 1: outlying flux rejected because at least one of the QC tests provided a moderate evidence of error; 2: flux removed because at least one of the QC test provided a severe evidence of error).

H_OUTLYING_FLAG

Flag for H denoting outliers (0: no outlying flux; 1: outlying flux).

LE_OUTLYING_FLAG

Flag for LE denoting outliers (0: no outlying flux; 1: outlying flux).

NEE_OUTLYING_FLAG

Flag for NEE denoting outliers (0: no outlying flux; 1: outlying flux).

H_FMR_STAT

Fraction of Missing Records in raw, high-frequency, data used for H flux estimation.

H_FMR_FLAG

Flag for the FMR test for H (0: negligible evidences of error, IF FMR<5; 1: moderate evidences of error, 1 IF 5FMR15; 2: severe evidences of error, 2 IF FMR>15).

H_LGD_STAT

Longest Gap Duration in raw, high-frequency, data used for H flux estimation.

H_LGD_FLAG

Flag for the LGD test for H (0: negligible evidences of error, IF LGD<90; 1: moderate evidences of error, IF 90LGD180; 2: severe evidences of error, IF LGD>180).

LE_FMR_STAT

Fraction of Missing Records in raw, high-frequency, data used for LE flux estimation.

LE_FMR_FLAG

Flag for the FMR test for LE (0: negligible evidences of error, IF FMR<5; 1, moderate evidences of error, IF 5FMR15; 2, severe evidences of error, IF FMR>15).

LE_LGD_STAT

Longest Gap Duration in raw, high-frequency, data used for LE flux estimation.

LE_LGD_FLAG

Flag for the LGD test for LE (0: negligible evidences of error, IF LGD<90; 1: moderate evidences of error, IF 90LGD180; 2: severe evidences of error, IF LGD>180).

FC_FMR_STAT

Fraction of Missing Records in raw, high-frequency, data used for FC flux estimation.

FC_FMR_FLAG

Flag for the FMR test for FC (0: negligible evidences of error, IF FMR<5; 1: moderate evidences of error, IF 5FMR15; 2: severe evidences of error, IF FMR>15).

FC_LGD_STAT

Longest Gap Duration in raw, high-frequency, data used for FC flux estimation.

FC_LGD_FLAG

Flag for the LGD test for FC (0: negligible evidences of error, IF LGD<90; 1: moderate evidences of error, IF 90LGD180; 2: severe evidences of error, IF LGD>180).

SA_DIAG_FLAG

Flag for Sonic Anemometer (SA) instrumental diagnostics (0: negligible evidences of error; 2: severe evidences of error).

GA_DIAG_FLAG

Flag for gas analyzer (GA) instrumental diagnostics (0: negligible evidences of error; 2: severe evidences of error).

WD

Wind direction.

WSECT_FLAG

Footprint quality flag indicating periods when wind was blowing from directions known to significantly affect the turbulent flow (0: negligible evidences of error; 2: severe evidences of error).

H_LSR_STAT

Statistic of the Low Signal Resolution test for H.

H_LSR_FLAG

Flag for the LSR test for H (0: negligible evidences of error, IF LSR_STAT>0.995; 1: moderate evidences of error, IF 0.99LSR_STAT0.995; 2: severe evidences of error, IF LSR_STAT<0.99).

LE_LSR_STAT

Statistic of the Low Signal Resolution test for LE.

LE_LSR_FLAG

Flag for the LSR test for LE (0: negligible evidences of error, IF LSR_STAT>0.995; 1: moderate evidences of error, IF 0.99LSR_STAT0.995; 2: severe evidences of error, IF LSR_STAT<0.99).

FC_LSR_STAT

Statistic of the Low Signal Resolution test for FC.

FC_LSR_FLAG

Flag for the LSR test for FC (0: negligible evidences of error, IF LSR_STAT>0.995; 1: moderate evidences of error, IF 0.99 LSR_STAT0.995; 2: severe evidences of error, IF LSR_STAT<0.99).

W_HF5_STAT

Statistic of the homogeneity test applied on vertical wind velocity fluctuations (percentage of data exceeding μ+/-5σ).

W_HF5_FLAG

Flag for the homogeneity test applied on vertical wind velocity fluctuations (0: negligible evidences of error, IF HF5_STAT<2; 1: moderate evidences of error, IF 2HF5_STAT4; 2: severe evidences of error, IF HF5_STAT>4).

W_HF10_STAT

Statistic of the homogeneity test applied on vertical wind velocity fluctuations (percentage of data exceeding μ+/-10σ).

W_HF10_FLAG

Flag for the homogeneity test applied on vertical wind velocity fluctuations (0: negligible evidences of error, IF HF10_STAT<0.5; 1: moderate evidences of error, IF 0.5HF10_STAT1; 2: severe evidences of error, IF HF10_STAT>1).

W_HD5_STAT

Statistic of the homogeneity test applied on differenced vertical wind velocity (percentage of data exceeding μ+/-5σ).

W_HD5_FLAG

Flag for the homogeneity test applied on differenced vertical wind velocity (0: negligible evidences of error, IF HD5_STAT<2; 1: moderate evidences of error, IF 2HD5_STAT4; 2: severe evidences of error, IF HD5_STAT>4).

W_HD10_STAT

Statistic of the homogeneity test applied on differenced vertical wind velocity (percentage of data exceeding μ+/-10σ).

W_HD10_FLAG

Flag for the homogeneity test applied on differenced vertical wind velocity (0: negligible evidences of error, IF HD10_STAT<0.5; 1: moderate evidences of error, IF 0.5HD10_STAT1; 2: severe evidences of error, IF HD10_STAT>1).

T_SONIC_HF5_STAT

Statistic of the homogeneity test applied on sonic temperature fluctuations (percentage of data exceeding μ+/-5σ).

T_SONIC_HF5_FLAG

Flag for the homogeneity test applied on sonic temperature fluctuations (0: negligible evidences of error, IF HF5_STAT<2; 1: moderate evidences of error, IF 2HF5_STAT4; 2: severe evidences of error, IF HF5_STAT>4).

T_SONIC_HF10_STAT

Statistic of the homogeneity test applied on sonic temperature fluctuations (percentage of data exceeding μ+/-10σ).

T_SONIC_HF10_FLAG

Flag for the homogeneity test applied on sonic temperature fluctuations (0: negligible evidences of error, IF HF10_STAT<0.5; 1: moderate evidences of error, IF 0.5HF10_STAT1; 2: severe evidences of error, IF HF10_STAT>1).

T_SONIC_HD5_STAT

Statistic of the homogeneity test applied on differenced sonic temperature (percentage of data exceeding μ+/-5σ).

T_SONIC_HD5_FLAG

Flag for the homogeneity test applied on differenced sonic temperature (0: negligible evidences of error, IF HD5_STAT<2; 1: moderate evidences of error, IF 2HD5_STAT4; 2: severe evidences of error, IF HD5_STAT>4).

T_SONIC_HD10_STAT

Statistic of the homogeneity test applied on differenced sonic temperature (percentage of data exceeding μ+/-10σ).

T_SONIC_HD10_FLAG

Flag for the homogeneity test applied on differenced sonic temperature (0: negligible evidences of error, IF HD10_STAT<0.5; 1: moderate evidences of error, IF 0.5HD10_STAT1; 2: severe evidences of error, IF HD10_STAT>1).

H2O_HF5_STAT

Statistic of the homogeneity test applied on water vapor fluctuations (percentage of data exceeding μ+/-5σ).

H2O_HF5_FLAG

Flag for the homogeneity test applied on water vapor fluctuations (0: negligible evidences of error, IF HF5_STAT<2; 1: moderate evidences of error, IF 2HF5_STAT4; 2: severe evidences of error, IF HF5_STAT>4).

H2O_HF10_STAT

Statistic of the homogeneity test applied on water vapor fluctuations (percentage of data exceeding μ+/-10σ).

H2O_HF10_FLAG

Flag for the homogeneity test applied on water vapor fluctuations (0: negligible evidences of error, IF HF10_STAT<0.5; 1: moderate evidences of error, IF 0.5HF10_STAT1; 2: severe evidences of error, IF HF10_STAT>1).

H2O_HD5_STAT

Statistic of the homogeneity test applied on differenced water vapor (percentage of data exceeding μ+/-5σ).

H2O_HD5_FLAG

Flag for the homogeneity test applied on differenced water vapor (0: negligible evidences of error, IF HD5_STAT<2; 1: moderate evidences of error, IF 2HD5_STAT4; 2: severe evidences of error, IF HD5_STAT>4).

H2O_HD10_STAT

Statistic of the homogeneity test applied on differenced water vapor (percentage of data exceeding μ+/-10σ).

H2O_HD10_FLAG

Flag for the homogeneity test applied on differenced water vapor (0: negligible evidences of error, IF HD10_STAT<0.5; 1: moderate evidences of error, IF 0.5HD10_STAT1; 2: severe evidences of error, IF HD10_STAT>1).

CO2_HF5_STAT

Statistic of the homogeneity test applied on carbon dioxide fluctuations (percentage of data exceeding μ+/-5σ).

CO2_HF5_FLAG

Flag for the homogeneity test applied on carbon dioxide fluctuations (0: negligible evidences of error, IF HF5_STAT<2; 1: moderate evidences of error, IF 2HF5_STAT4; 2: severe evidences of error, IF HF5_STAT>4).

CO2_HF10_STAT

Statistic of the homogeneity test applied on carbon dioxide fluctuations (percentage of data exceeding μ+/-10σ).

CO2_HF10_FLAG

Flag for the homogeneity test applied on carbon dioxide fluctuations (0: negligible evidences of error, IF HF10_STAT<0.5; 1: moderate evidences of error, IF 0.5HF10_STAT1; 2: severe evidences of error, IF HF10_STAT>1).

CO2_HD5_STAT

Statistic of the homogeneity test applied on differenced carbon dioxide (percentage of data exceeding μ+/-5σ).

CO2_HD5_FLAG

Flag for the homogeneity test applied on differenced carbon dioxide (0: negligible evidences of error, IF HD5_STAT<2; 1: moderate evidences of error, IF 2HD5_STAT4; 2: severe evidences of error, IF HD5_STAT>4).

CO2_HD10_STAT

Statistic of the homogeneity test applied on differenced carbon dioxide (percentage of data exceeding μ+/-10σ).

CO2_HD10_FLAG

Flag for the homogeneity test applied on differenced carbon dioxide (0: negligible evidences of error, IF HD10_STAT<0.5; 1: moderate evidences of error, IF 0.5HD10_STAT1; 2: severe evidences of error, IF HD10_STAT>1).

W_KID_STAT

Kurtosis Index of Differenced vertical wind velocity.

W_KID_FLAG

Flag for the W_KID_STAT (0: negligible evidences of error, IF KID_STAT<30; 1: moderate evidences of error, IF 30KID_STAT50; 2: severe evidences of error, IF KID_STAT>50).

T_SONIC_KID_STAT

Kurtosis Index of Differenced sonic temperature.

T_SONIC_KID_FLAG

Flag for the T_SONIC_KID_STAT (0: negligible evidences of error, IF KID_STAT<30; 1: moderate evidences of error, IF 30KID_STAT50; 2: severe evidences of error, IF KID_STAT>50).

H2O_KID_STAT

Kurtosis Index of Differenced water vapor.

H2O_KID_FLAG

Flag for the H2O_KID_STAT (0: negligible evidences of error, IF KID_STAT<30; 1: moderate evidences of error, IF 30KID_STAT50; 2: severe evidences of error, IF KID_STAT>50).

CO2_KID_STAT

Kurtosis Index of Differenced carbon dioxide.

CO2_KID_FLAG

Flag for the CO2_KID_STAT (0: negligible evidences of error, IF KID_STAT<30; 1: moderate evidences of error, IF 30KID_STAT50; 2: severe evidences of error, IF KID_STAT>50).

ITC_STAT

Statistic of the Integral Turbulence Characteristics test (Foken and Wichura, 1996).

ITC_FLAG

Flag for the ITC test (0: negligible evidences of error, IF ITC_STAT<30; 1: moderate evidences of error, IF 30ITC_STAT50; 2: severe evidences of error, IF ITC_STAT>50).

H_SCF_STAT

Spectral correction factor for H.

LE_SCF_STAT

Spectral correction factor for LE.

FC_SCF_STAT

Spectral correction factor for FC.

H_M98_STAT

Statistic of the nonstationarity ratio test by Mahrt (1998) for H.

H_M98_FLAG

Flag of the H_M98_STAT (0: negligible evidences of error, IF M_98_STAT<2; 1: moderate evidences of error, IF 2M98_STAT3; 2: severe evidences of error, IF M98_STAT>3).

LE_M98_STAT

Statistic of the nonstationarity ratio test by Mahrt (1998) for LE.

LE_M98_FLAG

Flag of the LE_M98_STAT (0: negligible evidences of error, IF M_98_STAT<2; 1: moderate evidences of error, IF 2M98_STAT3; 2: severe evidences of error, IF M98_STAT>3).

FC_M98_STAT

Statistic of the nonstationarity ratio test by Mahrt (1998) for FC.

FC_M98_FLAG

Flag of the FC_M98_STAT (0: negligible evidences of error, IF M_98_STAT<2; 1: moderate evidences of error, IF 2M98_STAT3; 2: severe evidences of error, IF M98_STAT>3).

CO2

Carbon Dioxide (CO2) in mole fraction of wet air.

CO2_SIGMA

Standard deviation of carbon dioxide in mole fraction of wet air.

H2O

Water (H2O) vapor mole fraction.

H2O_SIGMA

Standard deviation of water vapor mole fraction.

T_SONIC

Sonic temperature.

T_SONIC_SIGMA

Standard deviation of sonic temperature.

WS

Wind speed.

USTAR

Friction velocity.

W_SIGMA

Standard deviation of vertical velocity fluctuations.

U_SIGMA

Standard deviation of lateral velocity fluctuations (towards main-wind direction after coordinates rotation).

V_SIGMA

Standard deviation of lateral velocity fluctuations (cross main-wind direction after coordinates rotation).

ZL

Monin-Obukhov stability parameter.

MO_LENGTH

Monin-Obukhov length.

AT

Air temperature.

AP

Air pressure.

RHO

Air density.

CP

Air heat capacity.

CANOPY_HEIGHT

Canopy height.

SA_HEIGHT

Sonic anemometer height.

SA_NORTH_OFFSET

Sonic anemometer north offset.

INVALID_WIND_SECTOR_c1

Center of the first invalid wind sector, if any.

INVALID_WIND_SECTOR_w1

Width of the first invalid wind sector, if any.

INVALID_WIND_SECTOR_c2

Center of the second invalid wind sector, if any.

INVALID_WIND_SECTOR_w2

Width of the second invalid wind sector, if any.

INVALID_WIND_SECTOR_c3

Center of the third invalid wind sector, if any.

INVALID_WIND_SECTOR_w3

Width of the third invalid wind sector, if any.

Author(s)

Domenico Vitale

References

Vitale, D. Fratini, G. Bilancia, M. Nicolini, G. Sabbatini, S. Papale, D. A robust data cleaning procedure for eddy covariance flux measurements, Biogeosciences Discussions, 2019, pp 1-36, doi: https://doi.org/10.5194/bg-2019-270.

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
PATH_WORKSET <- system.file("extdata", "ecworkset_example.csv", package = "RFlux")
PATH_ECMD <- system.file("extdata", "DE-HoH_ecmd.csv", package = "RFlux")

cleanset <- cleanFlux(path_workset=PATH_WORKSET,
                      path_ecmd=PATH_ECMD,
                      path_output=NULL,
                      FileName=NULL,
                      plotQC=FALSE,
                      storage=TRUE)
str(cleanset)

domvit81/RFlux documentation built on Nov. 20, 2019, 8:02 a.m.