ltdl.fix.df: Replace Negative Values Representing Less Than Detects in a...

In rgr: Applied Geochemistry EDA

Description

Function to process a matrix or data frame to replace negative values representing less than detects (<value) with positive half that value. This permits processing of these effectively categorical data as real numbers and their display on logarithmically scaled axes. Some software packages replace blank fields that should be interpreted as NAs, i.e. no information, with zeros, the facility is provided to replace any zero values with NAs. In other instances data files have been built using an integer code, e.g., -9999, to indicate 'no data', i.e. the equivalent of NAs. The facility is provided to replace any so coded values with NAs. If required, all <values and may be replaced with NAs, e.g. when estimating analytical precision with anova1 using only duplicate analyses with >DL values. Any factor variables in the input matrix or data frame are passed to the output matrix or data frame.

If a single vector is to be processed, use ltdl.fix

A report of the changes made is displayed on the current device.

Usage

ltdl.fix.df(x, negs2na = FALSE, zero2na = FALSE, coded = NA)

Arguments

x	name of the matrix or data frame to be processed.
negs2na	to replace any -ve values with NAs, set negs2na = TRUE.
zero2na	to replace any zero values with NAs, set zero2na = TRUE.
coded	to replace any numeric coded values, e.g., -9999 with NAs, set coded = -9999.

Value

A matrix or data frame identical to that input but where any negative values have been replaced by half their positive values, or NAs, and optionally any zero values or numeric coded values have been replaced by NAs.

Note

Great care needs to be taken when processing data where a large proportion of the data are less than detects (<value). In such cases parametric statistics have limited value, and can be missleading. Records should be kept of variables containing <values, and the fixed replacement values changed in tables for reports to the appropriate <values. Thus, in tables of percentiles the <value should replace the fixed value computed from absolute(-value)/2. Various rules have been proposed as to how many less than detects treated in this way can be tolerated before means, variances, etc. become biassed and of little value. Less than 5% in a large data set is usually tolerable, with greater than 10% concern increases, and with greater than 20% alternate procedures for processing the data should be sought. For example, the procedures outlined in Helsel (2005). Alternately replacement non-detect values may be imputed with R packages robCompositions (Hron et al., 2010; Templ et al., 2011) or zCompositions (Martin-Fernandez et al., 2012; Palarea-Albaladejo and Martin-Fernandez, 2013), specifically function lrEM in the latter, both may be used with closed, constant-sum, data.

Author(s)

Robert G. Garrett and David Lorenz

References

Helsel, D.R., 2005. Nondetects and Data Analysis: Statistics for Censored Data. John Wiley & Sons, Ltd., 250 p.

Hron, K., Templ, M. and Filzmoser, P., 2010. Imputation of missing values for compositional data using classical and robust methods. Computational Statistics and Data Analysis, 54(12):3095-3107. 3107.

Templ, M., Hron, K. and Filzmoser, P., 2010. robCompositions: An R-package for Robust Statistical Analysis of Composition data. In: Compositional Data Analysis: Theory and Applications, V. Pawowsky and A. Buccianti (Eds.), Chapter 25, pp. 342-355. John Wiley & Sons, Ltd.

Martin-Fernadez, J.A., Hron, K., Templ, M., Filzmoser, P. and Palarea-Albaladejo, J., 2012. Model-based replacement of rounded zeros in compositional data: classical and robust approaches. Computational Statistics and Data Analysis, 56(9):2688-2704.

Palarea-Albaladejo, J. and Martin-Fernandez, J.A., 2013. Values below detection limit in compositional chemical data. Analytica Chemica Acta, 764:32-43.

See Also

ltdl.fix

Examples

## Replace any missing data coded as -9999 with NAs and any remaining
## negative values representing less than detects with Abs(value)/2
data(fix.test)
fix.test
fix.test.fixed <- ltdl.fix.df(fix.test, coded = -9999)
fix.test.fixed

## As above, and replace any zero values with NAs
fix.test.fixed <- ltdl.fix.df(fix.test, coded = -9999, zero2na = TRUE)
fix.test.fixed

## As above, but replace any negative values with NAs
fix.test.fixed <- ltdl.fix.df(fix.test, negs2na = TRUE, coded = -9999, zero2na = TRUE)
fix.test.fixed

## Clean-up
rm(fix.test.fixed)

Example output

Loading required package: MASS
Loading required package: fastICA
    F1 F2    X1    X2
C15  A  X    15    29
C14  A  Y     8    28
C13  C  Y    19    40
C12  B  X    -5    18
C11  C  X    12    30
B10  A  X    NA   -10
B09  B  Y    17    26
B08  B  X    10    NA
B07  A  Y    NA -9999
B06  C  Y    NA    NA
A05  A  Y    11    22
A04  B  X    15   -10
A03  A  Y -9999    35
A02  C  X     0    36
A01  A  Y    18    26
  n = 15 by p = 2 matrix checked, 5 NA(s) present
  2 factor variable(s) present
  2 value(s) coded -9999 set to NA
  3 -ve value(s) set to +ve half the negative value
    F1 F2   X1 X2
C15  A  X 15.0 29
C14  A  Y  8.0 28
C13  C  Y 19.0 40
C12  B  X  2.5 18
C11  C  X 12.0 30
B10  A  X   NA  5
B09  B  Y 17.0 26
B08  B  X 10.0 NA
B07  A  Y   NA NA
B06  C  Y   NA NA
A05  A  Y 11.0 22
A04  B  X 15.0  5
A03  A  Y   NA 35
A02  C  X  0.0 36
A01  A  Y 18.0 26
  n = 15 by p = 2 matrix checked, 5 NA(s) present
  2 factor variable(s) present
  2 value(s) coded -9999 set to NA
  1 zero (abs(x) < 10^-5) value(s) set to NA
  3 -ve value(s) set to +ve half the negative value
    F1 F2   X1 X2
C15  A  X 15.0 29
C14  A  Y  8.0 28
C13  C  Y 19.0 40
C12  B  X  2.5 18
C11  C  X 12.0 30
B10  A  X   NA  5
B09  B  Y 17.0 26
B08  B  X 10.0 NA
B07  A  Y   NA NA
B06  C  Y   NA NA
A05  A  Y 11.0 22
A04  B  X 15.0  5
A03  A  Y   NA 35
A02  C  X   NA 36
A01  A  Y 18.0 26
  n = 15 by p = 2 matrix checked, 5 NA(s) present
  2 factor variable(s) present
  2 value(s) coded -9999 set to NA
  1 zero (abs(x) < 10^-5) value(s) set to NA
  3 -ve value(s) set to NA
    F1 F2 X1 X2
C15  A  X 15 29
C14  A  Y  8 28
C13  C  Y 19 40
C12  B  X NA 18
C11  C  X 12 30
B10  A  X NA NA
B09  B  Y 17 26
B08  B  X 10 NA
B07  A  Y NA NA
B06  C  Y NA NA
A05  A  Y 11 22
A04  B  X 15 NA
A03  A  Y NA 35
A02  C  X NA 36
A01  A  Y 18 26

rgr documentation built on May 2, 2019, 6:09 a.m.