ltdl.fix: Replace Negative Values Representing Less Than Detects in a...

In rgr: Applied Geochemistry EDA

Description

Function to process a numeric vector to replace negative values representing less than detects (<DLs) with positive half that value. This permits processing of these effectively categorical data as real numbers and their display on logarithmically scaled axes. Some software packages replace blank fields that should be interpreted as NAs, i.e. no information, with zeros, the facility is provided to replace any zero values with NAs. In other instances data files have been built using an integer code, e.g., -9999, to indicate 'no data', i.e. the equivalent of NAs. The facility is provided to replace any so coded values with NAs. If required, all <values and may be replaced withNAs, e.g. when estimating analytical precision with anova1 using only duplicate analyses with >DL values.

A report of the changes made is displayed on the current device.

For processing data matrices or data frames, see ltdl.fix.df.

Usage

ltdl.fix(x, negs2na = FALSE, zero2na = FALSE, coded = NA)

Arguments

x	name of the vector to be processed.
negs2na	to replace any -ve values with NAs, set negs2na = TRUE.
zero2na	to replace any zero values with NAs, set zero2na = TRUE.
coded	to replace any numeric coded values, e.g., -9999 with NAs, set coded = -9999.

Value

A numeric vector identical to that input but where any negative values have been replaced by half their positive values, or optionally by NAs, and optionally any zero or numeric coded values have been replaced by NAs.

Note

If data are being accessed through an ODBC link to a database, rather than from a data frame that can be processed by ltdl.fix.df, it may be important to run this function on the retrieved vector prior to any subsequent processing. The necessity for such vector processing can be ascertained using the range function, e.g., range(na.omit(x)) or range(x, na.rm = TRUE), where x is the variable name, to determine the presence of any negative values. The presence of any NAs in the vector will return NAs by range if they are not removed.

Great care needs to be taken when processing data where a large proportion of the data are less than detects (<value). In such cases parametric statistics have limited value, and can be missleading. Records should be kept of variables containing <values, and the fixed replacement values changed in tables for reports to the appropriate <values. Thus, in tables of percentiles the <value should replace the fixed value computed from absolute(-value)/2. Various rules have been proposed as to how many less than detects treated in this way can be tolerated before means, variances, etc. become biassed and of little value. Less than 5% in a large data set is usually tolerable, with greater than 10% concern increases, and with greater than 20% alternate procedures for processing the data should be sought, for example, the procedures outlined in Helsel (2005). Alternately replacement non-detect values may be imputed with R packages robCompositions (Hron et al., 2010; Templ et al., 2011) or zCompositions (Martin-Fernandez et al., 2012; Palarea-Albaladejo and Martin-Fernandez, 2013), specifically function lrEM in the latter, both may be used with closed, constant-sum, data.

Author(s)

Robert G. Garrett

References

Helsel, D.R., 2005. Nondetects and Data Analysis: Statistics for Censored Data. John Wiley & Sons, Ltd., 250 p.

Hron, K., Templ, M. and Filzmoser, P., 2010. Imputation of missing values for compositional data using classical and robust methods. Computational Statistics and Data Analysis, 54(12):3095-3107. 3107.

Templ, M., Hron, K. and Filzmoser, P., 2010. robCompositions: An R-package for Robust Statistical Analysis of Composition data. In: Compositional Data Analysis: Theory and Applications, V. Pawowsky and A. Buccianti (Eds.), Chapter 25, pp. 342-355. John Wiley & Sons, Ltd.

Martin-Fernadez, J.A., Hron, K., Templ, M., Filzmoser, P. and Palarea-Albaladejo, J., 2012. Model-based replacement of rounded zeros in compositional data: classical and robust approaches. Computational Statistics and Data Analysis, 56(9):2688-2704.

Palarea-Albaladejo, J. and Martin-Fernandez, J.A., 2013. Values below detection limit in compositional chemical data. Analytica Chemica Acta, 764:32-43.

See Also

ltdl.fix.df

Examples

## Replace any missing data coded as -9999 with NAs and any remaining
## negative values representing less than detects with Abs(value)/2
data(fix.test)
x.fixed <- ltdl.fix(fix.test[, 3], coded = -9999)
x.fixed

## As above, and replace any zero values with NAs
x.fixed <- ltdl.fix(fix.test[, 3], coded = -9999, zero2na = TRUE)
x.fixed

## As above, but replace any negative values with NAs
x.fixed <- ltdl.fix(fix.test[, 3], negs2na = TRUE, coded = -9999, zero2na = TRUE)
x.fixed

## Make test data kola.o available, setting a -9999, indicating a
## missing pH measurement, to NA
data(kola.o)
attach(kola.o)
pH.fixed <- ltdl.fix(pH, coded = -9999)

## Display relationship between pH in one pH unit intervals and Cu in 
## O-horizon (humus) soil, extending the whiskers to the 2nd and 98th
## percentiles, finally removing the temporary data vector pH.fixed
bwplots(split(Cu,trunc(pH.fixed+0.5)), log = TRUE, wend = 0.02, 
	xlab = "Soil pH to the nearest pH unit",
	ylab = "Cu (mg/kg) in < 2 mm Kola O-horizon soil")
rm(pH.fixed)

## Or directly
bwplots(split(Cu,trunc(ltdl.fix(pH, coded = -9999)+0.5)), log = TRUE,  
	wend = 0.02, xlab = "Soil pH to the nearest pH unit",
	ylab = "Cu (mg/kg) in < 2 mm Kola O-horizon soil")

## Clean-up and detach test data
rm(x)
rm(x.fixed)
rm(pH.fixed)
detach(kola.o)

Example output

Loading required package: MASS
Loading required package: fastICA
  15 records checked, 3 NA(s) present
  1 value(s) coded -9999 set to NA
  1 -ve record(s) set to +ve half the negative value
 [1] 15.0  8.0 19.0  2.5 12.0   NA 17.0 10.0   NA   NA 11.0 15.0   NA  0.0 18.0
  15 records checked, 3 NA(s) present
  1 value(s) coded -9999 set to NA
  1 zero (abs(x) < 10^-5) record(s) set to NA
  1 -ve record(s) set to +ve half the negative value
 [1] 15.0  8.0 19.0  2.5 12.0   NA 17.0 10.0   NA   NA 11.0 15.0   NA   NA 18.0
  15 records checked, 3 NA(s) present
  1 value(s) coded -9999 set to NA
  1 zero (abs(x) < 10^-5) record(s) set to NA
  1 -ve record(s) set to NA
 [1] 15  8 19 NA 12 NA 17 10 NA NA 11 15 NA NA 18
  617 records checked, 0 NA(s) present
  1 value(s) coded -9999 set to NA
  0 -ve record(s) set to +ve half the negative value
  617 records checked, 0 NA(s) present
  1 value(s) coded -9999 set to NA
  0 -ve record(s) set to +ve half the negative value
Warning message:
In rm(x) : object 'x' not found
Warning message:
In rm(pH.fixed) : object 'pH.fixed' not found

rgr documentation built on May 2, 2019, 6:09 a.m.