noise_procedures: Record / add noise

In bpollner/aquap2: Multivariate Data Analysis Tools for R including Aquaphotomics Methods

Record / add noise

Description

Record a special noise-data file on your spectroscopy device, and use this file to calculate data that can then be used to specifically add 'custom' noise to your datasets. Alternatively, if you do not want to record a specific noise file, you can also add static noise (with mean=0) to your datasets. Use the parameter noi_addMode in the settings file to choose from one of the modes of adding noise, please see section 'Modes for noise calculation' for details.

Details

The noise data file is an R-data file generated / imported via the gfd) function, but containing only noise-spectra recorded following the recommendations below. This R-data file (where the factory-default filename is 'NoiseData') is then moved by the user in to the AQUAP2SH folder. When generating datasets (a 'cube') using the function gdmm, it depends on whether you choose to add noise to the datasets or not (parameter spl.do.noise in the analysis procedure resp. your input. If, and only if, you choose to add noise, first (and only once per R-session) the noise R-data file is read in and used to calculate the specific noise distribution. It is stored in an object where the name is starting with nd_ (for 'noise distribution') in the search-path object aquap2_globalData; (ls(aquap2_globalData, all.names=T)). This noise distribution is then used to specifically add, according to the selected mode (see section 'Modes for noise calculation' below), noise to each single wavelength in each single observation in the individual dataset within the cube. If you do not want to use a specific noise-data file, you can set the noise mode to 'static' (parameter noi_addMode in the settings file).

Procedure

The procedure to work with noise-data files and use them to obtain a wavelength-specific noise distribution consists of the following steps:

Record noise spectra

Use the function genNoiseRecExp to generate a folder structure for an experiment, then record the noise-spectra, for what the following procedure is suggested: XXX. Finally, use gfd to import the noise-data and create the R-data file (in the folder 'R-data' in the working directory of the experiment).

Move R-data file

Move the resulting R-data file containing the noise- spectra from the R-data folder into your AQUAP2SH folder, i.e. the folder also containing e.g. the settings.r file.

Choose to add noise to dataset

In your actual experiment, in order to add noise to the dataset(s) you have to set the parameter spl.do.noise in the analysis procedure to TRUE. Alternatively, you can override the value from 'spl.do.noise' in the analysis procedure via the ... argument in getap in gdmm - see examples.

Specify noise-data file

In your actual experiment, specify the name of the file (residing in the folder AQUAP2SH) containing the noise-spectra either in the metadata or at the argument noiseFile in the function gdmm - see examples.

Evaluate difference

Use any of the provided methods to check if there is a difference between the datasets without and with added noise.

Modes for noise calculation

Via the parameter noi_addMode in the settings file it can be specified which one of the available methods to calculate and add noise to the actual dataset should be used. The following values for noi_addMode are possible:

sdNorm

Factory default. Standard deviation based, sampling from normal distribution. The noise distribution is described by Noise_Mean +/- Noise_sd whithin each single wavelength of the noise dataset. If noise should be added now to an actual dataset, for each single observation and there for each single wavelength the noise-value to be added to the actual spectral value can be described by noiseValue = sample(rnorm(noi_sampleSize, mean=Noise_Mean, sd=Noise_sd),1). Additionally, all values outside the range of the noise-data in this specific wavelength get clipped. (noi_sampleSize is a parameter in the settings file.)

sdUnif

Standard deviation based, sampling from uniform distribution. The noise distribution is described by Noise_Mean +/- Noise_sd whithin each single wavelength of the noise dataset. If noise should be added now to an actual dataset, for each single observation and there for each single wavelength the noise-value to be added to the actual spectral value can be described by noiseValue = sample(runif(noi_sampleSize, min=Noise_Mean-Noise_sd, max=Noise_Mean-Noise_sd),1). (noi_sampleSize is a parameter in the settings file.)

extrema

The noise distribution simply represents the range of absorbance values within each single wavelength over all observations of the noise dataset. If noise should be added now to a dataset, for each single observation and there for each single wavelength a random process is selecting one of the two values from the corresponding wavelength in the noise distribution to be added to the original NIR data. In other words, for each single wavelength the original NIR-data absorbance values get increased or reduced by the maximum of (positive or negative) noise occuring at this specific wavelength.

static

For this mode, no noise-data file is required. For every observation in the actual dataset and there for every single wavelength, a static noise normal-distribution is calculated using the parameters noi_sampleSize and noi_staticValue from the settings file: From the resulting pool of values then one is sampled and used to add to the spectral value of the actual dataset. (noiseValue = sample(rnorm(noi_sampleSize, mean=0, sd=noi_staticValue),1))

See Also

gdmm, settings_file, genFolderStr

Other Noise procedures: do_addNoise(), genNoiseRecExp()

Examples

## Not run: 
fd <- gfd()
cu <- gdmm(fd, getap(spl.do.noise=TRUE))
cu <- gdmm(fd, getap(spl.do.noise=TRUE), noiseFile="FooBar") # instead of the 
# default noise-data filename resp. the filename specified in the metadata 
# file, use instead the noise-data file named 'FooBar'.
plot_pca(cu)

## End(Not run)

bpollner/aquap2 documentation built on March 29, 2024, 7:33 a.m.