read.adp.rdi: Read a Teledyne/RDI ADP File

Description Usage Arguments Details Value Memory considerations Author(s) References See Also

View source: R/adp.rdi.R

Description

Read a Teledyne/RDI ADCP file (called 'adp' in oce).

Usage

1
2
3
4
read.adp.rdi(file, from, to, by, tz = getOption("oceTz"), longitude = NA,
  latitude = NA, type = c("workhorse"), monitor = FALSE,
  despike = FALSE, processingLog, testing = FALSE,
  debug = getOption("oceDebug"), ...)

Arguments

file

a connection or a character string giving the name of the file to load. (For read.adp.sontek.serial, this is generally a list of files, which will be concatenated.)

from

indication of the first profile to read. This can be an integer, the sequence number of the first profile to read, or a POSIXt time before which profiles should be skipped, or a character string that converts to a POSIXt time (assuming UTC timezone). See “Examples”, and make careful note of the use of the tz argument. If from is not supplied, it defaults to 1.

to

an optional indication of the last profile to read, in a format as described for from. As a special case, to=0 means to read the file to the end. If to is not supplied, then it defaults to 0.

by

an optional indication of the stride length to use while walking through the file. If this is an integer, then by-1 profiles are skipped between each pair of profiles that is read, e.g. the default by=1 means to read all the data. (For RDI files only, there are some extra features to avoid running out of memory; see “Memory considerations”.)

tz

character string indicating time zone to be assumed in the data.

longitude

optional signed number indicating the longitude in degrees East.

latitude

optional signed number indicating the latitude in degrees North.

type

character string indicating the type of instrument.

monitor

boolean, set to TRUE to provide an indication of progress in reading the file, either by printing a dot for each profile or by writing a textual progress bar with txtProgressBar.

despike

if TRUE, despike will be used to clean anomalous spikes in heading, etc.

processingLog

if provided, the action item to be stored in the log. (Typically only provided for internal calls; the default that it provides is better for normal calls by a user.)

testing

logical value (IGNORED).

debug

a flag that turns on debugging. Set to 1 to get a moderate amount of debugging information, or to 2 to get more.

...

additional arguments, passed to called routines.

Details

As of 2016-09-25, this function has provisional functionality to read data from the new "SentinelV" series ADCP – essentially a combination of a 4 beam workhorse with an additional vertical centre beam.

If a heading bias had been set with the EB command during the setup for the deployment, then a heading bias will have been stored in the file's header. This value is stored in the object's metadata as metadata$heading.bias. Importantly, this value is subtracted from the headings stored in the file, and the result of this subtraction is stored in the objects heading value (in data$heading). It should be noted that read.adp.rdi() was tested for firmware version 16.30. For other versions, there may be problems. For example, the serial number is not recognized properly for version 16.28.

In Teledyne/RDI ADP data files, velocities are coded to signed 2-byte integers, with a scale factor being used to convert to velocity in metres per second. These two facts control the maximum recordable velocity and the velocity resolution, values that may be retrieved for an ADP object name d with d[["velocityMaximum"]] and d[["velocityResolution"]].

Value

An adp object, i.e. one inheriting from adp-class.

Memory considerations

For RDI files only, and only in the case where by is not specified, an attempt is made to avoid running out of memory by skipping some profiles in large input files. This only applies if from and to are both integers; if they are times, none of the rest of this section applies.

A key issue is that RDI files store velocities in 2-byte values, which is not a format that R supports. These velocities become 8-byte (numeric) values in R. Thus, the R object created by read.adp.rdi will require more memory than that of the data file. A scale factor can be estimated by ignoring vector quanties (e.g. time, which has just one value per profile) and concentrating on matrix properties such as velocity, backscatter, and correlation. These three elements have equal dimensions. Thus, each 4-byte slide in the data file (2 bytes + 1 byte + 1 byte) corresponds to 10 bytes in the object (8 bytes + 1 byte + 1 byte). Rounding up the resultant 10/4 to 3 for safety, we conclude that any limit on the size of the R object corresponds to a 3X smaller limit on file size.

Various things can limit the size of objects in R, but a strong upper limit is set by the space the operating system provides to R. The least-performant machines in typical use appear to be Microsoft-Windows systems, which limit R objects to about 2e6 bytes [3]. Since R routinely duplicates objects for certain tasks (e.g. for call-by-value in function evaluation), read.adp.rdi uses a safety factor in its calculation of when to auto-decimate a file. This factor is set to 3, based partly on the developers' experience with datasets in their possession. Multiplied by the previously stated safety factor of 3, this suggests that the 2 GB limit on R objects corresponds to approximately a 222 MB limit on file size. In the present version of read.adp.rdi, this value is lowered to 200 MB for simplicity. Larger files are considered to be "big", and are decimated unless the user supplies a value for the by argument.

The decimation procedure has two cases.

  1. Case 1. If from=1 and to=0 (or if neither from or to is given), then the intention is to process the full span of the data. If the input file is under 200 MB, then by defaults to 1, so that all profiles are read. For larger files, by is set to the ceiling of the ratio of input file size to 200 MB.

  2. Case 2. If from exceeds 1, and/or to is nonzero, then the intention is to process only an interior subset of the file. In this case, by is calculated as the ceiling of the ratio of bbp*(1+to-from) to 200 MB, where bbp is the number of file bytes per profile. Of course, by is set to 1, if this ratio is less than 1.

If the result of these calculations is that by exceeds 1, then messages are printed to alert the user that the file will be decimated, and also monitor is set to TRUE, so that a textual progress bar is shown.

Author(s)

Dan Kelley and Clark Richards

References

1. Teledyne-RDI, 2007. WorkHorse commands and output data format. P/N 957-6156-00 (November 2007). (Section 5.3 h details the binary format, e.g. the file should start with the byte 0x7f repeated twice, and each profile starts with the bytes 0x80, followed by 0x00, followed by the sequence number of the profile, represented as a little-endian two-byte short integer. read.adp.rdi uses these sequences to interpret data files.)

2. Teledyne-RDI, 2015. V Series output data format. P/N 95D-6022-00 (May 2015).

3. See Memory-limits for more on the 2 GB limit for R on windows machines (but note that this documentation erroneously states the unit as Gb, which is typically used for gigabits).

See Also

Other things related to adp data: [[,adp-method, [[<-,adp-method, adp-class, adpEnsembleAverage, adp, as.adp, beamName, beamToXyzAdp, beamToXyzAdv, beamToXyz, beamUnspreadAdp, binmapAdp, enuToOtherAdp, enuToOther, plot,adp-method, read.ad2cp, read.adp.nortek, read.adp.sontek.serial, read.adp.sontek, read.adp, read.aquadoppHR, read.aquadoppProfiler, read.aquadopp, subset,adp-method, summary,adp-method, toEnuAdp, toEnu, velocityStatistics, xyzToEnuAdp, xyzToEnu


oce documentation built on Aug. 29, 2017, 1:05 a.m.