repackageRecoveredSGdata: repackage recovered data into hourly files with appropriate...

In jbrzusto/motus-R-package: Operate a server to process data for http://motus.org

Description

When raw SG records are recovered from a failed SD card, they need to be grouped into files so that the usual processing code can handle them. This function does so, then optionally submits the repackaged archive as a file upload to the motus server so that it will be incorporated into the appropriate receiver's DB and then processed.

Usage

repackageRecoveredSGdata(filename, serno, bootnum = 1000, path = getwd(),
  submit = FALSE)

Arguments

filename	path to file containing recovered records.
serno	character scalar receiver serial number; this is not available in records themselves, unfortunately. Care must be taken to ensure the correct serial number is given, otherwise data will appear to come from the wrong receiver. The GPS records are used to provide a sanity check on this, if a deployment for the receiver is known. example: "SG-1234BBBK5678"
bootnum	starting boot number of the data, if known. Default: 1000. This function tries to detect reboots by looking for large backward time jumps, but this doesn't always work, as SGs with cape GPS usually set their system time from battery-backed realtime clock clock before any file records are written.
path	character scalar: path in which to create the .zip archive of hourly files. Default: getwd()
submit	logical scalar: if TRUE, "upload" the file so it is processed by the motus data-processing server. Default: FALSE

Details

data records can sometimes be extracted from a failed SD card by directly reading blocks from the device and searching for strings of the appropriate form. With a VFAT file system and the usual SG filing method of "write .txt and .txt.gz files in tandem; delete .txt when .txt.gz is complete", the records written to and deleted as uncompressed .txt files are added to the back of the filesystem's free block list, so are rarely overwritten. When an SD card is represented by /dev/mmcblk1, the records can be extracted by a root shell (on whichever device holds the SD card) with this command:

strings /dev/mmcblk1 | grep -P '^(G|S|C|T|p[0-9]),[0-9]+([.0-9]+)?,' > recovered_records.txt

or, if the card is installed in a remote SG which has connected to the motus/SG server with reverse tunnel port 43210 (for example), this command on the server:

sshpass -p root ssh -T -f -p 43210 root@localhost "strings /dev/mmcblk1 | grep -P '^(G|S|C|T|p[0-9]),[0-9]+([.0-9]+)?,'" > recovered_records.txt &

will extract the records on the SG and send them via SSH to the file recovered_records.txt on the server.

Value

character scalar; path to the .zip archive of hourly files created by this function.

Author(s)

John Brzustowski jbrzusto@REMOVE_THIS_PART_fastmail.fm

jbrzusto/motus-R-package documentation built on May 18, 2019, 7:03 p.m.