In sapfluxnet/sapfluxnetQC1: Package with functions related to sapfluxnet project data

Sometimes manual changes in the received data are necessary. If it is the case, all changes must be documented following established rules. This vignette sets those rules and explains the protocol to be followed to document the manual changes.

File characteristics and location

All changes for a site data must be documented in a txt file, located in the site folder of the Data folder (i.e. Data/XXX_YYY_ZZZ/).
Name of the file is the site code plus manual_changes (XXX_YYY_ZZZ_manual_changes.txt).
Lines must not exceed 80 characters, to ensure complete visibility in most of the screens and in the sfn_monitor app.

Example:

Data/ESP_VAL_SOR/ESP_VAL_SOR_manual_changes.txt

File content and organization

All manual changes, no matter how small, must be included and documented in the file. File content is a list, with one element for each change, coded by date of the change.

Example

  Data/ESP_VAL_SOR/ESP_VAL_SOR_manual_changes.txt


  2016/07/27: Fixed errors in the TIMESTAMP:
              TIMESTAMP presented errors of continuity due to bad conversion.
              Errors were detected in the QC process, but were not automatically
              fixed in the same process.
              In order to fix those errors, TIMESTAMP wrong values were manually
              changed. List of fixed TIMESTAMP values can be found below:
              1. 2007/02/06 12:00:00 changed to 2007/02/06 12:15:00 in row 208
              2. 2007/06/14 08:30:00 changed to 2007/06/14 08:00:00 in row 10567

  2016/07/30: Fixed wrong latitude in metadata:
              Latitude was wrongly added to the metadata, with no possible
              autofix.
              si_lat variable was changed from 42.9 to 38.2 in the ESP_VAL_SOR_metadata.xlsx
              file

Complete process

The complete process for manual changes is as follows:

1. Download the data to be changed

2. Locally perform the manual changes and save them. Create the mandatory txt file (as explained above)

3. Before uploading the changed-data the server must be prepared:

   • move the data from /data/metadata to /sapfluxnet/discarded_data:

     $ cd /data/metadata /data/metadata$ sudo mv FOO_FOO_metadata.xlsx /sapfluxnet/discarded_data/

   • remove the old data from /sapfluxnet/received_data:

     /data/metadata$ cd /sapfluxnet/received_data /sapfluxnet/received_data$ sudo rv FOO_FOO_metadata.xlsx

   • if data has been processed before in the production instance, status for the site must be resetted (in RStudio):

     ```

     df_reset_status('FOO_FOO') ```

4. Follow the usual upload process (upload form in sapfluxnet webpage) to re-send the changed data

5. Run the main_script.R to generate the new report

6. If a manual changes txt already exists in the data folder in the server, update the content to reflect the new changes. If not, upload the file created in step 2.

sapfluxnet/sapfluxnetQC1 documentation built on May 29, 2019, 1:50 p.m.