In poissonconsulting/nrp: Kootenay Lake Nutrient Restoration Program Data Loading

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

nrp

nrp is an R package developed for the Nutrient Restoration Program (NRP). It currently provides the following functionality for CTD and EMS data:

Read in raw data files
The package checks the validity of input data and does some cleaning and manipulation to make the data more informative and useful to the user.

For CTD data, the date and parameter units are extracted from the file meta data. The SiteID is pulled from the raw data filename, and validated by cross referencing it to known sites in the database.

For EMS data, the data is filtered to only include the parameters and sites of interest for the Nutrient Restoration Program. The data is then restructured in a 'wide' format that lends itself to easy analysis/plotting. During the process, parameters are assigned the correct units, and values that were measured as the detection limit are set to 0. Detection limit columns for each parameter are created to inform the user of the detection limit each time a 0 value is encountered. EMS metals and EMS standard are stored in separate tables in the database.

Upload data to the NRP SQLite database
When uploading new data to the database, the package checks to ensure that no duplicate data is present, and numerous checks are performed to ensure that the data's structure is correct and compatible with the database.

Download CTD data
Data can then be downloaded from the database with optional filtering by SiteID, date range, etc.

Installation

To install the latest version from GitHub

# install.packages("remotes")
remotes::install_github("poissonconsulting/nrp")

Demonstration

For simplicity, this demonstration uses the function nrp_create_db to create an empty, temporary database to read and write to, and uses some test data that travels with the package. When connecting to a proper database, just provide the file path to the database for the argument db_path in all function calls and the connection will be made automatically. Alternatively you can connect to a database in R and then supply that connection object for the argument db_path.

library(nrp)

# create empty database
# path = ":memory:" creates a temporary, in memory database
conn <- nrp_create_db(path = ":memory:") 

# provide a path to a .cnv file
path <-  system.file("extdata", "ctd/2018/KL1_27Aug2018008downcast.cnv",
                       package = "nrp", mustWork = TRUE)

# read in file
ctd_data <- nrp_read_ctd_file(path, db_path = conn)

# upload data to the database
nrp_upload_ctd(data = ctd_data, db_path = conn)

# download data from database, filtering by Date and SiteID and parameter
db_ctd_data <- nrp_download_ctd(start_date = "2018-08-27",
                            end_date = "2018-08-27",
                            sites = "KL1",
                            parameters = "all",
                            db_path = conn)

When updating EMS data, the user must supply their own data set which can be easily obtained using a function from the rems package: get_ems_data(which = "4yr"). The four year data set must be used as the two year data has different structure that does not contain all the parameters needed for use with nrp functions. Once you have the four year data set loaded in the environment, you can easily upload it to the database with nrp functions. You don't have to worry about uploading duplicate data as the nrp function for uploading will automatically trim the input data so that only new data is added to the database. For this demonstration we use a light weight sample of raw EMS data that travels with the package.

# read in raw ems data
path <-  system.file("extdata", "ems/test_ems.rds", package = "nrp", mustWork = TRUE)
ems_raw <- readRDS(path)

# extract into wide fromat that fits the nrp database, chooing either the
# 'metals' or 'standard' analysis_type
ems_data <- nrp_extract_ems(data = ems_raw, db_path = conn, analysis_type = "standard")

# upload data to database
nrp_upload_ems_standard(data = ems_data, db_path = conn)

# download data from database, filtering by Date, SiteID, and analysis_type. 
# You can also choose to exlude detection limit columns
db_ems_data_standard <- nrp_download_ems(start_date_time = "2018-07-03 13:48:00",
                                     end_date_time = "2018-07-31 13:28:00",
                                     sites = NULL,
                                     db_path = conn,
                                     analysis_type = "standard",
                                     show_detection_limits = TRUE)

There are additional functions in the nrp package that allow for easy downloading of other tables in the database.

# Lakes and BasinArm tables
lakes <- nrp_download_lakes(db_path = conn)
basinArm <- nrp_download_ctd_basin_arm(db_path = conn)

# site table
sites <- nrp_download_sites(db_path = conn)

# ctd visit table
ctd_visit <- nrp_download_ctd_visit(db_path = conn)

# Tip: the nrp functions can also use global options to automatically know
# what database to connect to. Once you've set the global option "nrp.db_path"
# for the database you want to connect to, you don't have to provide
# the db_path argument any more within that R session

options(nrp.db_path = conn) # "conn" can also be a file path to a database

# now we can do things like:
lakes <- nrp_download_lakes()


# we conclude the demo by closing the database connection
# If you are providing a file path to the database, you do not need to do this
readwritesqlite::rws_disconnect(conn)

Getting Help or Reporting an Issue

To report bugs/issues/feature requests, please file an issue.

How to Contribute

If you would like to contribute to the package, please see our CONTRIBUTING guidelines.

Code of Conduct

Please note that the nrp project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

License

The code is released under the Apache License 2.0

Copyright 2019 Province of British Columbia

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

poissonconsulting/nrp documentation built on Jan. 28, 2024, 4:14 p.m.