In scoyoc/rainDanceR: Import Data from Onset Data Loggers

Introduction

This R package imports data from Onset temperature, relative humidity, and precipitation (event) data loggers into R. Data collected in the field using Onset loggers are exported to comma delimited (csv) files using the Onset HOBOware application. This package imports the csv files into R and summarizes the data.

The structure of the csv files generated from HOBOware vary wildly. They can have anywhere from 4 to 10 columns and the logger details are usually in two "hidden" columns following the data. This makes data from the temperature, relative humidity, and precipitation loggers difficult to read into R. This package imports data collected by Onset loggers used in the Southeast Utah Group (SEUG) national parks long-term vegetation monitoring program (LTVMP) and then summarize these data. This package was written to be used with the dataprocessR package, https://github.com/scoyoc/dataprocessR, that exports the raw and summarized data to the SEUG LTVMP database (a Microsoft Access database).

knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library("raindancer")

Installation

This package is available on GitHub at https://github.com/scoyoc/raindancer. Dependent packages include dplyr, glue, lubridate, RODBC, stringr, tibble, tidyr, and utils. Suggested pacakges include janitor, knitr, rmarkdown, and readr.

if (!"devtools" %in% installed.packages()[, "Package"]) {
  install.packages("devtools")
}
devtools::install_github("scoyoc/raindancer")
library("raindancer")

Import Data into R

There are two functions that import the csv files generated from HOBOware into R, import_hobo_2008() and import_hobo(). Below is a table of Onset loggers that these functions can import data from.

Table 1. Onset loggers used in the SEUG LTVMP.

|Product |Element |Year Used | |---------------------------------|-----------|------------| |H07 Logger |PRCP |2008-2019 | |HOBO UA-003-64 Pendant Temp/Event|PRCP & TEMP|2019-present| |H08 Logger |TEMP |2008-2019 | |HOBO UA-001-64 Pendant Temp |TEMP |2019-present| |HOBO U23-001 Temp/RH |TEMP & RH |2019-present|

Using import_hobo()

The functions import_hobo_2008() and import_hobo() return a list with three components:
1. file_info is a one row data frame that contains information about the file, the logger, and data.
2. details is a data frame of logger and sampling event information.
3. data_raw is a data frame of raw data.

Lets start by usgin list.files() to bring a list of csv files into R.

file_list <- list.files(path = "C:/path/to/data", pattern = ".csv", 
                        full.names = TRUE, recursive = FALSE)

There are some files included in this package for examples, so we'll use these for the vignette.

file_list <- list.files(path = system.file("extdata", package = "raindancer"),
                        pattern = ".csv", full.names = TRUE, recursive = FALSE)
print(file_list)

The first four csv files are from 2010 and 2012, so lets start by using import_hobo_2008().

dat.1 <- import_hobo_2008(file_list[1])

Let's examine the components of the list returned by the import_hobo_2008() function. The first component returns information about the file, the logger, and the data.

dplyr::glimpse(dat.1$file_info)

The second component returns a three column data frame with information about the logger and sampling event.

dat.1$details

And the third component returns a data frame of raw data.

dplyr::glimpse(dat.1$data_raw)

In 2019 SEUG resource staff upgraded the loggers in hopes of preventing logger failure and maintaining the weather data set for the LTVMP. The new loggers had a different file structure that the older loggers, requiring a new function to import data into R. Files five though eight are examples from the new loggers. Let's use import_hobo() to bring data from one of these files into R and examine the information and data.

dat.6 <- import_hobo(file_list[6])
str(dat.6)

Summarize Data

Let's summarize these data up to daily values now that we have data read into R. The data are either event data from precipitation gauges or recorded at set intervals through out the day for temperature and relative humidity data.

Precipitation (Event) Data with raindance()

The precipitation loggers record an event every time the bucket inside the rain gauge fills with 0.254 mm of water and tips to trigger an event. The data recorded is simply an event (e.g., event 1, event 2, event 3, and so on). The raindance() function calculates hourly precipitation, provides the number of tips per hour, and provides an estimate of intensity with maximum tips per minute in a given hour. This function requires the data_raw component from import_hobo() or import_hobo_2008() functions and returns a data frame of hourly data.

raindance(dat.1$data_raw)

Temperature and Relative Humidity Data with sundance()

Temperature and relative humidity loggers record data at set intervals through out the day. The sundance() function summarizes these data to daily values, providing mean, minimum, maximum, and the number of measurements (n) for a given day. This function also returns the time that the minimum and maximum were recorded. This function requires the data_raw component from import_hobo() or import_hobo_2008() functions and returns a data frame of daily data.

sundance(dat.6$data_raw)

Processing HOBO files

The process_hobo() function was developed to summarize all the csv files in a directory or folder. It evaluates the elements of the data from the file_info component and uses raindance() or sundance() to summarize the data. This function requires an object returned form import_hobo() or import_hobo_2008() and adds the summarized data to the original list, returning a four component list. An effective way to use this function is with lapply().

lapply(file_list[2:4], function(this_file){
  dat <- import_hobo_2008(this_file) |> process_hobo()
  print(basename(this_file))
  print(data.class(dat)); print(names(dat))
  dat$data
})

Reporting Errors and Issues

The data from 2021 were mostly formatted the same as data from 2020, with a few exceptions. There is a likely that some future csv file will have anomolies that will require improvements to the import_hobo() function. Please submit any problems on the Issues page of this GitHub repository, https://github.com/scoyoc/raindancer/issues, or contact the author of the package if this happens.

Next Steps

This package was designed to work with the dataprocessR package. See the dataprocessR vignette for how to export these data to the SEUG LTVMP database.

scoyoc/rainDanceR documentation built on Jan. 10, 2023, 3:29 p.m.