In aWhereAPI/aWhere-R-Charts: Make Charts using output from aWhere API

knitr::opts_chunk$set(echo = TRUE,
                      message = FALSE,
                      warning = FALSE,
                      results='hide')

working.dir <- ''

Note

This tutorial is intended for those users that have trial access to aWhere's API; such access is limited to being able to retrive 6 months worth of historical daily data. If you are currently have greater account privlidges, you are encouraged to reference the tutorial "3-Create_Charts_fullAccount.RMD" as it will cover functions/features of the R package that cannot be leveraged using a trial account

Introduction

This document describes how to generate aWhere weather data charts for a given location and timespan. Current/forecasted weather data is pulled from the aWhere API along with long-term normals for that same time span. Displaying line plots for variables of interest such as maximum temperature and P/PET allows us to observe weather trends and how current conditions compare to the past.

Weather variables for additional selected years of interest can also be added to these charts for further analysis and comparison between years. This tutorial adds 2016 and 2017 data onto charts for Accumulated Precipitation, P/PET, and effective P/PET.

Setup

Install / load R packages

Install required packages from aWhere and CRAN (the official repository of R code).

# install CRAN packages
list.of.packages.CRAN = c("tidyr", "dplyr", "ggplot2", "ggthemes", "curl", "zoo")

new.packages.CRAN = list.of.packages.CRAN[!(list.of.packages.CRAN %in% 
                                              installed.packages()[,"Package"])]
if(length(new.packages.CRAN)) install.packages(new.packages.CRAN)


# install aWhere code packages
list.of.packages.Github = c("aWhereAPI", "aWhereCharts")

new.packages.Github = list.of.packages.Github[!(list.of.packages.Github %in% 
                                                  installed.packages()[,"Package"])]
if(length(new.packages.Github) > 0) {
  for (x in 1:length(new.packages.Github)) {
    if (new.packages.Github[x] == 'aWhereAPI') {
      devtools::install_github("aWhereAPI/aWhere-R-Library")
    } else if (new.packages.Github[x] == 'aWhereCharts') {
      devtools::install_github("aWhereAPI/aWhere-R-Charts")
    }
  }
} 

\newpage

Load the R packages that contain functions used in this script.

# load packages 
library(tidyr)
library(dplyr)
library(ggplot2)
library(ggthemes)
library(curl)
library(zoo)
library(knitr)
library(aWhereAPI)
library(aWhereCharts)

Define paths, input files, and parameters

Define your working directory. This is the place on your computer where your input files are located for this tutorial, and it is also where the output files will be written. Instead of typing out the entire path to every file that we need (“absolute” paths), you can just use their filenames or locations within this working directory (“relative” paths) since we are setting this working directory as our starting point for the rest of the tutorial.

NOTE For this tutorial and ease of use for learning, all files will be written to the directory where this file is run from. An example of how one would set the working directory is below for reference

#working directory - where input files are located and outputs will be saved.
working.dir <- c('~/aWhere/') 

# set the working directory
setwd(working.dir)

With R, it is also necessary to create the directory structure you will write data files too beforehand. If one fails to do this, an error will be thrown when the data files are written

# create necessary output directories
dir.create(path = 'outputCSVs/',showWarnings = FALSE, recursive = TRUE)
dir.create(path = 'figures/',showWarnings = FALSE, recursive = TRUE)

Use the source function to load functions within .R files on your local machine. For this tutorial, the WriteJpeg and ClipValues functions are utilized within the "0-supporting_functions.R" file. There is a zero in the beginning of this filename so it appears at the top when files are sorted alphabetically.

# load external R functions in local file
source(paste0(working.dir, "0-supporting_functions.R"))

To pull data from the aWhere API, you’ll need a Consumer Key and Secret, like a username and password. Put these in a text file with the following format: line 1 is the Consumer Key, line 2 is the Consumer Secret, and line 3 is a blank line. Set the credentials.file variable equal to the name of this text file. Place your credentials file in your working directory.

# filename containing your aWhere credientials (key and secret),
credentials.file <- paste0(working.dir,'credentials.txt')

# load the aWhere API credentials file 
aWhereAPI::load_credentials(credentials.file) 

What location do you want to produce weather data charts for?

Specify the latitude and longitude, along with a name describing the place. This name will be used for the chart names and output file names.

# latitude, longitude, and name of location 
lat <- 8.45
lon <- 80.4
location.name <- paste("Ana") 

\newpage

What timespan to you want to acquire weather data for? Specify the years and dates here. Years is a vector with the starting year in position 1 and the ending year in position 2, for the long-term analysis.

The starting and ending days should have the format “YYYY-MM-DD” and should not exceed a period greater than 365 days. If you want the ending date to be n days from now (including some forecasted weather data), add or subtract n to the current date (which can be acquired using the Sys.Date() function). In the code below, when n = 7, the end date is seven days from now.

# starting and ending years and days for the time series. each is a vector with 
# the starting value in position 1 and the ending value in position 2. 
years <- c(2010, 2017)

#starting and ending days with format "YYYY-MM-DD"
day.start <- "2018-08-01"

#day.end <- "2018-03-08" # specific date
day.end <- as.character(Sys.Date() + 7) # today plus n
day.end <- as.character(Sys.Date() + 6) # today plus n

# combine the days into a single vector 
days <- c(day.start, day.end)

Some of the charts will require additional parameters. For instance, effective precipitation is calculated using a threshold (eP in the code below). Rolling average calculations require a window size or number of days to average over (roll.avg in the code below).

# effective precip amount for consistency across plots
eP <- 30    

# size of rolling average window for consistency across plots
roll.avg <- 30

Processing Steps

Some of the charts require rolling average calculations. When the roll.avg variable is set to 30, this means that the first data point of the rolling average calculation does not start until 30 days after the specified start date! To avoid having 30 days of no data in our rolling average charts, let's adjust the start date to be 30 days earlier for the rolling average calculation.

# adjust the start date to be "roll.avg" days earlier
day.start.extended <- as.character(as.Date(day.start) - roll.avg)

# substitute this earlier date into the "days" vector for pulling data
days[1] <- day.start.extended

The duration for our time range cannot exceed 365 days. Let’s make sure our day range is an acceptable length.

# check if time range is more than 365 days
duration <- as.Date(days[2]) - as.Date(days[1])

if(duration > 365) { 
  print ("Period too long")
}

Pull the aWhere weather data for the specified location and timespan.

The R function that we use to pull the data is generateaWhereDataset, which comes from the aWhereCharts R package. It’s good practice to specify which package each function is from, since multiple packages may have different functions with the same name. The syntax for doing this is to use two colons between the package and function names, like this: package::function().

# pull the datasets for the specified location and time period   
weather.df <- aWhereCharts::generateaWhereDataset(lat = lat, lon = lon, 
                                                  day_start = days[1], 
                                                  day_end = days[2], 
                                                  year_start = years[1], 
                                                  year_end = years[2])

Let’s take a look at the weather data using the head function from the utils R package.

# select the first five columns of the data frame using [,1:5]
# and show the first 10 rows using the "n" argument
utils::head(weather.df[,1:5], n = 10)

The variables of interest include accumulated precipitation, P/PET, PET and precipitation (to calculate effective precipitation). They are combined with the rest of the weather data and written to a .csv file for later analysis.

Let's take a look at the last few columns of the weather data frame. We will see the additional selected year data that was obtained in the loop above, and what the column names look like.

# reorder the columns in the data frame
weather.df <- weather.df %>% 
              dplyr::select(day, date, latitude, longitude, everything())

# look at the last few columns of the weather data frame. 
utils::head(weather.df[,(ncol(weather.df)-2):(ncol(weather.df))], n = 6)

Write weather data to .csv file

Write the weather data to a .csv file for further analysis. Use the latitude, longitude, date range, and year range in the filename so future-you will know exactly what’s inside this file.

# write forecast to .csv file 
utils::write.csv(weather.df, 
                 file = paste0('outputCSVs/',
                              paste(location.name, 
                              paste(days, collapse="_"),
                              paste(years, collapse="_"),
                              ".csv", sep="_")))

# since the rolling averages have already been calculated and the 
# weather data extending out to an earlier start date (30 days in this case)
# has been saved to file, let's filter our weather dataframe down to span
# the original date range for chart creation. 
weather.df.extended <- weather.df

weather.df <- weather.df %>% 
  dplyr::filter(as.Date(date) >= day.start)

Create charts

Within the aWhereCharts package, there are two functions used here to create time series charts.

(1) generateaWhereChart - makes line plots comparing the current and long-term normals for each variable.

(2) generateaWhereStdDevChart - plots aWhere weather data with one standard deviation's shading above and below the long-term normalline. This function is meant to help users understand whether the current weather conditions are significantly outside the norm.

Plots with standard deviation are named with "1" at the end of the variable, and those without standard deviation are named with "2".

Create chart: maximum temperature with standard deviation

Contruct the title for this chart using the variable name, location name, and the latitude/longitude.

# create a variable containing the lat and lon, for plot titles
lat.lon <- paste0("(", lat, ", ", lon, ")") 

# construct title
max.temp.1.title <- paste0(location.name, "_Maximum Temp w StdDev")

Generate the plot using the generateaWhereStdDevChart function from the aWhereCharts package.

# generate the plot 
max.temp.1 <- aWhereCharts::generateaWhereStdDevChart(data = weather.df, 
                                              variable = "maxTemp", 
                                              title = paste0(max.temp.1.title,
                                                              lat.lon))

To display the plot, use the plot object name.

# display the plot
max.temp.1

In the following sections, these same steps are performed for a series of other variables in the aWhere weather data set.

Minimum temperature with standard deviation

min.temp.1.title <- paste0(location.name, "_Minimum Temp w StdDev")

min.temp.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                              "minTemp", 
                                              title = paste0(min.temp.1.title,
                                                              lat.lon))

Potential evapotranspiration (PET) with standard deviation

pet.1.title <- paste0(location.name, "_PET w StdDev")

pet.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                                 "pet", 
                                                 title = paste0(pet.1.title,
                                                                lat.lon))

Daily precipitation with standard deviation

precip.1.title <- paste0(location.name, "_Daily Precipitation w StdDev")

precip.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                              "precipitation",  
                                              title = paste0(precip.1.title,
                                                              lat.lon))

Daily precipitation without standard deviation

precip.2.title <- paste0(location.name, "_Daily Precipitation")

precip.2 <- aWhereCharts::generateaWhereChart(weather.df,
                                              "precipitation",    
                                              title = paste0(precip.2.title,
                                                             lat.lon))

Accumulated Precipitation with StdDev but no Effective Precipitation

no.eprecip.1.title <- paste0(location.name, 
                             "_Accumulated Precipitation w StdDev")

no.eprecip.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                            "accumulatedPrecipitation",
                                            title = paste0(no.eprecip.1.title,
                                                            lat.lon))

Precipitation and Effective Precipitation, Accumulated

eprecip.1.title <- paste0(location.name, "_Precipitation and",
                          " Effective Precipitation, Accumulated w Std Dev")

eprecip.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                              "accumulatedPrecipitation",
                                              title = paste0(eprecip.1.title,
                                                            lat.lon),
                                              e_precip = TRUE, 
                                              e_threshold = eP)

Accumulated Precipitation

acc.precip.2.title <- paste0(location.name, "_Accumulated Precipitation")

acc.precip.2 <- aWhereCharts::generateaWhereChart(weather.df, 
                                            "accumulatedPrecipitation", 
                                            title = paste0(acc.precip.2.title,
                                                            lat.lon))

Accumulated PET

acc.pet.1.title <- paste0(location.name, "_Accumulated PET w StdDev")

acc.pet.1 <- aWhereCharts::generateaWhereStdDevChart(weather.df, 
                                                 "accumulatedPet",
                                                 title = paste0(acc.pet.1.title,
                                                               lat.lon))

P/PET

# ppet rarely is interpretable on a daily chart 
ppet.2.title <- paste0(location.name,"_PPET ")

ppet.2 <- aWhereCharts::generateaWhereChart(weather.df, 
                                             "ppet", 
                                             title = paste0(ppet.2.title,
                                                            lat.lon))

30-day rolling average eP/PET and P/PET

# no eprecip/PET shows up if all rainfall events are less than the e_threshold
rolling.avg.ppet.2.title <- paste0(location.name,
                                   "_30 day rolling avg eP PET and P PET")

# use the "extended" weather data, since it contains dates 30 days prior
# to the original start date and will allow for rolling average values to
# be calculated from the original start date. 
rolling.avg.ppet.2 <- aWhereCharts::generateaWhereChart(weather.df.extended, 
                                      "rollingavgppet",
                                      title = paste0(rolling.avg.ppet.2.title,
                                                     lat.lon),
                                      e_precip = TRUE, 
                                      e_threshold = eP, 
                                      rolling_window = roll.avg) + 
                        xlim(as.Date(day.start), as.Date(day.end))

\newpage

Mulitplot

Select any of the above charts to display together in a multiplot using their variable names:

• max.temp.1
• min.temp.1
• pet.1
• precip.1
• precip.2
• no.eprecip.1
• eprecip.1
• acc.precip.2
• acc.precip.2.addyears
• acc.pet.1
• ppet.2
• rolling.avg.ppet.2
• rolling.avg.eppet.2.addyears

# set the graphics device parameters to write a .JPEG
jpeg(paste0('figures/',location.name,"_4chart.jpeg"), 
     width = 12, height = 6, 
     units = 'in', res = 500)

# generate the multiplot & write to JPEG
aWhereCharts::generateMultiplot(acc.precip.2, 
                                rolling.avg.ppet.2, 
                                max.temp.1, 
                                pet.1, 
                                cols = 2, fontsize = 10, 
                                title = paste0("Current vs LTN at ", 
                                               location.name," (", 
                                               lat, ", ", lon, ")", 
                                               "   eP = ",eP,"mm"))
# close the current plot object
invisible(dev.off())

Display the multiplot. In each of the four charts, the vertical dashed line marks the current date, beyond which the weather data is forecasted.

knitr::include_graphics(paste0('figures/',location.name,"_4chart.jpeg"))

\newpage

Write charts to file

# Maximum temperature
#max.temp.1 

# write the plot to file using the WriteJpeg function, an external R function
# in the "supporting_functions.R" file.
WriteJpeg(plt = max.temp.1, plt.title = paste0('figures/',max.temp.1.title))


# Minimum temperature with standard deviation
#min.temp.1 
WriteJpeg(plt = min.temp.1, plt.title = paste0('figures/',min.temp.1.title))


# Potential evapotranspiration (PET) with standard deviation 
#pet.1 
WriteJpeg(plt = pet.1, plt.title = paste0('figures/',pet.1.title))


# Daily precipitation with standard deviation  
#precip.1 
WriteJpeg(plt = precip.1 , plt.title = paste0('figures/',precip.1.title))


# Daily precipitation without standard deviation  
#precip.2 
WriteJpeg(plt = precip.2, plt.title = paste0('figures/',precip.2.title))


# Accumulated Precipitation with StdDev but no Effective Precipitation
#no.eprecip.1
WriteJpeg(plt = no.eprecip.1, plt.title = paste0('figures/',no.eprecip.1.title))


# Precipitation and Effective Precipitation, Accumulated 
#eprecip.1
WriteJpeg(plt = eprecip.1, plt.title = paste0('figures/',eprecip.1.title))


# Accumulated Precipitation 
#acc.precip.2
WriteJpeg(plt = acc.precip.2, plt.title = paste0('figures/',acc.precip.2.title))

# Accumulated PET 
#acc.pet.1
WriteJpeg(plt = acc.pet.1, plt.title = paste0('figures/',acc.pet.1.title))


# P/PET 
#ppet.2
WriteJpeg(plt = ppet.2, plt.title = paste0('figures/',ppet.2.title))


# 30-day rolling average eP/PET and P/PET 
#rolling.avg.ppet.2
WriteJpeg(plt = rolling.avg.ppet.2, 
          plt.title = paste0('figures/',rolling.avg.ppet.2.title))

aWhereAPI/aWhere-R-Charts documentation built on Dec. 30, 2021, 12:58 p.m.