In CfSOtago/GREENGridData: Processing NZ GREEN Grid project data to create a 'safe' version for data archiving and re-use

knitr::opts_chunk$set(echo = FALSE) # by default turn off code echo

# Set start time ----
startTime <- proc.time()

# Local parameters ----


# Local functions ----

\newpage

About

Report circulation:

• Public - this report is intended to accompany the data release.

License

Citation

If you wish to use any of the material from this report please cite as:

• Anderson, B. (r lubridate::year(today())) r params$title r paste0(params$subtitle, params$hhID) r ggrParams$pubLoc.

This work is (c) r lubridate::year(today()) the University of Southampton.

History

• Report history

Requirements:

This report uses the safe version of the grid spy 1 minute data which has been processed using https://github.com/CfSOtago/GREENGridData/tree/master/dataProcessing/gridSpy.

Support

\newpage

Introduction

This report provides summary analysis of one household as an example.

Load r params$hhID data

The data used to generate this report is:

• r gsFile
• r hhFile

hhDT <- data.table::as.data.table(readr::read_csv(hhFile)) # load hh data

keepCols <- c("linkID","nAdults", "nTeenagers13_18", "Location", "hasLongSurvey", "hasShortSurvey", "hasApplianceSummary")

# just keep the above columns for reporting
t <- hhDT[hhID == params$hhID, ..keepCols]

kableExtra::kable(t, 
             booktabs = TRUE,
             caption = paste0("Summary of household data for ", params$hhID)) %>%
  kable_styling()

Table \@ref(tab:summaryTable) shows key household attributes such as how many people live in this household.

# just keep the appliance columns for reporting
t <- hhDT[hhID == params$hhID, 1:17]

tt <- data.table(variable = names(t), value = transpose(t))

kableExtra::kable(tt, 
             booktabs = TRUE,
             caption = paste0("Summary of appliances recorded for ", params$hhID)) %>%
  kable_styling()

Table \@ref(tab:applianceTable) shows the appliances recorded for this household.

gsDT <- GREENGridData::getCleanGridSpyFile(gsFile) # load gs data

t <- summary(gsDT)

kableExtra::kable(t, caption = paste0("Summary of grid spy data for", params$hhID)) %>%
  kable_styling()

# create some useful derived date & time variables
gsDT <- gsDT[, month := lubridate::month(r_dateTime, label = TRUE)]
gsDT <- gsDT[, year := lubridate::year(r_dateTime)]
gsDT <- gsDT[, obsHour := lubridate::hour(r_dateTime)]
gsDT <- gsDT[, obsDate := lubridate::date(r_dateTime)]
gsDT <- gsDT[, obsTime := hms::as.hms(r_dateTime)]

Table \@ref(tab:loadGsData) shows a summary of the grid spy 1 minute power data.

Note that:

• the original dataTime (dateTime_orig) and TZ (TZ_orig) have been retained so that the user can check for parsing errors (see https://cfsotago.github.io/GREENGridData/gridSpy1mProcessingReport_v1.0.html#dateTimeChecks) if required;
• r_datetime is the correct dateTime of each observation in UTC and will have loaded as your local timezone. If you are conducting this analysis outside NZ then you will get strange results until you use lubridate to tell R to use tz = "Pacific/Auckland" with this variable;

Plot monthly power profiles

This section plots overall mean power (W) per minute per month for each circuit to show:

• patterns of missing data (no lines)
• patterns of consumption
• possible dateTime issues (where consumption patterns seem to be stangely shifted in time)
• possible PV installation

plotDT <- gsDT[, .(meanW = mean(powerW)), keyby = .(circuit, month, year, obsTime)
             ] # aggregate by circuit to preserve unique circuit labels in households 
# (e.g. rf_46) where names are re-used but with different ids. see ?fixCircuitLabels_rf_46

# set attributes for plot
vLineAlpha <- 0.4
vLineCol <- "#0072B2" # http://www.cookbook-r.com/Graphs/Colors_(ggplot2)/#a-colorblind-friendly-palette
timeBreaks <- c(hms::as.hms("04:00:00"), 
                hms::as.hms("08:00:00"),
                hms::as.hms("12:00:00"),
                hms::as.hms("16:00:00"),
                hms::as.hms("20:00:00"),
                hms::as.hms("24:00:00")
)

# set default caption
myCaption <- paste0("GREENGrid Grid Spy household electricity demand data (https://dx.doi.org/10.5255/UKDA-SN-853334)",
                        "\n", min(gsDT$r_dateTime), 
                        " to ", max(gsDT$r_dateTime),
                        "\nTime = Pacific/Auckland",
                        "\n (c) ", lubridate::year(now())," University of Otago")

myPlot <- ggplot2::ggplot(plotDT, aes(x = obsTime, y = meanW/1000, colour = circuit)) +
  geom_line() + 
  scale_colour_manual(values=ggrParams$cbPalette) + # use colour-blind friendly palette
  facet_grid(month  ~ year) + 
  theme(strip.text.y = element_text(angle = 0, vjust = 0.5, hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 0.5)) + 
  theme(legend.position = "bottom") + 
  labs(title = paste0("Montly mean power profiles by circuit for household: ", params$hhID),
       y = "Mean kW per minute", 
       caption = myCaption
       )

myPlot + 
  scale_x_time(breaks = timeBreaks) +
  geom_vline(xintercept = timeBreaks, alpha = vLineAlpha, colour = vLineCol)

ggplot2::ggsave(paste0(ggrParams$repoLoc,"/examples/outputs/", params$hhID, "_testHouseholdPower.png"))

Figure \@ref(fig:plotProfiles) shows the plot for this household (r params$hhID). Can you see anything interesting or unusual?

Runtime

t <- proc.time() - startTime

elapsed <- t[[3]]

Analysis completed in r round(elapsed,2) seconds ( r round(elapsed/60,2) minutes) using knitr in RStudio with r R.version.string running on r R.version$platform.

R environment

R packages used

• base R [@baseR]
• bookdown [@bookdown]
• GREENGridData [@GREENGridData] which depends on:
  • data.table [@data.table]
  • dplyr [@dplyr]
  • hms [@hms]
  • lubridate [@lubridate]
  • progress [@progress]
  • readr [@readr]
  • readxl [@readxl]
  • reshape2 [@reshape2]
• kableExtra [@kableExtra]
• knitr [@knitr]
• rmarkdown [@rmarkdown]

Session info

sessionInfo()

References

CfSOtago/GREENGridData documentation built on June 10, 2022, 8:03 p.m.