flog: flog: functional logging for R pipelines

In russHyde/flogr: An R Package for Functional Logging in Data Analysis Pipelines

Description

Runs a consecutive pipeline of functions (modifiers) on an input dataset and after each modifier is ran, computes some logging information related to having ran that step. The logging information is passed through the pipeline in a side-effect-free manner. The user must specify a list of modifier functions and a list of logging functions (either by setting both modifiers and loggers or by setting logsteps, which uses the LoggingStep class). The user either provides lists of modifier and logger functions, or provides a list of LoggingStep objects that encapsulates these functions. Returns a LoggingTuple (you can extract the dataset or logging data using get_dataset or get_logdata, respectively). Separate flog() functions can be piped together, by separate calls to flog() since the output format is the same as the input format (since LoggingTuple is a valid input); see the examples - that's where the examples are.

Usage

flog(.data, modifiers, loggers, logsteps)

Arguments

.data	A LoggingTuple or a raw dataset. If the input is a raw dataset, this will be converted to a LoggingTuple prior to being passed into the pipeline of functions. flog() applies each of the modifier functions, sequentially, to the 'dataset' entry of .data and the result of this pipeline is returned in the 'dataset' entry of LoggingTuple output by 'flog()'. After each modifier has been applied, the corresponding logger function is used to generate a summary of the differences to the dataset that were induced by the modifier function. The logging results are returned as the list 'logdata' that is present in the returned LoggingTuple.
modifiers	A list of functions that sequentially modify the input dataset. If a single function is provided it need not be in a list and will be repeated to match the length of 'loggers'. If provided, the user must also define the list 'loggers' and must not provide a 'logsteps' argument.
loggers	A list of functions that compare output to input at each stage of the dataset pipeline. If a single function is provided it need not be in a list and will be repeated to match the length of 'modifiers' (ie, if 6 modifiers are provided and only one logger, the logger will be used after each of the modifiers has ran). If provided, the user must also define the argument 'modifiers' and must not provide a 'logsteps' argument.
logsteps	A list (or a singleton) of LoggingStep objects. Each of these specifies a modifier function and a logger function that modify and log the changes in a dataset at each step of a flog() pipeline. Providing a 'logsteps' argument excludes the user from providing a pair of 'modifiers' or 'loggers' arguments to flog().

Value

A LoggingTuple, where the dataset entry is the result of running .data@dataset %>% modifiers1 %>% modifiers2 %>% ... %>% modifiersN and the logdata entry contains the logging information for each of the N processing steps (for the k'th step, this is given by comparing the input to the k'th modifier to the output from running the k'th modifier using the k'th logger function).

Examples

  .tuple <- LoggingTuple(dataset = c(2, 1, 1), logdata = list())
  logF <- function(post, pre){
    data.frame(before = length(pre), after = length(post))
    }

  flog(.tuple, modifiers = sum, loggers = logF)

  # `.tuple` is the first arg so that flog() steps can be piped together
  # Hence, the next code is equivalent to
  # .tuple %>% flog(unique, logF) %>% flog(sort, logF) %>% flog(sum, logF)
  # and
  # .tuple %>% flog(c(unique, sort, sum), c(logF, logF, logF))
  # since, if either modifiers (loggers) is a single entry, it is recycled
  # to match the number of entries in loggers (modifiers, resp.)

  flog(.tuple, c(unique, sort, sum), logF)

  # You can use a raw dataset (ie, without logdata) as input as follows:

  my_data <- c("I'm", "spaRtacus")
  # - Explicit set-up of LoggingTuple:
  #  flog(.data = LoggingTuple(dataset = my_data, logdata = list()),
  #       tolower, logF)
  # - Implicit set-up of `dataset` and `logdata` entries of in_list:
  #  flog(.data = LoggingTuple(my_data, list()), tolower, logF)
  #  flog(.data = LoggingTuple(my_data),         tolower, logF)
  # - Use bare dataset (and flog will convert it to a LoggingTuple)
  flog(my_data, modifiers = tolower, loggers = logF)

russHyde/flogr documentation built on May 29, 2019, 9:10 a.m.