In nathaneastwood/sparkplugs: Common Wrappers and Missing Functionality from 'sparklyr' and 'dplyr'

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

{flicker}

Overview

{flicker} is a collection of useful wrapper functions and extensions to the {dplyr} API which also work with Spark.

Installation

You can install:

• the development version from GitHub with

# install.packages("remotes")
remotes::install_github("nathaneastwood/flicker")

• the latest release from CRAN with

install.packages("flicker")

Usage

Grouped Operations

These functions offer the benefit over the scoped variants of being able to explicitly specify the parameters for each expression to evaluate.

library(flicker)
mtcars %>%
  summarise_groups(
    .groups = c("am", "cyl"),
    avgMpg = mean(mpg, na.rm = TRUE),
    avgDisp = mean(disp, na.rm = TRUE)
  )

Scoped Variant "when"

These functions are subtly different from the scoped _if() variants of {dplyr} functions in that they can evaluate any predicate. They are useful when used within a chain of commands.

previous_result <- 42
mtcars %>% filter_when(previous_result < 42, cyl == 4)
mtcars %>% filter_when(previous_result >= 42, cyl == 4)

But we can also perform these checks as if using the scoped variants of {dplyr} functions.

mtcars %>% filter_when("mpg" %in% colnames(.), cyl == 4)

Union Select

This function will union the records from multiple data sets returning only the requested columns.

a <- data.frame(col1 = 1:5, col2 = 6, col3 = rnorm(5))
b <- data.frame(col1 = 1:3, col2 = 4, col3 = rnorm(3))
c <- data.frame(col1 = c(0, 1, 1, 2, 3, 5, 8), col3 = rnorm(7))

union_select(.data = list(a, b, c), c("col1", "col3"))

Cross Joins

As of {dplyr} 1.0.0, cross joins have been available through the use of full_join(by = character()) but this is not a natural way to perform the operation in my opinion. {flicker} provides a way to perform cross joins for earlier versions of {dplyr}.

x <- data.frame(id = 1:2, val = rnorm(2))
y <- data.frame(run = 1:2, res = rnorm(2))
cross_join(x, y)

nathaneastwood/sparkplugs documentation built on Feb. 28, 2021, 4:57 p.m.