Home

/

GitHub

/

In rebelrebel04/xplor: Data Exploration Utilities

knitr::opts_chunk$set(
  echo = TRUE,
  collapse = TRUE,
  comment = "##",
  fig.path = "README-"
)
library(dplyr)
library(xplor)

xplor

A package of lightweight utility functions for assisting with interactive data exploration, particularly geared toward understanding record layouts of relational data (e.g., the grain of a table, mappings between primary and affiliate dimensions, extent of normalization/denormalization, etc.). NSE and SE versions of core functions are provided to facilitate interactive and programming use. Functions are designed to work well with dplyr.

Installation

Install xplor from github with:

# install.packages("devtools")
devtools::install_github("rebelrebel04/xplor")

Core Functions

cpf

Create cumulative percentile/frequency tables with cpf. The most common use case is interactive crosstabs (though note that the output frequency table is always in long format).

data(mtcars)
cpf(mtcars, gear)

You can also request a lightly-formatted kable when running cpf within an Rmarkdown file:

cpf(mtcars, gear, kable = TRUE)

A nice feature is that you can pass cpf computed variables using bare variable names; optionally provide descriptive names for the table columns:

cpf(mtcars, carb >= 4)

cpf(mtcars, has_cyl = !is.na(cyl))

Optionally specify a weighting variable for frequencies:

cpf(mtcars, carb, wt = "cyl")

A chi-square test can be requested for frequency tables that have two or more dimensions:

cpf(mtcars, cyl > 4, hp > 123, chi_square = TRUE)

cpf returns a dataframe containing the frequency table, which can be useful for downstream processing or plotting:

cpf(mtcars, gear, cyl, margin = FALSE) %>% 
  ggplot2::ggplot(ggplot2::aes(x = gear, y = cyl, fill = pct)) +
  ggplot2::geom_tile() +
  ggplot2::scale_fill_gradient(label = scales::percent)

The SE version of cpf_ is provided in the event you have variable names built up through some other means:

vars <- names(mtcars)[grepl("^c", names(mtcars))]
cpf_(mtcars, .dots = vars)

has

has is a wrapper around the common pattern of using cpf to cross-tab missing value (NA) counts across multiple dimensions:

mtcars.na <- mtcars
set.seed(1234)
mtcars.na[runif(10, 1, nrow(mtcars.na)), "gear"] <- NA
mtcars.na[runif(5, 1, nrow(mtcars.na)), "cyl"] <- NA

has(mtcars.na, gear, cyl, carb, kable = TRUE)
# This is equivalent to:
# cpf(mtcars.na, has_gear = is.na(gear), has_cyl = is.na(cyl), has_carb = is.na(carb), kable = TRUE)

dup

Check for duplicate keys in a dataframe with dup:

dup(mtcars, gear, cyl)

dup divides cases into those with unique keys (which appear only once in the dataset) and those with non-unique keys. This varies from the duplicated function in base R, which only counts cases as duplicates after the first (or last) occurrence. This is why dup reports the number of unique vs. non-unique cases by the specified keys.

dup invisibly returns the original dataframe filtered to rows with non-unique keys, with a variable .n attached indicating the total number of cases with each key combination:

print(dup(mtcars, disp))

A weight can be supplied to generate weighted duplication rates. This can be useful to determine if duplication is limited to low-frequency cases when weighting by a separate frequency variable:

dup(mtcars, disp, wt = "mpg")

ss

Produce a quick table of summary statistics with ss. The function accepts a named list of functions (specified as formulas) via the funs argument, with the default list covering the basics. If no variables are specified the table will summarize all numeric columns in the dataset. If plot = TRUE a facet-wrapped plot of histograms will be produced as a side-effect.

ss(mtcars, plot = TRUE, kable = TRUE)

A dplyr-grouped dataframe can be passed to produce a dimensional summary table:

library(dplyr)
mtcars %>% 
  group_by(cyl) %>% 
  ss(disp, mpg, kable = TRUE)

Pass a custom list of summary functions as formulas on x:

mtcars %>% 
  ss(
    funs = list(
      "variance" = ~var(x, na.rm = TRUE), 
      "skewness" = ~moments::skewness(x, na.rm = TRUE), 
      "kurtosis" = ~moments::kurtosis(x, na.rm = TRUE)
    ), 
    kable = TRUE
  )

mapping

For dimensions x and y in a dataset, there may be only one unique value of y associated with each unique value of x, or there may be multiple unique values of y associated with each unique value of x. mapping summarizes the association between x and y in a dataset, specified as a formula x ~ y:

mapping(cyl ~ gear, mtcars)

In this example, there is one value of the "x" variable cyl that has a single unique "y" (gear) value associated with it, and two values of cyl that each have two unique gear values associated with them.

findGrain

Often with a new dataset, you may not know the grain of the table (i.e., the combination of dimensions at which cases are unique). This function searches through candidate dimensions and ranks the top combinations in descending degree of de-duplication (aka normalization). In theory, the true grain of a table would be defined by the smallest combination of dimensions that yields 0 duplicates.

data(sleep)
findGrain(sleep, group, ID)

vjoin

Verbose join wrapper around dplyr join functions to provide diagnostic output. Joined result is returned invisibly.

a <- mtcars[1:10, c("hp", "mpg", "disp")]
b <- mtcars[11:15, c("hp", "cyl")]
vjoin(a, b, "hp")

Match rates can be weighted:

vjoin(a, b, "hp", wt = "disp")

%&%

Why does paste(NA) actually print "NA"?

paste(NA)

I don't know, but this is one of those places where base R is kind of dumb. Besides just making concatenation easier to read, % % ignores NAs (silently replacing them with ""), which is probably what you wanted R to do in the first place 99.9% of the time:

first_names <- c("Abe", "George", "Thomas")
middle_names <- c("J.", NA, "J.")
last_names <- c("Lincoln", "Washington", "Jefferson")
first_names % % middle_names % % last_names