Contrast Patterns

In nuggets: Fast and Extensible Pattern Discovery in Tabular Data

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

options(tibble.width = Inf)

Introduction

Contrast patterns identify conditions under which numeric variables show statistically significant differences. In nuggets, this family is represented by three related functions:

• dig_baseline_contrasts() for testing whether a variable differs from a chosen baseline value under a condition,
• dig_complement_contrasts() for comparing rows satisfying a condition with the remaining rows,
• dig_paired_baseline_contrasts() for comparing two paired variables inside a condition.

These pattern families answer different questions:

• Is a variable unusually high or low in some subgroup?
• Does a subgroup differ from the rest of the data?
• Do two paired measurements differ inside a subgroup?

Before going further, load the packages used in this vignette:

library(nuggets)
library(dplyr)     # for data manipulation

For the overall package workflow, see vignette("nuggets").

A Small Working Dataset

To demonstrate all three contrast types, we prepare a version of iris that contains:

• logical columns that can define conditions,
• numeric variables that can be compared by the contrast functions,
• derived variables whose interpretation is easy to explain.

iris_contrasts <- iris |>
    mutate(long_sepal = Sepal.Length >= median(Sepal.Length),
           wide_petal = Petal.Width >= median(Petal.Width),
           length_gap = Sepal.Length - Petal.Length,
           width_gap = Sepal.Width - Petal.Width,
           sepal_ratio = Sepal.Length / Sepal.Width,
           petal_ratio = Petal.Length / Petal.Width) |>
    partition(Species)

head(iris_contrasts, n = 3)

The Species factor is expanded into dummy predicates, while the logical helper columns remain available as additional condition predicates. The numeric columns are then used as the variables being tested.

For more information on creating predicate columns, see vignette("data-preparation").

Selecting Conditions and Variables

The contrast functions use slightly different argument names, but they all rely on the same idea:

• condition selects columns from which conditions are generated,
• vars selects numeric variables for one-sample or two-sample contrasts,
• xvars and yvars select paired numeric variables.

These arguments accept tidyselect expressions, so you can target specific sets of predicates and variables without manually listing every column.

Baseline Contrasts

Baseline contrasts search for conditions under which a numeric variable differs from a chosen baseline value h0.

The basic scheme is:

var != h0 | condition

Here we test whether two derived gap variables differ from zero inside the discovered subgroups. With method = "t", the underlying test is stats::t.test() (one-sample, testing whether the mean equals h0):

baseline_result <- dig_baseline_contrasts(iris_contrasts,
                                          condition = where(is.logical),
                                          vars = c(length_gap, width_gap),
                                          min_length = 1,
                                          max_length = 2,
                                          min_support = 0.2,
                                          method = "t",
                                          max_p_value = 0.01)

head(baseline_result, n = 6)

This result tells us under which conditions the mean gap is significantly different from zero.

• condition - the generated condition, support - relative frequency of the condition,
• var - the tested variable,
• estimate - the estimated mean difference from the baseline,
• statistic - the test statistic (determined by method argument),
• df - degrees of freedom for the test,
• p_value - significance of the test,
• n - number of rows in the corresponding sub-data,
• conf_lo, conf_hi - confidence-interval bounds,
• stderr - standard error of the estimate,
• condition_length - number of predicates in the condition,
• alternative, method, comment - additional information about the test.

Non-parametric Baseline Contrasts

If you prefer a rank-based one-sample test, use method = "wilcox". This applies stats::wilcox.test() (Wilcoxon signed-rank test), which tests whether the pseudo-median equals h0:

baseline_wilcox <- dig_baseline_contrasts(iris_contrasts,
                                          condition = starts_with("Species"),
                                          vars = length_gap,
                                          min_length = 1,
                                          max_length = 1,
                                          min_support = 0.2,
                                          method = "wilcox",
                                          max_p_value = 0.01)

baseline_wilcox

This is useful when you want a method that is less sensitive to departures from normality.

Complement Contrasts

Complement contrasts compare a subgroup with the rest of the dataset. Their scheme is:

(var | condition) != (var | not condition)

This is often the most natural contrast pattern when you want to know whether a condition identifies an unusual subgroup. With method = "t", the underlying test is stats::t.test() (two-sample Welch t-test, comparing the means of the condition subgroup and its complement):

complement_result <- dig_complement_contrasts(iris_contrasts,
                                              condition = where(is.logical),
                                              vars = c(Sepal.Length, Petal.Length, petal_ratio),
                                              min_length = 1,
                                              max_length = 2,
                                              min_support = 0.2,
                                              method = "t",
                                              max_p_value = 0.01)

head(complement_result, n = 6)

The output contains separate estimates for the subgroup and its complement:

• estimate_x average (or median) value of selected variable for rows satisfying the condition,
• estimate_y average (or median) value of selected variable for rows not satisfying the condition,
• n_x and n_y for the corresponding sample sizes.

Comparing Variability Instead of Location

dig_complement_contrasts() also supports method = "var" for testing whether the variability in one group differs from the variability in its complement. This uses stats::var.test() (F-test of equality of variances):

complement_var <- dig_complement_contrasts(iris_contrasts,
                                           condition = starts_with("Species"),
                                           vars = Petal.Length,
                                           min_length = 1,
                                           max_length = 1,
                                           min_support = 0.2,
                                           method = "var",
                                           max_p_value = 0.01)

complement_var

This is helpful when a subgroup is not mainly distinguished by a higher or lower mean, but by being more or less variable.

Paired Baseline Contrasts

Paired baseline contrasts compare two numeric variables observed on the same rows under generated conditions.

The scheme is:

(xvar - yvar) != 0 | condition

This is appropriate for paired measurements such as "before vs after", "left vs right", or two alternative measurements recorded for the same case. With method = "t", the underlying test is stats::t.test() (paired t-test, testing whether the mean difference equals zero):

paired_result <- dig_paired_baseline_contrasts(iris_contrasts,
                                               condition = where(is.logical),
                                               xvars = c(Sepal.Length, Sepal.Width),
                                               yvars = c(Petal.Length, Petal.Width),
                                               min_length = 1,
                                               max_length = 1,
                                               min_support = 0.2,
                                               method = "t",
                                               max_p_value = 0.01)

head(paired_result, n = 6)

The result reports the condition, the selected variable pair (xvar, yvar), the estimated difference, test statistic, p-value, and sample size.

Non-parametric Paired Contrasts

For a paired rank-based alternative, use the Wilcoxon signed-rank test by setting method = "wilcox". This calls stats::wilcox.test() with paired = TRUE, testing whether the pseudo-median of the pairwise differences equals zero:

paired_wilcox <- dig_paired_baseline_contrasts(iris_contrasts,
                                               condition = starts_with("Species"),
                                               xvars = Sepal.Length,
                                               yvars = Petal.Length,
                                               min_length = 1,
                                               max_length = 1,
                                               min_support = 0.2,
                                               method = "wilcox",
                                               max_p_value = 0.01)

paired_wilcox

Controlling the Search

All three contrast functions support the usual search controls:

• min_length, max_length for condition complexity,
• min_support, max_support for subgroup size,
• max_results to stop long searches early,
• max_p_value to keep only statistically significant results.

Which Contrast Family Should You Use?

The three contrast types complement each other:

• use baseline contrasts when you have a meaningful reference value,
• use complement contrasts when you want to compare a subgroup with the rest of the data,
• use paired baseline contrasts when two measurements belong to the same observational units.

In practice, it is often useful to start with complement contrasts to locate interesting subgroups, then refine the analysis with baseline or paired contrasts depending on the scientific question.

Notes on Interpretation

When reading discovered contrast patterns, keep the following points in mind:

• a large absolute estimate indicates a stronger effect,
• p_value reflects statistical evidence for the chosen alternative, but should be interpreted with caution due to multiple comparisons (see below),
• support and n describe how much data contributed to the pattern,
• longer conditions describe more specific subgroups, but usually with smaller support.

Multiple Comparisons

A typical contrast-pattern search tests many condition–variable combinations simultaneously. When hundreds of tests are run at level 0.05, several spurious discoveries are expected by chance alone, even if no true effect exists.

The patterns returned by the dig_*_contrasts() functions are therefore best understood as generated hypotheses - promising associations that deserve further scrutiny - rather than as confirmed findings. This is known as the problem of simultaneous statistical inference or multiple comparisons.

A standard remedy is to adjust the p-values to control either the family-wise error rate (FWER) or the false discovery rate (FDR):

• FWER-controlling methods (e.g. Bonferroni, Holm) ensure that the probability of making any false discovery across all tests stays below a chosen threshold. They are conservative when the number of tests is large.
• FDR-controlling methods (e.g. Benjamini–Hochberg, abbreviated BH) allow a small proportion of discoveries to be false positives. This is less stringent than FWER control and retains more patterns in exploratory analyses.

R's built-in p.adjust() function supports both families. The example below applies Holm correction (FWER) and Benjamini–Hochberg correction (FDR) to the result of a complement-contrast search:

complement_result$p_holm <- p.adjust(complement_result$p_value, method = "holm")
complement_result$p_bh   <- p.adjust(complement_result$p_value, method = "BH")

complement_result[, c("condition", "var", "p_value", "p_holm", "p_bh")]

After adjustment, you can filter by the corrected p-values:

complement_result[complement_result$p_bh < 0.05, ]

In a large exploratory search you may prefer the FDR approach (BH) because it keeps more patterns visible while still limiting the expected fraction of false discoveries. Use FWER control (Holm) when you need stronger guarantees.

Related Tools

The contrast functions focus on built-in statistical tests. If you need custom statistics under generated conditions, dig() and dig_grid() provide the general framework; see vignette("custom-patterns").

Conditional correlations are another related pattern family for subgroup-based analysis of numeric variables; see vignette("conditional-correlations").

For interactive inspection of discovered patterns, you can use:

explore(complement_result, iris_contrasts)

Summary

This vignette introduced the main contrast-pattern workflows in nuggets:

1. Baseline contrasts test whether a variable differs from a reference value under a condition.
2. Complement contrasts compare a subgroup with the remaining data.
3. Paired baseline contrasts compare two paired variables within a subgroup.
4. Search controls such as support, condition length, and p-value thresholds help keep the result focused and interpretable.
5. Tidyselect-based column selection makes it easy to describe both the condition predicates and the tested variables.

For related material, see:

• vignette("data-preparation") for preparing predicate columns,
• vignette("conditional-correlations") for subgroup-based correlation analysis,
• vignette("custom-patterns") for custom statistical pattern searches,
• vignette("nuggets") for the package overview.

nuggets documentation built on Aug. 20, 2026, 5:07 p.m.