Sequence Data Validation and Preparation
In gp3sequences: Transparent Analysis of Ordered Categorical Sequences

knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(gp3sequences)

Why preparation is explicit

Ordered categorical data can contain missing states, duplicated positions, unsorted rows, consecutive repeats, zero durations, unknown states, and inconsistent metadata. Silent repair can change the analytical object. gp3sequences therefore separates non-modifying audit and validation from policy-driven preparation.

A deliberately problematic synthetic input

The example includes an unsorted sequence, a duplicated position, a missing state, a consecutive repeat, a zero duration, an unexpected state, and an unused factor level. Participant and group metadata remain constant within each sequence.

problem_data <- data.frame(
  sequence_id = c("s2", "s1", "s1", "s1", "s1", "s2", "s2", "s2", "s2"),
  sequence_order = c(2, 2, 1, 2, 3, 1, 2, 3, 4),
  state = factor(
    c("search", "search", "home", "search", NA,
      "home", "home", "product", "other"),
    levels = c("home", "search", "product", "checkout", "other", "unused")
  ),
  duration = c(120, 100, 90, 110, 80, 100, 0, 150, 130),
  participant_id = c("p2", "p1", "p1", "p1", "p1", "p2", "p2", "p2", "p2"),
  group = c("interface_b", "interface_a", "interface_a", "interface_a",
            "interface_a", "interface_b", "interface_b", "interface_b",
            "interface_b"),
  stringsAsFactors = FALSE
)

expected_states <- c("home", "search", "product", "checkout")
problem_data

Audit without modification

audit_sequence_data() reports one row per issue using stable issue codes and severity values. It does not repair the data.

audit <- audit_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

audit
as.data.frame(table(audit$severity), stringsAsFactors = FALSE)
as.data.frame(table(audit$issue_code), stringsAsFactors = FALSE)

Compact validation contract

A review-level issue does not automatically invalidate an input. Error-level issues must be resolved through source correction or an explicit supported policy.

validation <- validate_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

validation[c("valid", "status", "n_errors", "n_reviews", "n_info")]
validation$mapping

Apply explicit preparation policies

This example deliberately chooses to:

drop rows with missing states;
retain the first row at duplicated positions;
collapse consecutive repeated states;
drop zero-duration rows;
drop states absent from the declared state set;
drop unused factor levels.

These are analytical choices, not universal defaults.

prepared <- prepare_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states,
  missing_state_policy = "drop",
  duplicate_position_policy = "first",
  repeated_state_policy = "collapse",
  zero_duration_policy = "drop",
  unknown_state_policy = "drop",
  unused_state_levels = "drop"
)

prepared$status
prepared$decisions
prepared$data
prepared$audit

Revalidate the canonical result

The prepared table uses stable canonical columns while preserving unmapped metadata and original-row provenance.

revalidation <- validate_sequence_data(
  prepared$data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

revalidation[c("valid", "status", "n_errors", "n_reviews", "n_info")]
prepared$mapping
prepared$state_levels

Errors that require source correction

Some conditions are intentionally not repaired automatically. Examples include missing sequence identifiers, missing or non-finite order values, negative or non-finite durations, absent mapped columns, duplicated column names, invalid column types, and metadata that varies within a sequence. These conditions require correction or an explicit redefinition of the sequence unit.

Reporting recommendations

A reproducible report should record the input mapping, expected states, every preparation policy, the audit table, the decision log, original and prepared row counts, and the final state levels. These records describe data handling; they do not validate a substantive interpretation of the resulting sequence patterns.