Data requirements and standardization
In PDRobust: Robust Longitudinal Effects Under Truncation by Death

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  eval = TRUE,
  echo = TRUE,
  include = TRUE,
  fig.show = "asis",
  fig.keep = "high",
  warning = FALSE,
  message = FALSE
)

Introduction

The complete workflow is illustrated as follows. This article focuses on the package's data requirements and the first three steps, including functions Mapping(), DataCheck() and DataStandard().

data -> Mapping() -> DataCheck() -> DataStandard()
     -> prediction / diagnostic / analysis functions

library(PDRobust)
data("ImperfectConSample", package = "PDRobust")
data("BiSample", package = "PDRobust")

Two built-in datasets are included with the package. The datasets are stored in the data/ directory. Their generation script is available in data-raw/ in the source repository; development scripts are excluded from the CRAN archive.

The first dataset, BiSample, is a standardized longitudinal dataset that satisfies the package’s data requirements. It contains no nonstructural missing values; outcome values are missing only when they are structurally unobservable due to any kind of truncation. Each row represents one subject at a specific time. The dataset includes a subject identifier (id), assessment time (time), survival status (S), treatment assignment (A), a binary outcome (Y), and six subject-level covariates (X1–X6) .

data("BiSample", package = "PDRobust")
head(BiSample)

The second one, ImperfectConSample, is designed to resemble longitudinal data collected in a clinical trial with repeated follow-up assessments. Each row represents a patient observation at a scheduled visit. The dataset includes a patient identifier (patient_id), visit time (visit_month), survival status (alive_status), treatment assignment (treatment), a continuous clinical outcome, and six patient-level covariates (X1–X6) representing relevant clinical and demographic information.

head(ImperfectConSample)

Mapping raw columns and times

Mapping() is the sole source of truth for structural columns, analysis window on the raw time scale, the covariated used to estimate treatment effects, the variables of interest, and the outcome type.

The arguments id, time, treatment, survival, outcome specify the corresponding column names in the input dataset.

The arguments baseline_time and cutoff_time define the beginning and end of the analysis window, respectively. Each must be specified as a single finite numeric value on the raw time scale, with baseline_time <= cutoff_time. Equal endpoints define a single-time analysis. For example, when observed time points are (0, 1, ..., 4), the baseline may be set to 0 or 1, whereas the cutoff may be set to a later time point, such as 4. For ImperfectConSample, the observed time points are (0, 6, 12); therefore, the analysis window is defined using baseline_time = 0, cutoff_time = 12.

The argument covariates and interest_vars are character vectors containing the column names of the relevant covariates. Variables specified in interest_vars must also be included in covariates . The argument y_type specifies the outcome type. It should be set to B for a binary outcome and C for a continuous outcome.

map <- Mapping(
  id = "patient_id",
  time = "visit_month",
  treatment = "treatment",
  survival = "alive_status",
  outcome = "clinical_outcome",
  baseline_time = 0,
  cutoff_time = 12,
  covariates = paste0("X", 1:6),
  interest_vars = c("X1", "X2"),
  y_type = "C" # "B"
)

Users can inspect the mapping details using print() and the attributes() function may additionally be used to inspect object-level metadata, such as its class.

print(map)

attributes(map)

Validation without modification

DataCheck() evaluates whether the input dataset satisfies the structural and analytical requirements of PDRobust without altering the data. It identifies potential issues, reports their severity and recommended handling, and determines whether the dataset is ready for analysis, can be standardized using DataStandard(), or requires manual resolution.

Each check includes an action-oriented message. When strict = FALSE, which is the default, DataCheck() returns a validation report without modifying the input dataset. When strict = TRUE, the function raises an error if one or more failed checks are marked as analysis-blocking. Missing required columns or empty data cause an early return containing only the checks that can be performed at that stage.

check <- DataCheck(ImperfectConSample, map, strict = FALSE)

attributes(check)

DataCheck() returns an object of class pd_data_check. The object contains the following components:

| Component | Description | |----|----| | valid | Indicates whether all checks with severity "error" have passed. Informational messages and warnings do not by themselves make the report invalid. | | ready_for_analysis | TRUE when no failed check is marked as analysis-blocking. Raw data still need DataStandard() to attach the mapping and readiness metadata required by the HTE and diagnostic interfaces. | | manual_resolution_required | Indicates whether at least one failed check requires manual review or correction. Such issues are not automatically resolved by DataStandard(). | | can_standardize | TRUE when no failed check requires manual resolution. Deletion may still require drop = TRUE, leave no observations, or remove a treatment group, so this flag does not guarantee success or final analysis readiness. | | checks | A data frame containing the itemized validation results, including the status, severity, diagnostic summary, analysis implications, and recommended handling for each check. | | settings | Records the settings used during validation. The current implementation stores the validated mapping object in settings$mapping. | | diagnostics | Contains detailed supporting information, such as affected row numbers, subject identifiers, missingness summaries, treatment-group counts, and problematic covariates. |

check$valid
check$ready_for_analysis
check$manual_resolution_required
check$can_standardize

The following are diagnostics of check for ImperfectConSample. Users can find detailed descriptions of all validation items in the article Details-for-DataCheck.

head(check$diagnostics)

Standardization and audit attributes

DataStandard() returns a prepared data for later analysis.

pd_data <- DataStandard(ImperfectConSample, map, drop = TRUE)
class(pd_data)
head(pd_data)

The returned pd_data object contains several attributes that document its structure and the transformations applied during standardization. These attributes can be inspected using:

names(attributes(pd_data))

| Attribute | Description | |:---|:---| | names | Stores the column names of the standardized data frame. | | row.names | Stores the row identifiers used by the data frame. | | class | Identifies the object classes, including its data-frame and package-specific classes. | | pd_mapping | Stores the mapping used by subsequent package functions. Column names retain their input names, while baseline and cutoff times refer to the standardized grid. | | pd_original_mapping | Preserves the original user-supplied mapping on the raw data scale before identifiers, time points, and encodings were standardized. | | pd_check | Stores the validation results associated with the standardized dataset, including readiness indicators, detected issues, and recommended handling. | | pd_standardization | Contains time_map, id_map, attrition, and initial_check, recording identifier/time conversions, exclusion counts and subject-level reasons, and the input validation report. |

For exmaple, the original and standardized mappings can be compared using:

attr(pd_data, "pd_original_mapping")
attr(pd_data, "pd_mapping")

Among these attributes, pd_standardization is particularly important because it provides the primary audit trail for the changes made to the input dataset. It can be inspected directly using:

standardization <- attr(pd_data, "pd_standardization")
names(standardization)

It retains all observed assessment times within the analysis window defined by the mapping object and transforms the ordered time grid to consecutive integers (0, 1, ..., n).

standardization$time_map

Subject identifiers are similarly mapped to consecutive integers, explicitly recognized binary encodings are converted safely, and the resulting longitudinal dataset is sorted by subject and standardized analysis time.

For a subject to be retained, the dataset must contain one usable record at each retained assessment time.

head(standardization$id_map)

Rows outside the mapped time window are always removed. With drop = TRUE, rows with missing identifiers or times can also be removed; remaining subjects with missing visits or required analysis values are excluded in full. The attrition report records the row-removal counts and subject-level exclusions. Outcome values that are structurally unobservable after death or other trunction are distinguished from ordinary missing outcomes among surviving subjects and are therefore handled separately during validation and standardization.

standardization$attrition

Standardization performs a second validation on the retained data. If dropping subjects removes a treatment group, it returns a warning and a pd_check attribute with ready_for_analysis = FALSE; the HTE interfaces reject that object. Inspect this flag before continuing:

attr(pd_data, "pd_check")$ready_for_analysis

The ID audit map contains one row per retained subject, and detailed reports can contain row or subject indices. These attributes therefore grow with the data and the number of detected problems. They describe the standardization call; subsequent editing or subsetting does not recompute them. Revalidate changed data before analysis.

Finally, even when a built-in dataset such as BiSample, or a user-supplied dataset, already satisfies all PDRobust data requirements, it should still be processed through the package’s data-preparation workflow before analysis. This ensures that the dataset is formally validated, standardized, and supplied with the mapping and audit attributes required by downstream functions.