In nyuglobalties/panelcleaner: An interactive interface to homogenize messy panel data into a long format

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

panelcleaner

Sometimes during data collection, survey structures may change over time due to a litany of potential issues. In order to combine these datasets together into a long format, panelcleaner attempts to identify as many issues as possible prior to binding datasets together by thoroughly documenting the state of each variable for each wave. Moreover, with the assistance of rcoder, categorical data can be easily recoded into a single, homogenized coding while not losing the labels or any associated metadata.

Usage

library(panelcleaner)
library(magrittr)
set.seed(9381)

panelcleaner is built around a panel mapping file, which is a spreadsheet like this:

panel_map_csv <- tibble::tribble(
  ~ name_1, ~ description_1, ~ coding_1, ~ name_2, ~ description_2, ~ coding_2, ~ panel, ~ homogenized_name, ~ homogenized_coding,
  "id", "Participant ID", "", "id", "Partcipant ID", "", "test_panel", "id", "",
  "question1", "The first question", "", "q1", "The first question", "", "test_panel", "question_1", "",
  "question2", "The second question", "coding(code('Yes', 1), code('No', 0))", "q2", "The second question", "coding(code('Yes', 'YES'), code('No', 'NO'))", "test_panel", "question_2", "coding(code('Yes', 1), code('No', 0))"
)

kableExtra::kable(panel_map_csv)

The "name", "description", and "coding" columns refer to how the data was structured in waves 1 & 2. The "code" columns are specific to categorical variables, and the coding() objects (written out as strings to be interpreted later) represent the categorical level structure. See rcoder for more information.

Once the mapping is defined, the homogenization process can begin. The first step in the process is to gather waves of data into a panel object:

wave1 <- tibble::tibble(
  id = LETTERS[1:4],
  question1 = sample(1:100, 4, replace = TRUE),
  question2 = sample(0:1, 4, replace = TRUE)
)

wave2 <- tibble::tibble(
  id = LETTERS[1:4],
  q1 = sample(1:100, 4, replace = TRUE),
  q2 = sample(c("NO", "YES"), 4, replace = TRUE)
)

test_panel <- enpanel("test_panel", wave1, wave2)
test_panel

The panel name is "test_panel". Since multiple panels can be defined in the same mapping file, a panel name must be defined to distinguish which mapping belongs to which panel. Moreover, the panel is an unhomogenized panel, which means that waves cannot yet be bound together to form a single, long-format dataset.

The next step is to attach a panel mapping to the created panel:

panel_map <-
  panel_map_csv %>% 
  panel_mapping(waves = c(1, 2))

test_panel <-
  test_panel %>% 
  add_mapping(panel_map)

test_panel

panel_mapping() needs information about the specific waves mapped out, so we pass the loaded CSV data.frame through panel_mapping() to check that the CSV adheres to the panel mapping spec. Then we add the mapping to the panel so that the homogenization process can refer to it. Finally, we homogenize the panel.

homogenized_panel <-
  test_panel %>% 
  homogenize_panel() %>% 
  bind_waves()

homogenized_panel

To extract the underlying data, use as.data.frame() to convert the homogenized panel to a mapped_df, a data.frame-like object that carries the panel mapping as an attribute.

str(as.data.frame(homogenized_panel))

Installation

As panelcleaner does not exist on CRAN yet, you can install the latest development version of this package via:

# install.packages("remotes")
remotes::install_github("Global-TIES-for-Children/panelcleaner")

Contributing

Please note that the panelcleaner project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

nyuglobalties/panelcleaner documentation built on March 30, 2023, 11:01 a.m.