In zdaarau/c2d: Download Data from the Referendum Database (RDB), Which Covers Direct Democratic Votes Worldwide

NOTES

• This file is not included in the source package because of the .nopurl suffix in its filename.

• The chunks below have to be manually executed in order to update the backed up data.

Setup

library(rlang,
        include.only = c("%|%", "%||%"))
library(magrittr,
        include.only = c("%>%", "%<>%", "%T>%", "%!>%", "%$%"))

path_backup_rdb <- "data-raw/backups/rdb.rds"

Back up RDB data

NOTE: Full backup used to be unreliable at times due to behind-the-scenes changes (rate limiting?) by CCM Design. Since we're hosting the infra ourselves now (with support from PIT Solutions), this shouldn't be an issue anymore. If such issues should still occur, see the chunk below for a workaround.

#| eval: TRUE

rdb:::rfrnds(is_draft = NULL,
             incl_archive = TRUE,
             use_cache = FALSE) |>
  readr::write_rds(file = path_backup_rdb,
                   compress = "xz",
                   version = 3L)

Update RDB backup

NOTE: Alternative to creating a full backup (chunk above).

#| eval: FALSE

data_latest <- rdb:::rfrnds(date_time_last_edited_min = clock::as_date(pal::path_mod_time(path_backup_rdb)),
                            is_draft = NULL,
                            incl_archive = TRUE,
                            use_cache = FALSE)

readr::read_rds(path_backup_rdb) |>
  dplyr::filter(!(id %in% data_latest$id)) |>
  dplyr::bind_rows(data_latest) |>
  readr::write_rds(file = path_backup_rdb,
                   compress = "xz",
                   version = 3L)

Back up RDB file attachments

NOTE: Make sure to first commit pending changes to data-raw/backups/rdb.rds to ensure file attachments from recently added referendums are backed up, too.

path_backup_s3 <- "~/Arbeit/ZDA/Lokal/Projekte/RDB/s3_backup"

rdb::rfrnds(is_draft = NULL,
            use_cache = FALSE) %$%
  files |>
  purrr::list_rbind() |>
  dplyr::filter(!is_deleted) %$%
  s3_object_key |>
  purrr::walk(.f = \(x) {
    if (!fs::file_exists(fs::path(path_backup_s3, x))) {
      rdb::download_file_attachment(s3_object_key = x,
                                    path = path_backup_s3)
    }
  },
  .progress = TRUE)

zdaarau/c2d documentation built on Dec. 18, 2024, 1:24 p.m.