README.md

codecov R-CMD-check-win-macos R-CMD-check-linux Slack

ctrdata for aggregating and analysing clinical trials

The package ctrdata provides functions for retrieving (downloading) information on clinical trials from public registers, and for aggregating and analysing this information; it can be used for the

The motivation is to understand trends in design and conduct of trials, their availability for patients and their detailled results. The package is to be used within the R system; this README was reviewed on 2021-08-22 for version 1.7.0.9001.

Main features:

Remember to respect the registers’ terms and conditions (see ctrOpenSearchPagesInBrowser(copyright = TRUE)). Please cite this package in any publication as follows: “Ralf Herold (2021). ctrdata: Retrieve and Analyze Clinical Trials in Public Registers. R package version 1.6, https://cran.r-project.org/package=ctrdata” Package ctrdata has been used for: Blogging on Innovation coming to paediatric research and a Report on The impact of collaboration: The value of UK medical research to EU science and health

Installation

1. Install package in R

Package ctrdata is on CRAN and on GitHub. Within R, use the following commands to install package ctrdata:

# Install CRAN version:
install.packages("ctrdata")

# Alternatively, install development version: 
install.packages("devtools")
devtools::install_github("rfhb/ctrdata", build_vignettes = TRUE)

These commands also install the package dependencies, which are nodbi, jsonlite, httr, curl, clipr, xml2, rvest,DBI and stringi.

2. Command line tools perl, sed, cat and php (5.2 or higher)

These are required for ctrLoadQueryIntoDb(), the main function of package ctrdata.

Vignettes

Overview of functions in ctrdata

The functions are listed in the approximate order of use in a user’s workflow.

| Function name | Function purpose | |------------------------------------|-----------------------------------------------------------------------------------------------------------------------| | ctrOpenSearchPagesInBrowser() | Open search pages of registers or execute search in web browser | | ctrFindActiveSubstanceSynonyms() | Find synonyms and alternative names for an active substance | | ctrGetQueryUrl() | Import from clipboard the URL of a search in one of the registers | | ctrLoadQueryIntoDb() | Retrieve (download) or update, and annotate, information on trials from a register and store in database | | dbQueryHistory() | Show the history of queries that were downloaded into the database | | dbFindIdsUniqueTrials() | Get the identifiers of de-duplicated trials in the database | | dbFindFields() | Find names of variables (fields) in the database | | dbGetFieldsIntoDf() | Create a data.frame from trial records in the database with the specified fields | | dfTrials2Long() 🔔 | Transform the data.frame from dbGetFieldsIntoDf() into a long name-value data.frame, including deeply nested fields | | dfName2Value() 🔔 | From a long name-value data.frame, extract values for variables (fields) of interest (e.g., endpoints) | | dfMergeTwoVariablesRelevel() | Merge two simple variables into a new variable, optionally map values to a new set of values | | installCygwinWindowsDoInstall() | Convenience function to install a Cygwin environment (MS Windows only) |

Example workflow

The aim is to download protocol-related trial information and tabulate the trials’ status of conduct.

library(ctrdata)
ctrOpenSearchPagesInBrowser()

# Please review and respect register copyrights:
ctrOpenSearchPagesInBrowser(copyright = TRUE)
q <- ctrGetQueryUrl()
# * Found search query from EUCTR: query=cancer&age=under-18&phase=phase-one&status=completed

q
#                                                   query-term  query-register
# 1 query=cancer&age=under-18&phase=phase-one&status=completed           EUCTR

Under the hood, scripts euctr2json.sh and xml2json.php (in ctrdata/exec) transform EUCTR plain text files and CTGOV XML files to ndjson format, which is imported into the database. The database is specified first, using nodbi (using RSQlite or MongoDB as backend); then, trial information is retrieved and loaded into the database:

# Connect to (or newly create) an SQLite database 
# that is stored in a file on the local system:
db <- nodbi::src_sqlite(
  dbname = "some_database_name.sqlite_file", 
  collection = "some_collection_name")

# See section Databases below 
# for MongoDB as alternative

# Retrieve trials from public register:
ctrLoadQueryIntoDb(
  queryterm = q,
  con = db)
# * Found search query from EUCTR: query=cancer&age=under-18&phase=phase-one&status=completed
# (1/3) Checking trials in EUCTR:
# Retrieved overview, multiple records of 66 trial(s) from 4 page(s) to be downloaded
# Checking helper binaries: done
# Downloading trials (4 pages in parallel)...
# Note: register server cannot compress data, transfer takes longer, about 0.4s per trial
# Pages: 4 done, 0 ongoing   
# (2/3) Converting to JSON, 248 records converted
# (3/3) Importing JSON records into database...
# = Imported or updated 248 records on 66 trial(s)                
# * Updated history ("meta-info" in "some_collection_name")

Tabulate the status of trials that are part of an agreed paediatric development program (paediatric investigation plan, PIP):

# Get all records that have values in the fields of interest:
result <- dbGetFieldsIntoDf(
  fields = c(
    "a7_trial_is_part_of_a_paediatric_investigation_plan", 
    "p_end_of_trial_status", 
    "a2_eudract_number"),
  con = db)

# Find unique trial identifiers for trials that have nore than 
# one record, for example for several EU Member States: 
uniqueids <- dbFindIdsUniqueTrials(con = db)
# Searching for duplicate trials... 
#  - Getting trial ids, 248 found in collection
#  - Finding duplicates among registers' and sponsor ids...
#  - 182 EUCTR _id were not preferred EU Member State record for 66 trials
#  - Keeping 66 records from EUCTR
# = Returning keys (_id) of 66 records in collection "some_collection_name"

# Keep only unique / de-duplicated records:
result <- result[ result[["_id"]] %in% uniqueids, ]

# Tabulate the selected clinical trial information:
with(result, 
     table(
       p_end_of_trial_status, 
       a7_trial_is_part_of_a_paediatric_investigation_plan))
#                     a7_trial_is_part_of_a_paediatric_investigation_plan
# p_end_of_trial_status      Information not present in EudraCT No Yes
#   Completed                                                 6 31  16
#   GB - no longer in EU/EEA                                  0  5   4
#   Ongoing                                                   0  1   0
#   Prematurely Ended                                         1  1   0
#   Restarted                                                 0  1   0
# Retrieve trials from another register:
ctrLoadQueryIntoDb(
  queryterm = "cond=neuroblastoma&rslt=With&recrs=e&age=0&intr=Drug", 
  register = "CTGOV",
  con = db)
# * Found search query from CTGOV: cond=neuroblastoma&rslt=With&recrs=e&age=0&intr=Drug
# (1/3) Checking trials in CTGOV:
# Retrieved overview, records of 40 trial(s) are to be downloaded
# Checking helper binaries: done
# Downloading: 580 kB     
# (2/3) Converting to JSON, 40 records converted
# (3/3) Importing JSON records into database...
# = Imported or updated 40 trial(s)                              
# * Updated history ("meta-info" in "some_collection_name")
# Retrieve trials from another register:
ctrLoadQueryIntoDb(
  queryterm = "https://www.isrctn.com/search?q=neuroblastoma",
  con = db)
# * Found search query from ISRCTN: q=neuroblastoma
# (1/3) Checking trials in ISRCTN:
# Retrieved overview, records of 9 trial(s) are to be downloaded
# Checking helper binaries: done
# Downloading: 89 kB       
# (2/3) Converting to JSON, 9 records converted
# (3/3) Importing JSON records into database...
# = Imported or updated 9 trial(s)                              
# * Updated history ("meta-info" in "some_collection_name")

Analyse some simple result details (see vignette for more examples):

# Get all records that have values in any of the specified fields
result <- dbGetFieldsIntoDf(
  fields = c(
    "clinical_results.baseline.analyzed_list.analyzed.count_list.count",
    "clinical_results.baseline.group_list.group",
    "clinical_results.baseline.analyzed_list.analyzed.units", 
    "study_design_info.allocation", 
    "location"),
  con = db)

# Transform all fields into long name - value format
result <- dfTrials2Long(df = result)
# Total 6140 rows, 12 unique names of variables

# [1.] get counts of subjects for all arms into data frame
# This count is in the group that has "Total" in its name
nsubj <- dfName2Value(
  df = result, 
  valuename = "clinical_results.baseline.analyzed_list.analyzed.count_list.count.value",
  wherename = "clinical_results.baseline.group_list.group.title",
  wherevalue = "Total"
)

# [2.] count number of sites
nsite <- dfName2Value(
  df = result, 
  # some ctgov records use
  # location.name, others use
  # location.facility.name
  valuename = "^location.*name$"
)
# count
nsite <- tapply(
  X = nsite[["value"]], 
  INDEX = nsite[["_id"]],
  FUN = length,
  simplify = TRUE
)
nsite <- data.frame(
  "_id" = names(nsite),
  nsite, 
  check.names = FALSE,
  stringsAsFactors = FALSE,
  row.names = NULL
)

# [3.] randomised?
ncon <- dfName2Value(
  df = result, 
  valuename = "study_design_info.allocation"
) 

# merge sets
nset <- merge(nsubj, nsite, by = "_id")
nset <- merge(nset, ncon, by = "_id")

# Example plot
library(ggplot2)
ggplot(data = nset) + 
  labs(title = "Neuroblastoma trials with results",
       subtitle = "clinicaltrials.gov") +
  geom_point(
    mapping = aes(
      x = nsite,
      y = value.x,
      colour = value.y == "Randomized")) + 
  scale_x_log10() + 
  scale_y_log10() 
ggsave(filename = "inst/image/README-ctrdata_results_neuroblastoma.png",
       width = 5, height = 3, units = "in")

Neuroblastoma
trials

Databases

The database connection object con is created by calling nodbi::src_*(), with parameters that are specific to the database (e.g., url) and with a special parameter collection that is used by ctrdata to identify which table or collection in the database to use. Any such connection object can then be used by ctrdata and generic functions of nodbi in a consistent way, as shown in the table:

| Purpose | SQLite | MongoDB | |---------------------------------------|---------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------| | Create database connection | dbc <- nodbi::src_sqlite(dbname = ":memory:", collection = "name_of_my_collection") | dbc <- nodbi::src_mongo(db = "name_of_my_database", collection = "name_of_my_collection", url = "mongodb://localhost") | | Use connection with ctrdata functions | ctrdata::{ctr,db}*(con = dbc) | ctrdata::{ctr,db}*(con = dbc) | | Use connection with nodbi functions | nodbi::docdb_*(src = dbc, key = dbc$collection) | nodbi::docdb_*(src = dbc, key = dbc$collection) |

Features in the works

Acknowledgements

Issues and notes

Trial records’ JSON in databases

MongoDB

Example JSON representation in
MongoDB

SQLite

Example JSON representation in
SQLite



Try the ctrdata package in your browser

Any scripts or data that you put into this service are public.

ctrdata documentation built on Aug. 23, 2021, 1:09 a.m.