externalValidateDbPlp: externalValidateDbPlp - Validate a model on new databases

In OHDSI/PatientLevelPrediction: Develop Clinical Prediction Models Using the Common Data Model

externalValidateDbPlp - Validate a model on new databases

Description

This function extracts data using a user specified connection and cdm_schema, applied the model and then calcualtes the performance

Usage

externalValidateDbPlp(
  plpModel,
  validationDatabaseDetails = createDatabaseDetails(),
  validationRestrictPlpDataSettings = createRestrictPlpDataSettings(),
  settings = createValidationSettings(recalibrate = "weakRecalibration"),
  logSettings = createLogSettings(verbosity = "INFO", logName = "validatePLP"),
  outputFolder = NULL
)

Arguments

plpModel	The model object returned by runPlp() containing the trained model
validationDatabaseDetails	A list of objects of class databaseDetails created using createDatabaseDetails
validationRestrictPlpDataSettings	A list of population restriction settings created by createRestrictPlpDataSettings()
settings	A settings object of class validationSettings created using createValidationSettings
logSettings	An object of logSettings created using createLogSettings specifying how the logging is done
outputFolder	The directory to save the validation results to (subfolders are created per database in validationDatabaseDetails)

Details

Users need to input a trained model (the output of runPlp()) and new database connections. The function will return a list of length equal to the number of cdm_schemas input with the performance on the new data

Value

An externalValidatePlp object containing the following components

• model: The model object

• executionSummary: A list of execution details

• prediction: A dataframe containing the predictions

• performanceEvaluation: A dataframe containing the performance metrics

• covariateSummary: A dataframe containing the covariate summary

Examples

 
data("simulationProfile")
plpData <- simulatePlpData(simulationProfile, n=1000)
# first fit a model on some data, default is a L1 logistic regression
saveLoc <- file.path(tempdir(), "development")
results <- runPlp(plpData, 
                  outcomeId = 3,
                  saveDirectory = saveLoc,
                  populationSettings = 
                   createStudyPopulationSettings(requireTimeAtRisk=FALSE)
                  )
connectionDetails <- Eunomia::getEunomiaConnectionDetails()
Eunomia::createCohorts(connectionDetails)
# now validate the model on Eunomia
validationDatabaseDetails <- createDatabaseDetails(
  connectionDetails = connectionDetails,
  cdmDatabaseSchema = "main",
  cdmDatabaseName = "main",
  cohortDatabaseSchema = "main",
  cohortTable = "cohort",
  outcomeDatabaseSchema = "main",
  outcomeTable = "cohort",
  targetId = 1, # users of celecoxib
  outcomeIds = 3, # GIbleed
  cdmVersion = 5)
path <- file.path(tempdir(), "validation")
externalValidateDbPlp(results$model, validationDatabaseDetails, outputFolder = path)
# clean up
unlink(saveLoc, recursive = TRUE)
unlink(path, recursive = TRUE)

OHDSI/PatientLevelPrediction documentation built on Feb. 14, 2025, 9:44 a.m.