knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) source(system.file("extdata", "vignette-helpers.R", package = "ospsuite.reportingengine"))
require(ospsuite.reportingengine)
This vignette introduces the notions and objects implemented in the OSP-Suite Reporting Engine package (ospsuite.reportingengine
).
The ospsuite.reportingengine
package aims at facilitating the design and build of reports evaluating PBPK models developed on PK-Sim or/and MoBi.
To this end, the package benefits from the concept of R6 classes (similar to reference classes) which allows to define structured objects whose fields can be any object such as properties, methods or even other R6 objects. R6 objects, or objects from R6 class, presents some peculiarities.
$new(...)
(e.g. r6Object <- R6Class$new()
).$fieldName
or [[fieldName]]
(e.g. r6Object$method()
or r6Object[[property]]
).The ospsuite.reportingengine
package is built around 3 central R6 classes MeanModelWorkflow
, PopulationWorkflow
and QualificationWorkflow
derived from the parent R6 class Workflow
.
The creation of MeanModelWorkflow
or PopulationWorkflow
objects provides a structured environment with methods and properties that facilitate the design, run and report of the evaluations of PBPK models developed on PK-Sim or/and MoBi.
For QualificationWorkflow
objects, more details are available in a dedicated article.
The Worklow
objects combine 2 features defined as R6 objects:
SimulationSet
and PopulationSimulationSet
objects defines which and how model(s) and/or data can be used by the workflow (developed in more details in section 3).
These objects need to be created before the creation of workflow objects as they are required to create the workflow.Task
objects define the evaluations to be performed by the workflow. These objects are directly created and included upon creation of the Workflow
objects (developed in more details in section 4). Their settings can be checked and updated after creation.Table 1 enumerates some of the main instances of the ospsuite.reportingengine
package usually defined by users during the design of a workflow.
The tab Reference documents available classes and functions exported by the ospsuite.reportingengine
package.
The comprehensive documentation and usage of each of these objects is accessible in the tab Reference.
Table 1: Nomenclature of ospsuite.reportingengine
main instances
|Workflow objects|Use case| |--------------------|------------| |MeanModelWorkflow|Defined by user| |PopulationWorkflow|Defined by user| |QualificationWorkflow|Built in loadQualificationWorkflow| |Simulation Set objects|Use case| |SimulationSet|Defined by user| |PopulationSimulationSet|Defined by user| |ConfigurationPlan|Built in loadQualificationWorkflow| |Output path and PK parameter objects|Use case| |Output|Defined by user| |PkParameterInfo|Defined by user| |Workflow task objects|Use case| |Task|Built in workflow objects| |SimulationTask|Built in workflow objects| |SensitivityAnalysisTask|Built in workflow objects| |PopulationSensitivityAnalysisTask|Built in workflow objects| |QualificationTask|Built in QualificationWorkflow objects| |PlotTask|Built in workflow objects| |PopulationPlotTask|Built in PopulationWorkflow objects| |GofPlotTask|Built in workflow objects| |Workflow task settings objects|Use case| |PlotSettings|Built in workflow objects| |SensitivityPlotSettings|Built in workflow objects|
In order to define and run appropriately a Workflow object, a few steps are required:
ospsuite.reportingengine
packagelibrary(ospsuite.reportingengine)
Output$new()
SimulationSet$new()
Workflow$new()
workflow$activateTasks()
workflow$runWorkflow()
A working example of workflow script is provided below for reporting a time profile plot on Minimodel2.pkml.
Code
# Get the pkml simulation file: "MiniModel2.pkml" simulationFile <- system.file("extdata", "MiniModel2.pkml", package = "ospsuite.reportingengine" ) # Create Output objects which define output paths that will be plotted by the workflow outputA <- Output$new( path = "Organism|A|Concentration in container", displayName = "Concentration of A" ) outputB <- Output$new( path = "Organism|B|Concentration in container", displayName = "Concentration of B" ) # Create a SimulationSet object which defines the model and outputs to evaluate myExampleSet <- SimulationSet$new( simulationSetName = "My example", simulationFile = simulationFile, outputs = c(outputA, outputB) ) # Create the workflow object myExampleWorkflow <- MeanModelWorkflow$new( simulationSets = myExampleSet, workflowFolder = "myExample-Results" ) # Set the workflow tasks to be run myExampleWorkflow$activateTasks(c("simulate", "plotTimeProfilesAndResiduals")) # Run the workflow myExampleWorkflow$runWorkflow()
For this example, the list of files and folders generated by the workflow are:
list.files(myExampleWorkflow$workflowFolder)
The associated report will be as follows:
cat(includeReportFromWorkflow(myExampleWorkflow))
# Remove the workflow folders unlink(myExampleWorkflow$workflowFolder, recursive = TRUE)
Workflow
objects define which models are evaluated, how they are evaluated and how the evaluations are reported.
In the ospsuite.reportingengine
package, two types of Workflow
are available and can be created:
1) MeanModelWorkflow
dedicated on evaluations specific to mean models
2) PopulationWorkflow
dedicated on evaluations specific to population models
Workflows defined by MeanModelWorkflow
objects can be illustrated by Figure 1.
The blue frame corresponds to the inputs of the workflow.
These inputs need to be defined using SimulationSet
objects.
For workflows with multiple simulations and observed datasets,
a list of SimulationSet
objects can be input instead of one unique SimulationSet
object.
Within a Workflow
object, the simulation sets are accessible from the simulationStructures
lists which also provides relevant information about input/output files and folders
The black frames correspond to the tasks or evaluations performed by the workflow.
These tasks are defined using Task
objects built in every Workflow
object, meaning there is no need to input them when creating a Workflow
object.
Users access, update and switch the task on/off tasks after creation of their Workflow
.
# After running each vignette, their output is cleared when using devtools::check() # However, the content of "figures/" directory is required for docs/articles that display vignettes as a webpage # Thus, the file is temporarily copied to "figures/" dir.create("figures", showWarnings = FALSE) file.copy( from = system.file( "extdata", "mean-model-workflow-input-and-tasks.png", package = "ospsuite.reportingengine" ), to = "figures", overwrite = TRUE ) file.copy( from = system.file( "extdata", "population-workflow-input-and-tasks.png", package = "ospsuite.reportingengine" ), to = "figures", overwrite = TRUE )
knitr::include_graphics("figures/mean-model-workflow-input-and-tasks.png")
Workflows defined by PopulationWorkflow
objects can be illustrated by Figure 2.
The blue frame corresponds to the inputs of the workflow.
These inputs need to be defined using PopulationSimulationSet
objects.
For workflows with multiple simulations and observed datasets,
a list of PopulationSimulationSet
objects can be input instead of one unique PopulationSimulationSet
object.
Within a Workflow
object, the simulation sets are accessible from the simulationStructures
lists which also provides relevant information about input/output files and folders
The black frames correspond to the tasks or evaluations performed by the workflow.
These tasks are defined using Task
objects built in every Workflow
object, meaning there is no need to input them when creating a Workflow
object.
Users access, update and switch the task on/off tasks after creation of their Workflow
.
Some tasks of population workflows perform a direct comparison of the simulation sets (e.g. plotDemography
or plotPKParameters
).
Such comparisons can be different according to the selected population workflow types.
Three population workflow types are available in ospsuite.reportingengine
and defined in PopulationWorkflowTypes
:
1) Pediatric (workflowType = "pediatric"
)
- All properties (physiology and PK Parameter) are plotted vs. age and weight,
- The time profiles are plotted in comparison to a reference population the sensitivity analysis is done on all populations except the reference population
2) Parallel Comparison (workflowType = "parallelComparison"
)
- PK parameter are plotted parallel in Box-Whisker plots without a reference population,
- If a reference population is given, the time profiles are plotted in comparison to this population
- The sensitivity analysis is done on all populations
3) Ratio Comparison (workflowType = "ratioComparison"
)
- Same as parallel comparison, but for the PK Parameter additional the ratio of the PK Parameter to the reference population is calculated and plotted in box-whisker plots
knitr::include_graphics("figures/population-workflow-input-and-tasks.png")
Another type of workflow can be defined by QualificationWorkflow
objects using a configuration plan for qualification purposes.
The article Qualification Workflow details how to build and use such QualificationWorkflow
objects.
To create a MeanModelWorkflow
or a PopulationWorkflow
object, the method $new()
needs to be used with at least the inputs defined below.
myWorkflow <- MeanModelWorkflow$new(simulationSets, workflowFolder)
myWorkflow <- PopulationWorkflow$new(workflowType, simulationSets, workflowFolder)
where the input simulationSets
is a list of SimulationSet
objects for a mean model workflow, and a list of PopulationSimulationSet
objects for a population workflow.
The input workflowFolder
is the name of the folder in which the workflow outputs are saved.
As illustrated in Figures 1 and 2, some tasks use outputs obtained from previous tasks.
If the directory specified in workflowFolder
contains such outputs, they will be used by the current workflow.
This latter option can be useful for updating and running only parts of the workflow without having to perform all the simulations every time.
The input workflowType
is one of the 3 population workflow types as defined by PopulationWorkflowTypes
.
Other optional inputs can be added when creating a workflow:
createWordReport
: logical defining if a word version (.docx) of the report should be saved besides its markdown version (.md). The default value for this option is TRUE
, meaning that a report in word is also created. watermark
: character defining the text to display in figures background.
The default value is NULL
, in which case the text "preliminary results" will be displayed in the figures for all computers using an non-validated environment.simulationSetDescriptor
: character defining a descriptor of how simulation sets should be indicated in reports (e.g. "scenario", "set", "simulation", or "population").
The default value for this option is NULL
, meaning that there won't be any descriptor of the simulation sets.After creation of a Workflow
object, some built in methods and properties are available
(using the character $
after the name of the object).
Most of the workflow properties correspond to the inputs used to create the workflow and includes createWordReport
, workflowFolder
, reportFilePath
, the list simulationStructures
and the workflow tasks (developed in section 4).
Additionally, the methods listed below can be used to access relevant information about the workflow:
printReportingEngineInfo()
: prints information about the environment and packages currently used.getWatermark()
: prints the watermark to be displayed by the workflow.getParameterDisplayPaths()
: prints the mapping between parameters and their display pathsgetSimulationDescriptor()
: prints the simulation descriptor to be used by the workflowgetAllTasks()
: prints the names of all the workflow available tasksgetAllPlotTasks()
: prints the names of all the workflow plot tasksgetActiveTasks()
: prints the names of all the workflow active tasksgetInactiveTasks()
: prints the names of all the workflow inactive tasksTo update some of the workflow properties, the methods listed below can be used:
setWatermark(watermark)
: set the watermark to be displayed by the workflow.
The input watermark
is of type character.setParameterDisplayPaths(parameterDisplayPaths)
: set the the mapping between parameters and their display paths. The input parameterDisplayPaths
is of type data.frame and includes "parameter" and "displayPath" in its variables. External functions can also be used instead:setWorkflowParameterDisplayPaths(parameterDisplayPaths, workflow)
setWorkflowParameterDisplayPathsFromFile(fileName, workflow)
setSimulationDescriptor(text)
: set the simulation descriptor of the workflow.
The input text
is of type character. Finally, the method runWorkflow()
runs the simulations and analyses of the models, and also saves the evaluations and their report.
SimulationSet
and PopulationSimulationSet
objects includes all the relevant information needed to report the evaluations of a model.
Below is the syntax for creating such simulation sets, the next sub-sections will provide more details on each input:
Simulation Set
SimulationSet$new( simulationSetName, simulationFile, outputs, dataSource, dataSelection, applicationRanges, timeUnit, timeOffset, minimumSimulationEndTime )
Population Simulation Set
PopulationSimulationSet$new( referencePopulation, simulationSetName, simulationFile, populationFile, studyDesignFile, plotReferenceObsData, outputs, dataSource, dataSelection, applicationRanges, timeUnit, timeOffset, minimumSimulationEndTime )
A simulation file, simulationFile
is an export of a simulation from MoBi or PK-SIM in pkml format.
Display names for the simulation set can be provided using simulationSetName
.
A population file, populationFile
, is collection of parameter paths and parameter values normally an export of a population from PK-SIM in csv format.
It is also possible to use an M&S activity-specific population file, which has the “PK-Sim” format, but was manipulated after the export from PK-SIM or was generated outside of PK-Sim.
The generation of the population file than must be validated.
Display names for the population can be provided using simulationSetName
.
The study design file, studyDesignFile
, contains additional information on the study design, e.g. a body weight dependent dose in a pediatric study.
A regular csv format is expected for such a file.
The example below shows a template of such a study design content.
studyDesignFile <- system.file("extdata", "StudyDesign.csv", package = "ospsuite.reportingengine") exampleStudyDesign <- readObservedDataFile(studyDesignFile)
knitr::kable(exampleStudyDesign, caption = "Table 2: Example of study design content")
If the workflow uses data, two files must be provided within a DataSource
object as follows:
DataSource$new(dataFile, metaDataFile, caption = NULL)
The optional caption
argument defines the data source displayed caption in the report.
The Data file can be a blank separated text file or a csv file, column headers are used later as R variable names, and they must not contain special letters like blanks. The data is expected to follow a tidy format like Nonmem formatted datasets. All data columns used for figures and tables, must be numerical and listed in the dictionary/meta data file (details in section 3.3.2). For data filtering, these columns and additional ones can be used, the additional columns may also contain strings. But be careful that these strings do not contain the column separator blank. A column must be either numerical or of type string, they must not contain both.
dictionaryFile <- system.file("extdata", "tpDictionary.csv", package = "ospsuite.reportingengine") exampleDictionary <- readObservedDataFile(dictionaryFile)
The dictionary is a csv file mapping the data file. Unlike the Matlab version of the reporting engine, a regular csv with a comma (",") as separator is expected.
The dictionary file must contain the following variables: r paste0("'", paste0(names(exampleDictionary), collapse = "', '"), "'")
.
For time profile plots, you must provide 'time' and 'dv' in 'ID'.
The variable 'lloq', for lower limit of quantitation, can also be provided but is not necessary.
These variables need to be mapped to the variable names of the dataset using the dictionary variable 'nonmenColumn'.
Regarding units, two options are possible:
The example below shows content of the dictionary template tpDictionary.csv:
knitr::kable(exampleDictionary, caption = "Table 3: Template for data dictionary")
For any simulation set, Output
objects define which simulation paths, associated PK parameters and associated observed data are evaluated.
Output
objects are also R6 objects created using the method $new()
.
Below is the syntax for creating such Output
objects, the next sub-sections will provide more details on each input:
Output
Output$new( path, displayName, displayUnit, dataSelection, dataDisplayName, pkParameters )
The input variable path
indicates the path name within a simulation (e.g. 'Organism|PeripheralVenousBlood|Raltegravir|Plasma (Peripheral Venous Blood)') that needs to be included in the simulation run.
Display name and unit for this path can be provided using displayName
and displayUnit
.
For tasks such as goodness of fit, observed data can be used.
Usually, the data is included into one unique Nonmem data file which needs to be filtered and associated to the correct path
.
The input variable dataSelection
provides a filter for the Nonmem data file.
It must be R readable code, using the Nonmem headers as variable names (e.g. 'SID>0' or 'AGE<12 & SEX==1').
dataSelection
= 'NONE'. Consequently, no data is selected and no evaluation is performed.dataSelection
= 'ALL'.The input pkParameters
indicates the path
related PK parameters that the user wants to include in his analysis.
A list of pk parameter names can be directly provided to this input (e.g. c('C_max', 'AUC_inf')).
Display names and display units will be used as is from the PK parameter objects defined using the ospsuite
package.
In the case different display names or/and units are needed between the same PK parameters but from different paths, it is also possible to use PkParameterInfo
instances instead of pk parameter names directly.
PkParameterInfo$new( pkParameter, displayName, displayUnit )
However, in the case the same display names and units are used,
the better practice is to define or update the PK parameters, their display names and units beforehand with the ospsuite
package,
then to provide directly the list of their names to the input pkParameters
.
If the simulation set corresponds to multiple administrations, the reported evaluations on some application ranges can be switched on/off.
The input applicationRanges
corresponds to the names of application ranges to include in the report for the simulation set.
By default, all application ranges are included into the report.
The enum ApplicationRanges
provide the names of application ranges.
ApplicationRanges
This input is then translated in the SimulationSet
objects as a list of logical values defining whether the corresponding application range is included in the report or not.
The input timeOffset
shifts the time variable of time profile plots by its value in timeUnit
. By default this value is 0
.
This input can be useful for pediatric workflows comparing simulation sets in the same time profile plot.
In scenarios where multiple applications are defined, the time range of first application actually becomes the time range of first application since timeOffset
. Likewise, the time range of last application becomes the time range of last application since timeOffset
.
As illustrated in Figures 1 and 2, workflows perform multiple evaluations or tasks on a list of simulation sets.
Such evaluations are defined using Task
objects which are built-in properties of Workflow
objects.
To know which tasks your workflow can perform, the workflow method getAllTasks()
can be used.
Usually, not all the tasks are required to run in a workflow, which led Task
objects to include the logical field active
defining if a task should be performed or not.
The workflow methods getActiveTasks()
and getInactiveTasks()
provide the names of active and inactive tasks of the workflow, respectively.
To modify which tasks should be run, the active
field of any task can be updated.
The workflow methods activateTasks(taskNames)
and inactivateTasks(inactivateTasks)
provide a fast and easy way to update which tasks will be run.
In particular, their default value for the input taskNames
is getAllTasks()
, meaning that using activateTasks()
without argument will activate every task of the workflow.
Additionally, the enums StandardSimulationTasks
and StandardPlotTasks
respectively provide a quick access to standard simulation and plot tasks common to mean model and population workflows.
# Names of standard simulation tasks StandardSimulationTasks # Names of standard plot tasks StandardPlotTasks
Using the example shown in section 1.4.. The task names in the mean model workflow were:
# All workflow's tasks myExampleWorkflow$getAllTasks() # Only active tasks that are run myExampleWorkflow$getActiveTasks() # Only inactive tasks that are not run myExampleWorkflow$getInactiveTasks()
The activation/inactivation of tasks can be done as follows:
# Inactivate all tasks myExampleWorkflow$inactivateTasks() myExampleWorkflow$getActiveTasks() # Activate only tasks "simulate" and "plotTimeProfilesAndResiduals" tasks myExampleWorkflow$activateTasks(c("simulate", "plotTimeProfilesAndResiduals")) myExampleWorkflow$getActiveTasks()
As mentioned above, some tasks can be inactivated. Because some evaluations can take a lot of time, especially sensitivity analyses, the use of simulation results as an input of subsequent tasks is essential.
Before running any task, workflows will check if the task required input files are available.
This feature can also be accessed by users from the task method getInputs()
which can be combined with the function file.exists
# Get list of files or folders required to run task "plotTimeProfilesAndResiduals" myExampleWorkflow$plotTimeProfilesAndResiduals$getInputs()
Task
objects include many properties necessary to log the evaluations, to ensure a continuity between input and output files of different tasks and to build the final report of the workflow.
Among the Task
properties, the field settings
includes the task properties that can be updated for the evaluation, log or report of the workflow.
The task settings
are R6 objects whose properties can be directly accessed as shown below (using the character $
after settings
).
# Get value of settings property "showProgress" for the task "simulate" myExampleWorkflow$simulate$settings$showProgress # Get value of settings property "digits" for the task "plotPKParameters" myExampleWorkflow$plotPKParameters$settings$digits # Get the names of the "plotConfigurations" list used during the task "plotTimeProfilesAndResiduals" names(myExampleWorkflow$plotTimeProfilesAndResiduals$settings$plotConfigurations)
Once the workflow and its settings have been defined, the workflow method $runWorkflow() will run all the active tasks and generate all the associated results and report.
The following outputs will be generated in workflowFolder
directory: logs, results directories and reports.
Three types of logs can be generated during the creation, design and run of a workflow:
Each task will save its results in a dedicated directory. The names of the results directory are as follow:
simulate
task, as csv files.
It is possible to load the results using the ospsuite
method importResultsFromCSV().calculatePKParameters
task, as csv files.
It is possible to load the results using the ospsuite
method importPKAnalysesFromCSV().calculateSensitivity
task, as csv files.
It is possible to load the results using the ospsuite
method importSensitivityAnalysisResultsFromCSV().plotTimeProfilesAndResiduals
task and their source values as png and csv files.plotPKParameters
task and their source values as png and csv files.plotSensitivity
task and their source values as png and csv files.plotMassBalance
task and their source values as png and csv files.
This directory is specific of mean model workflows.plotAbsorption
task and their source values as png and csv files.
This directory is specific of mean model workflows.plotDemography
task and their source values as png and csv files.
This directory is specific of population model workflows.Default figure format is png, but it is possible to change this default using setPlotFormat()
.
Each plot task will save an appendix file besides its results as a markdown format report.
These appendices are saved directly within the workflowFolder
directory results directory with the following names:
plotTimeProfilesAndResiduals
task.plotPKParameters
task.plotSensitivity
task.plotMassBalance
task.plotAbsorption
task.plotDemography
task.At the end of the workflow run, all the available appendices are combined into final reports:
The option createWordReport
is set to TRUE as default but can be set to FALSE to prevent the conversion of Report-word.md into Report.docx using pandoc .
# Save templates in the templates folder templateDir <- system.file("extdata", package = "ospsuite.reportingengine") templateFiles <- c( "WorkflowInput.xlsx", "reference.docx", "qualification-workflow-template.R", "tpDictionary.csv", "StudyDesign.csv", "re-theme.json", "mass-balance-settings.json" ) dir.create("templates", showWarnings = FALSE) file.copy(from = file.path(templateDir, templateFiles), to = file.path("templates", templateFiles), overwrite = TRUE)
For creating or tuning reports, starting from scratch could be challenging. For this reason, some templates have been created to make it easier. The table below provides a list of available templates and their usage.
|Template |Link |Usage |
|---------|-----|---------|
|Excel workflow input |WorkflowInput.xlsx |createWorkflowFromExcelInput()
|
|Word reference report|reference.docx | wordConversionTemplate
argument of Worklfow
objects|
|Qualification workflow script|qualification-workflow-template.R | Source file to create function createQualificationReport()
|
|Observed data dictionary|tpDictionary.csv|observedMetaDataFile
argument of SimulationSet
objects|
|Study Design|StudyDesign.csv| studyDesignFile
argument of PopulationSimulationSet
objects|
|json theme|re-theme.json|setDefaultThemeFromJson()
|
|Mass Balance settings|mass-balance-settings.json|massBalanceFile
argument of SimulationSet
objects|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.