validation_diagnostics: Emulator Diagnostics

In hmer: History Matching and Emulation Package

Emulator Diagnostics

Description

Performs the standard set of validation diagnostics on emulators.

Usage

validation_diagnostics(
  emulators,
  targets = NULL,
  validation = NULL,
  which_diag = c("cd", "ce", "se"),
  analyze = TRUE,
  diagnose = "expectation",
  ...
)

Arguments

emulators	A list of Emulator objects.
targets	The list of observations for the outputs
validation	The validation set, containing all inputs and outputs.
which_diag	Which diagnostics should be performed (see description)
analyze	Should plotting and/or failing points be returned?
diagnose	For bimodal systems, should the expectation or variance be considered?
...	Any additional parameters to pass to the diagnostics (eg sd, cutoff, ...)

Details

All the diagnostics here can be performed with or without a validation (or ‘holdout’) set of data. The presence of a set of targets is optional for some checks but mandatory for others: the appropriate warnings will be given in the event that some checks cannot be applied.

The current options for diagnostics (with the codes for which_diag) are:

Standardised Errors (se)

Comparison Diagnostics (cd)

Classification Errors (ce)

All of the above (all)

For details of each of the tests, see the help file for get_diagnostic.

Value

A data.frame containing points that failed one or more diagnostic tests.

See Also

Other diagnostic functions: analyze_diagnostic(), classification_diag(), comparison_diag(), get_diagnostic(), individual_errors(), residual_diag(), standard_errors(), summary_diag()

Examples

validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation)
# data.frame of failed points (empty) and a 3x3 set of plots
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation,
 c('ce','cd'))
# empty data.frame and a 3x2 set of plots
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, SIRSample$validation,
 cutoff = 2, sd = 2)
# k-fold (with k = 3)
validation_diagnostics(SIREmulators$ems, SIREmulators$targets, k = 3)

hmer documentation built on June 22, 2024, 9:22 a.m.