validateEmulator: Validate the Emulator
In OakleyJ/MUCM: Gaussian process emulator methods based on the MUCM toolkit

Description Usage Arguments Details Value Author(s) References See Also

Uses diagnostics described in Bastos and O'Hagan (2009) to validate the emulator.

1 2	validateEmulator(emulator, new.outputs, emulator.predictions, which.output = 1, plot = TRUE)

`emulator`	Object of class `emulatorFit` generated by `fitEmulator` or `fitEmulatorData`
`new.outputs`	A matrix outputs used to validate the emulator predictions. This Matrix must have columns equal to the number of outputs.
`emulator.predictions`	Output generated from the `predict.emulatorFit` function.
`which.output`	Defaults to 1. Is an indicative integer that validates the output stated based on column number from the left.
`plot`	Optionally plot validation plots. Default is set to true.

validateEmulator returns a grid of 4 plots and prints out a value. These are described below:

Top Left plot	Plots validation output against the posterior mean generated using the emulator. The error bars are the 95percent confidence intervals of the predicted mean. Ideally, the plot should show all points close to the diagonal line with small error bars.
Top Right plot	Shows the QQ plot of cholesky residuals. If the points lie close to the 45 degree line then the normality assumptions for the simulator output is reasonable. If the gradient of the points is greater than 1 (less than 1), it suggests that the predictive variability was underestimated (over estimated). Curvature in the plot indicates nonnormality, and outliers at either end of the plot suggests local fitting problems or nonstationarity.
Bottom Left plot	Plot of Pivoted cholesky prediciton errors against the pivoting index. The pivoting index gives the order of the Pivoted cholesky prediciton errors with the largest conditional predictive variance. No patterns are expected. Too many large errors indicate an underestimation of variance and vice versa.Both cases can aslo suggest a nonstationary process. Either large or very small errors at the beginning of the plot (i.e., on the left side) indicates poor estimation of predictive variance or nonstationarity. However, large (or very small) errors at the end of the plot (i.e., on the right side) indicates overestimation (or underestimation) of the correlation length parameters, or that the chosen correlation structure is unsuitable.
Bottom right Plot	The red line on the plot is the value of the calculated Mahalanobis distance taking into account correlation among outputs. Ideally it should be between the boundaries of the curve. Extreme values (large or small) indicate a conflict between the emulator and simulator.

The class of 'validateEmulator' is a list containting atleast the following components:

`coverage`	Proportion of validation outputs in 95 percent confidence intervals of the emulator predicted output
`RMSE`	The root mean square error of the predicted values
`normRMSE`	The normalised value of `RMSE` over the range of the data