gp.model.adequacy: Model assessment based on Deviance information criterion...

In GPBayes: Tools for Gaussian Process Modeling in Uncertainty Quantification

Model assessment based on Deviance information criterion (DIC), logarithmic pointwise predictive density (lppd), and logarithmic joint predictive density (ljpd).

Description

This function computes effective number of parameters (pD), deviance information criterion (DIC), logarithmic pointwise predictive density (lppd), and logarithmic joint predictive density (ljpd). For detailed introduction of these metrics, see Chapter 7 of Gelman et al. (2013).

The deviance function for a model with a vector of parameters \boldsymbol \theta is defined as

D(\boldsymbol \theta) = -2\log p(\mathbf{y} \mid \boldsymbol \theta),

where \mathbf{y}:=(y(\mathbf{x}_1), \ldots, y(\mathbf{x}_n))^\top is a vector of n observations.

• The effective number of parameters (see p.172 of Gelman et al. 2013) is defined as

  pD = E_{\boldsymbol \theta| \mathbf{y}}[D(\boldsymbol \theta)] - D(\hat{ \boldsymbol \theta }),

  where \hat{\boldsymbol \theta} = E_{\boldsymbol \theta | \mathbf{y}}[\boldsymbol \theta]. The interpretation is that the effective number of parameters is the “expected" deviance minus the “fitted" deviance. Higher pD implies more over-fitting with estimate \hat{\boldsymbol \theta}.

• The Deviance information criteria (DIC) (see pp. 172-173 of Gelman et al. 2013) is

  DIC = E_{\boldsymbol \theta | \mathbf{y}}[D(\boldsymbol \theta)] + pD.

  DIC approximates Akaike information criterion (AIC) and is more appropriate for hierarchical models than AIC and BIC.

• The log predictive density (lpd) is defined as

  p(y(\mathbf{x}_0) \mid \mathbf{y}) = \int p(y(\mathbf{x}_0) \mid \boldsymbol \theta, \mathbf{y}) p(\boldsymbol \theta \mid \mathbf{y}) d \boldsymbol \theta,

  where \mathbf{y}:=(y(\mathbf{x}_1), \ldots, y(\mathbf{x}_n))^\top is a vector of n observations. \boldsymbol \theta contains correlation parameters and nugget parameter. This predictive density should be understood as an update of the likelihood since data is treated as prior information now. With a set of prediction locations \mathcal{X}:=\{x_0^i: i=1, \ldots, m\}. The log pointwise predictive density (lppd) is defined as

  lppd = \sum_{i=1}^m \log p(y(\mathbf{x}_0^i) \mid \mathbf{y}).

  The log joint predictive density (ljpd) is defined as

  ljpd = \log p(y(\mathcal{X})).

  The lppd is connected to cross-validation, while the ljpd measures joint uncertainty across prediction locations.

Usage

gp.model.adequacy(
  obj,
  testing.input,
  testing.output,
  pointwise = TRUE,
  joint = TRUE
)

Arguments

obj	a gp object.
testing.input	a matrix of testing inputs
testing.output	a vector of testing outputs
pointwise	a logical value with default value TRUE. If it is TRUE, lppd is calculated.
joint	a logical value with default value TRUE. If it is TRUE, ljpd is calculated.

Value

a list containing pD, DIC, lppd, ljpd.

Author(s)

Pulong Ma [e-mail]

References

• Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin (2013). Bayesian Data Analysis, Third Edition. CRC Press.

See Also

GPBayes-package, GaSP, gp,

GPBayes documentation built on May 29, 2024, 7:30 a.m.