Description Details Super class Public fields Methods Examples
Exact inference for GP regression; this inference method can only be used
with a Gaussian likelihood (see LikGauss
)
In Gaussian process (GP) regression, we consider that our outcomes y are noisy observations of an unknown function of the predictors X:
y = f (X) + ε.
We place a GP prior over the values of f(X), which says that before observing y, we believe the values of f(X) are distributed
f (X) ~ N ( m (X), K (X, X) ),
where m ( X ) is a mean function and K ( X, X ) is a covariance function. In other words, the prior mean over function outputs is a function of the predictors, as is the prior covariance over function outputs. Application of Bayes' rule in conjunction with Gaussian identities gives us the posterior distribution of the values of f(X) after observing y. This is a flexible Bayesian non-parametric framework to reason about the relationship between predictors and outcomes without making strong functional form assumptions.
For GP regression, the posterior distribution has an analytical solution:
f | y, X ~ N(μ + K K_y^{-1} (y - μ), K - K K_y^{-1} K),
where K denotes the prior covariance function (and we have suprressed in the notation that here it is evaluated at X), μ is the prior mean of f, and K_y^{-1} = (K + σ_y^2 I)^{-1}. For new test cases X*, f* is distributed
f* | y, X, X* ~ N(μ* + K(X*, X) K_y^{-1} (y - μ), K(X*) - K(X*, X) K_y^{-1} K(X, X*)),
where μ* is the prior mean function evaluated at X*.
gpmss::GPModel
-> GPR
y
The outcomes; should be a numeric vector. This field is usually generated automatically during object construction and does not generally need to be interacted with directly by the user.
X
The predictors; should be a numeric matrix. This field is usually generated automatically during object construction and does not generally need to be interacted with directly by the user.
terms
The terms
object related to the
formula provided in model construction (this is useful for
later prediction).
force_all_levs
A logical vector of length one recording the user's preference for dropping unused factor levels
meanfun
The prior mean function; must be an object of class
MeanFunction
covfun
The prior covariance function; must be an object of
class CovarianceFunction
likfun
The likelihood function; must be an object of class
LikelihoodFunction
L
A numeric matrix such that L L^T = K + σ_y^2 I
alpha
A numeric vector equal to (K + σ_y^2 I)^{-1} (y - μ)
post_mean
A numeric vector giving the posterior mean at the training data
post_cov
A numeric vector giving the posterior covariance at the training data
prior_mean
A numeric vector giving the prior mean at the training data
marginal_effects
A list with an element for each predictor in
the model marginal effects have been requested for (via the
margins()
method). Each element is a list of length two,
with an element "mean", each entry i of which gives the mean of
the distribution of the marginal effect of the predictor on the
ith observation, and an element "covariance", giving the
covariance matrix for the distribution of marginal effects of
that predictor.
average_marginal_effects
A dataframe with a row for each
predictor in the model that marginal effects have been requested
for (via the margins()
method) and columns:
"Variable", giving the name of the predictor;
"Mean", giving the mean of the average marginal effect of that
predictor;
"LB", giving the lower bound on the requested confidence
interval (see the margins()
method);
and "UB", giving the upper bound on the CI.
new()
Create a new GPModel object
GPR$new( formula, data, likfun = LikGauss, meanfun = MeanZero, covfun = CovSEard, optimize = FALSE, force_all_levs = FALSE, ... )
formula
A formula object giving the variable name of the
outcomes on the left-hand side and the predictors on the
right-hand side, a la lm
data
An optional data frame where the variables in
formula
are to be found. If not found there,
we search for the variables elsewhere, generally the
calling environment.
likfun
An object inheriting from class
LikelihoodFunction
, or a generator for such a
class. This is the likelihood function for the GP model.
This is restricted to be LikGauss
.
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
meanfun
An object inheriting from class
MeanFunction
, or a generator for such a
class. This is the mean function for the GP prior (see Details).
The default is MeanZero
.
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
covfun
An object inheriting from class
CovarianceFunction
, or a generator for such a
class.
This is the covariance function for the GP prior (see Details).
The default is CovSEard
.
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
optimize
A logical vector of length one; if TRUE
,
the hyperparameters of the mean, covariance, and likelihood
functions are optimized automatically as part of the model
construction process (see Details).
The default is FALSE
, meaning the hyperparameters given
at the time of creating those objects are used for inference.
force_all_levs
A logical vector of length one; if TRUE
,
unused factor levels in right-hand side variables are
not dropped. The default is FALSE
.
...
Other arguments specific to a particular model; unused. Train the GP model, providing a characterization of the posterior of the function of interest at the input values given in the training data.
train()
GPR$train(...)
...
Additional arguments affecting the inference calculations
predict()
Characterize the posterior predictive distribution of the function of interest at new test points.
GPR$predict(newdata, ...)
newdata
A data frame containing the data for the new test points
...
Additional arguments affecting the predictions produced
nlml()
Caclulate the negative log marginal likelihood of the GP model.
GPR$nlml(...)
...
Additional arguments affecting the calculation
dnlml()
Caclulate the gradient of the negative log marginal likelihood of the GP model with respect to the hyperparameters of the mean, covariance, and likelihood functions.
GPR$dnlml(...)
...
Additional arguments affecting the calculation
margins()
Caclulate marginal effects of predictors.
GPR$margins( variables = NULL, base_categories = NULL, differences = NULL, indices = NULL, ci = 0.95, force = FALSE, ... )
variables
A vector specifying the variables marginal effects are desired for; can be an integer vector, giving the column indices of X to get effects for, or a character vector; if NULL (the default), effects are derived for all variables.
base_categories
If X contains contrasts, the marginal effects will be the differences between the levels and a base level. By default, the base level is the lowest factor level of the contrast, but you can pass a named list to change the base level for some or all of the variables assigned to a contrast scheme.
differences
A named list of 2-length numeric vectors may be provided giving the low (first element of each vector) and high (second element) values to calculate the effect at for continuous variables. Any elements giving values for binary or categorical variables are ignored (this is meaningless for binary variables as there are only two values to choose from, and categorical variables should be controlled with the option base_categories). If NULL (the default), derivatives are used for marginal effects of continuous predictors.
indices
A numeric vector of indices over which to average marginal effects. If NULL (the default), all observations are used.
ci
A numeric vector of length one such that 0 < ci < 1 giving the width of the confidence interval for the average marginal effects. The default is 0.95, corresponding to a 95% confidence interval.
force
A logical vector of length one; should marginal effects for variables who have already had effects calculated be re-calculated? The default is FALSE. (This is useful in case the user has requested effects for continuous variables be calculated as differences but would now prefer derivatives, or vice versa).
...
Other arguments affecting the calculations (unused)
The derivative of a GP is also a GP, giving us easy access to the
distribution of marginal effects of predictors. The first time this
method is called, it calculates and stores the distribution of
pointwise partial derivatives of f for each specified predictor
in the model (if no predictors are specified, marginal effects are
calculated for all predictors). If a predictor is binary, instead
the distribution of the difference in f between the predictor
taking the binary variable's highest value and its lowest value is
calculated. If a predictor is categorical, a similar calculation is
made, but comparing each of the labels except one to a baseline
label. The user may specify to use a similar calculation for two
user specified values for continuous variables rather than using
the partial derivatives (in some cases this may be more easily
interpretable). Additional calls to margins()
may calculate
marginal effects for predictors whose marginal effects had not yet
been requested, but marginal effects for variables that have already
been calculated are not re-calculated unless force = TRUE
.
Every time the method is called, it stores a dataframe of average marginal effects; it calculates the mean and variance of the marginal effect of each predictor over all observations, or over specified indices of observations if provided.
clone()
The objects of this class are cloneable with this method.
GPR$clone(deep = FALSE)
deep
Whether to make a deep clone.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | ## Simulate data
set.seed(123)
n <- 100
x <- rnorm(n, sd = 2)
y <- sin(x) + rnorm(n, sd = sqrt(0.1))
## Train model
m <- GPR$new(y ~ x)
## Visualize posterior over f at training points
plot(x, y, pch = "+", col = "#33333387")
curve(sin(x), lty = 2, add = TRUE)
o <- order(x)
lines(x[o], m$post_mean[o])
s <- sqrt(diag(m$post_cov))[o]
h <- m$post_mean[o] + 1.96 * s
l <- m$post_mean[o] - 1.96 * s
polygon(x = c(x[o], rev(x[o])), y = c(h, rev(l)),
border = NA, col = "#87878733")
legend(x = -3, y = max(y) + 0.1, bty = "n",
pch = c(3, NA, NA, 15), lty = c(NA, 2, 1, NA),
col = c("#333333", "black", "black", "#87878733"),
legend = c("Observations", "True f(x)", "Post. mean", "95% CI"))
## Get negative log marginal likelihood of the model
m$nlml()
## Look at derivatives of negative log marginal likelihood wrt hypers
m$dnlml()
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.