GPModel: Gaussian Process Model
In duckmayr/gpmss: Gaussian Process Models for Social Science
Description
Details
Public fields
Methods
Description
GPModel provides a parent class for all Gaussian process (GP) model classes
in gpmss. A GP model class keeps track of the data and various
posterior quantities, as well as the inference method and likelihood,
mean, and covariance functions. It has a number of methods to produce,
summarize, or plot quantities of interest such as predictive distributions
or avarage marginal effects.
Details
In Gaussian process (GP) regression, we consider that our outcomes y
are noisy observations of an unknown function of the predictors X:
y = f ( X ) + \varepsilon.
We place a GP prior over the values of f(X),
which says that before observing y, we believe the values of
f(X) are distributed
f ( X ) \sim N ( m ( X ), K ( X, X ) ),
where m ( X ) is a mean function
and K ( X, X ) is a
covariance function.
In other words, the prior mean over function outputs is a function of the
predictors, as is the prior covariance over function outputs.
Application of Bayes' rule in conjunction with Gaussian identities gives
us the posterior distribution of the values of f(X)
after observing y. This is a flexible Bayesian non-parametric
framework to reason about the relationship between predictors and outcomes
without making strong functional form assumptions.
GP classification works similarly, though now f(X)
is a latent function that is pushed through a link function to produce
outcomes, not unlike generalized linear models. While the exact posterior
for GP regression can be derived analytically, in the classification case
the posterior is analytically untractable, so several approximations to
the posterior are available, and the posterior distribution can be
simulated via MCMC methods.
A GPModel child class implements one such inference method for GP
regression or classification. The GP models already implemented by
gpmss are
- GPR
-
Exact inference for GP regression.
This is only possible with a Gaussian likelihood.
- GPCLA
-
The Laplace approximation to the posterior for GP classification.
Users may also define their own GP models that utilize gpmss functions.
A GPModel subclass should provide a train() method,
a predict() method, a nlml() method, a dnlml()
method, and a margins() method.
A subclass may wish to provide a custome optimize() method for
optimizing hyperparameters, though the parent class' optimize()
method should be general purpose enough to work for any subclass designed
according to the description provided here.
It should have data members name, y, X,
meanfun (to hold an instance of a subclass of MeanFunction),
covfun (to hold an instance of a subclass of CovarianceFunction),
likfun (to hold an instance of a subclass of LikelihoodFunction),
marginal_effects, and average_marginal_effects,
but which other data members are appropriate may vary by inference method.
For most models, a data member L should be present,
which is a (lower) Cholesky decomposition of a matrix of central importance
in posterior inference, as well as alpha, which is usually
K^{-1} (μ_{post} - μ_{prior}),
where μ is the posterior or prior mean of the function of interest
respectively.
But, again, one should consult specific models' help files
as the data members that are useful will differ between models.
The train() method is used to characterize the posterior distribution
of the function of interest at the training cases provided in the
data argument of the constructor
(or found in the calling environment given the formula provided).
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor. However, an ellipses
(...) argument should always conclude the argument list.
The predict() method is used to characterize the posterior
predictive distribution of the function of interest at new test cases.
The first argument should be newdata,
a data frame containing the data for the new test cases.
For many models, these may be the only arguments needed,
but additional arguments may be appropriate.
An ellipses (...) argument should always conclude the argument list.
The nlml() method is used to calculate the negative log marginal
likelihood.
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor or generated during the
call to train(). However, an ellipses
(...) argument should always conclude the argument list.
The dnlml() method is used to calculate the gradient of the negative
log marginal likelihood with respect to the hyperparameters of the mean,
covariance, and likelihood functions.
(This is used for hyperparameter optimization).
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor or generated during the
call to train(). However, an ellipses
(...) argument should always conclude the argument list.
The margins() method is used to calculate marginal (and/or partial)
effects of predictors.
The arguments may be method dependent.
An ellipses (...) argument should always conclude the argument list.
The data member name should be hard-coded within the class definition;
it is used for printing purposes (potentially in other functions). It should
be a public member so that it can be accessed directly.
Public fields
yThe outcomes; should be a numeric vector.
Depending on the model there may be further restrictions
on the nature of the data.
This field is usually generated automatically during object
construction and does not generally need to be interacted
with directly by the user.
XThe predictors; should be a numeric matrix.
This field is usually generated automatically during object
construction and does not generally need to be interacted
with directly by the user.
termsThe terms object related to the
formula provided in model construction (this is useful for
later prediction).
force_all_levsA logical vector of length one recording the
user's preference for dropping unused factor levels
meanfunThe prior mean function; must be an object of class
MeanFunction
covfunThe prior covariance function; must be an object of
class CovarianceFunction
likfunThe likelihood function; must be an object of class
LikelihoodFunction
marginal_effectsA list with an element for each predictor in
the model marginal effects have been requested for (via the
margins() method). Each element is a list of length two,
with an element "mean", each entry i of which gives the mean of
the distribution of the marginal effect of the predictor on the
ith observation, and an element "covariance", giving the
covariance matrix for the distribution of marginal effects of
that predictor.
average_marginal_effectsA dataframe with a row for each
predictor in the model that marginal effects have been requested
for (via the margins() method) and columns:
"Variable", giving the name of the predictor;
"Mean", giving the mean of the average marginal effect of that
predictor;
"LB", giving the lower bound on the requested confidence
interval (see the margins() method);
and "UB", giving the upper bound on the CI.
Methods
Public methods
-
-
-
-
-
-
-
-
Method new()
Create a new GPModel object
Usage
GPModel$new(
formula,
data,
likfun = NULL,
meanfun = NULL,
covfun = NULL,
optimize = FALSE,
force_all_levs = FALSE,
...
)
Arguments
formulaA formula object giving the variable name of the
outcomes on the left-hand side and the predictors on the
right-hand side, a la lm
dataAn optional data frame where the variables in
formula are to be found. If not found there,
we search for the variables elsewhere, generally the
calling environment.
likfunAn object inheriting from class
LikelihoodFunction, or a generator for such a
class. This is the likelihood function for the GP model.
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
meanfunAn object inheriting from class
MeanFunction, or a generator for such a
class. This is the mean function for the GP prior (see Details).
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
covfunAn object inheriting from class
CovarianceFunction, or a generator for such a
class.
This is the covariance function for the GP prior (see Details).
If a generator is provided rather than an object,
an object will be created with the default value for the
hyperparameters.
optimizeA logical vector of length one; if TRUE,
the hyperparameters of the mean, covariance, and likelihood
functions are optimized automatically as part of the model
construction process (see Details).
The default is FALSE, meaning the hyperparameters given
at the time of creating those objects are used for inference.
force_all_levsA logical vector of length one; if TRUE,
unused factor levels in right-hand side variables are
not dropped. The default is FALSE.
...Other arguments specific to a particular model;
see the help file for specific models for more details.
Train the GP model, providing a characterization of the posterior
of the function of interest at the input values given in the
training data.
Method train()
Usage
GPModel$train(...)
Arguments
...Additional arguments affecting the inference calculations
Method predict()
Characterize the posterior predictive distribution of the function
of interest at new test points.
Usage
GPModel$predict(newdata, ...)
Arguments
newdataA data frame containing the data for the new test
points
...Additional arguments affecting the predictions produced
Method nlml()
Caclulate the negative log marginal likelihood of the GP model.
Usage
GPModel$nlml(...)
Arguments
...Additional arguments affecting the calculation
Method dnlml()
Caclulate the gradient of the negative log marginal likelihood of
the GP model with respect to the hyperparameters of the mean,
covariance, and likelihood functions.
Usage
GPModel$dnlml(...)
Arguments
...Additional arguments affecting the calculation
Method margins()
Caclulate marginal effects of predictors.
Usage
GPModel$margins(
variables = NULL,
base_categories = NULL,
differences = NULL,
indices = NULL,
ci = 0.95,
force = FALSE,
...
)
Arguments
variablesA vector specifying the variables marginal effects
are desired for; can be an integer vector, giving the column
indices of X to get effects for, or a character vector;
if NULL (the default), effects are derived for all variables.
base_categoriesIf X contains contrasts,
the marginal effects will be the differences between the levels
and a base level. By default, the base level is the lowest
factor level of the contrast, but you can pass a named list to
change the base level for some or all of the variables assigned
to a contrast scheme.
differencesA named list of 2-length numeric vectors
may be provided giving the low (first element of each vector)
and high (second element) values to calculate the effect at for
continuous variables. Any elements giving values for binary or
categorical variables are ignored (this is meaningless for
binary variables as there are only two values to choose from,
and categorical variables should be controlled with the option
base_categories).
If NULL (the default), derivatives are used for marginal
effects of continuous predictors.
indicesA numeric vector of indices over which to average
marginal effects. If NULL (the default), all observations are
used.
ciA numeric vector of length one such that 0 < ci < 1
giving the width of the confidence interval for the average
marginal effects. The default is 0.95, corresponding to a
95% confidence interval.
forceA logical vector of length one; should marginal effects
for variables who have already had effects calculated be
re-calculated? The default is FALSE. (This is useful in case the
user has requested effects for continuous variables be
calculated as differences but would now prefer derivatives,
or vice versa).
...Other arguments affecting the calculations
Details
The derivative of a GP is also a GP, giving us easy access to the
distribution of marginal effects of predictors. The first time this
method is called, it calculates and stores the distribution of
pointwise partial derivatives of f for each specified predictor
in the model (if no predictors are specified, marginal effects are
calculated for all predictors). If a predictor is binary, instead
the distribution of the difference in f between the predictor
taking the binary variable's highest value and its lowest value is
calculated. If a predictor is categorical, a similar calculation is
made, but comparing each of the labels except one to a baseline
label. The user may specify to use a similar calculation for two
user specified values for continuous variables rather than using
the partial derivatives (in some cases this may be more easily
interpretable). Additional calls to margins() may calculate
marginal effects for predictors whose marginal effects had not yet
been requested, but marginal effects for variables that have already
been calculated are not re-calculated unless force = TRUE.
Every time the method is called, it stores a dataframe of average
marginal effects; it calculates the mean and variance of the
marginal effect of each predictor over all observations, or over
specified indices of observations if provided.
Method optimize()
Set hyperparameters by optimizing the log marginal likelihood
Usage
GPModel$optimize(
method = "CG",
line_search = "Rasmussen",
step0 = "rasmussen",
...
)
Arguments
methodOptimization method; default is "CG"
line_searchType of line search; default is "Rasmussen"
step0Line search value for the first step; default is
"rasmussen"
...Other arguments passed to mize
Details
The default settings of this function should result in an
optimization routine that is very similar to that employed in the
MATLAB/Octave software GPML. For details on the optimization options,
see mize
Method clone()
The objects of this class are cloneable with this method.
Usage
GPModel$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
duckmayr/gpmss documentation built on Nov. 8, 2021, 5:48 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Description Details Public fields Methods
Description
GPModel provides a parent class for all Gaussian process (GP) model classes
in gpmss. A GP model class keeps track of the data and various
posterior quantities, as well as the inference method and likelihood,
mean, and covariance functions. It has a number of methods to produce,
summarize, or plot quantities of interest such as predictive distributions
or avarage marginal effects.
Details
In Gaussian process (GP) regression, we consider that our outcomes y are noisy observations of an unknown function of the predictors X:
y = f ( X ) + \varepsilon.
We place a GP prior over the values of f(X), which says that before observing y, we believe the values of f(X) are distributed
f ( X ) \sim N ( m ( X ), K ( X, X ) ),
where m ( X ) is a mean function and K ( X, X ) is a covariance function. In other words, the prior mean over function outputs is a function of the predictors, as is the prior covariance over function outputs. Application of Bayes' rule in conjunction with Gaussian identities gives us the posterior distribution of the values of f(X) after observing y. This is a flexible Bayesian non-parametric framework to reason about the relationship between predictors and outcomes without making strong functional form assumptions.
GP classification works similarly, though now f(X) is a latent function that is pushed through a link function to produce outcomes, not unlike generalized linear models. While the exact posterior for GP regression can be derived analytically, in the classification case the posterior is analytically untractable, so several approximations to the posterior are available, and the posterior distribution can be simulated via MCMC methods.
A GPModel child class implements one such inference method for GP
regression or classification. The GP models already implemented by
gpmss are
- GPR
-
Exact inference for GP regression. This is only possible with a Gaussian likelihood.
- GPCLA
-
The Laplace approximation to the posterior for GP classification.
Users may also define their own GP models that utilize gpmss functions.
A GPModel subclass should provide a train() method,
a predict() method, a nlml() method, a dnlml()
method, and a margins() method.
A subclass may wish to provide a custome optimize() method for
optimizing hyperparameters, though the parent class' optimize()
method should be general purpose enough to work for any subclass designed
according to the description provided here.
It should have data members name, y, X,
meanfun (to hold an instance of a subclass of MeanFunction),
covfun (to hold an instance of a subclass of CovarianceFunction),
likfun (to hold an instance of a subclass of LikelihoodFunction),
marginal_effects, and average_marginal_effects,
but which other data members are appropriate may vary by inference method.
For most models, a data member L should be present,
which is a (lower) Cholesky decomposition of a matrix of central importance
in posterior inference, as well as alpha, which is usually
K^{-1} (μ_{post} - μ_{prior}),
where μ is the posterior or prior mean of the function of interest
respectively.
But, again, one should consult specific models' help files
as the data members that are useful will differ between models.
The train() method is used to characterize the posterior distribution
of the function of interest at the training cases provided in the
data argument of the constructor
(or found in the calling environment given the formula provided).
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor. However, an ellipses
(...) argument should always conclude the argument list.
The predict() method is used to characterize the posterior
predictive distribution of the function of interest at new test cases.
The first argument should be newdata,
a data frame containing the data for the new test cases.
For many models, these may be the only arguments needed,
but additional arguments may be appropriate.
An ellipses (...) argument should always conclude the argument list.
The nlml() method is used to calculate the negative log marginal
likelihood.
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor or generated during the
call to train(). However, an ellipses
(...) argument should always conclude the argument list.
The dnlml() method is used to calculate the gradient of the negative
log marginal likelihood with respect to the hyperparameters of the mean,
covariance, and likelihood functions.
(This is used for hyperparameter optimization).
For many models, arguments will not be needed; the quantities necessary for
inference should be provided in the constructor or generated during the
call to train(). However, an ellipses
(...) argument should always conclude the argument list.
The margins() method is used to calculate marginal (and/or partial)
effects of predictors.
The arguments may be method dependent.
An ellipses (...) argument should always conclude the argument list.
The data member name should be hard-coded within the class definition;
it is used for printing purposes (potentially in other functions). It should
be a public member so that it can be accessed directly.
Public fields
yThe outcomes; should be a numeric vector. Depending on the model there may be further restrictions on the nature of the data. This field is usually generated automatically during object construction and does not generally need to be interacted with directly by the user.
XThe predictors; should be a numeric matrix. This field is usually generated automatically during object construction and does not generally need to be interacted with directly by the user.
termsThe
termsobject related to the formula provided in model construction (this is useful for later prediction).force_all_levsA logical vector of length one recording the user's preference for dropping unused factor levels
meanfunThe prior mean function; must be an object of class
MeanFunctioncovfunThe prior covariance function; must be an object of class
CovarianceFunctionlikfunThe likelihood function; must be an object of class
LikelihoodFunctionmarginal_effectsA list with an element for each predictor in the model marginal effects have been requested for (via the
margins()method). Each element is a list of length two, with an element "mean", each entry i of which gives the mean of the distribution of the marginal effect of the predictor on the ith observation, and an element "covariance", giving the covariance matrix for the distribution of marginal effects of that predictor.average_marginal_effectsA dataframe with a row for each predictor in the model that marginal effects have been requested for (via the
margins()method) and columns: "Variable", giving the name of the predictor; "Mean", giving the mean of the average marginal effect of that predictor; "LB", giving the lower bound on the requested confidence interval (see themargins()method); and "UB", giving the upper bound on the CI.
Methods
Public methods
Method new()
Create a new GPModel object
Usage
GPModel$new( formula, data, likfun = NULL, meanfun = NULL, covfun = NULL, optimize = FALSE, force_all_levs = FALSE, ... )
Arguments
formulaA formula object giving the variable name of the outcomes on the left-hand side and the predictors on the right-hand side, a la
lmdataAn optional data frame where the variables in
formulaare to be found. If not found there, we search for the variables elsewhere, generally the calling environment.likfunAn object inheriting from class
LikelihoodFunction, or a generator for such a class. This is the likelihood function for the GP model. If a generator is provided rather than an object, an object will be created with the default value for the hyperparameters.meanfunAn object inheriting from class
MeanFunction, or a generator for such a class. This is the mean function for the GP prior (see Details). If a generator is provided rather than an object, an object will be created with the default value for the hyperparameters.covfunAn object inheriting from class
CovarianceFunction, or a generator for such a class. This is the covariance function for the GP prior (see Details). If a generator is provided rather than an object, an object will be created with the default value for the hyperparameters.optimizeA logical vector of length one; if
TRUE, the hyperparameters of the mean, covariance, and likelihood functions are optimized automatically as part of the model construction process (see Details). The default isFALSE, meaning the hyperparameters given at the time of creating those objects are used for inference.force_all_levsA logical vector of length one; if
TRUE, unused factor levels in right-hand side variables are not dropped. The default isFALSE....Other arguments specific to a particular model; see the help file for specific models for more details. Train the GP model, providing a characterization of the posterior of the function of interest at the input values given in the training data.
Method train()
Usage
GPModel$train(...)
Arguments
...Additional arguments affecting the inference calculations
Method predict()
Characterize the posterior predictive distribution of the function of interest at new test points.
Usage
GPModel$predict(newdata, ...)
Arguments
newdataA data frame containing the data for the new test points
...Additional arguments affecting the predictions produced
Method nlml()
Caclulate the negative log marginal likelihood of the GP model.
Usage
GPModel$nlml(...)
Arguments
...Additional arguments affecting the calculation
Method dnlml()
Caclulate the gradient of the negative log marginal likelihood of the GP model with respect to the hyperparameters of the mean, covariance, and likelihood functions.
Usage
GPModel$dnlml(...)
Arguments
...Additional arguments affecting the calculation
Method margins()
Caclulate marginal effects of predictors.
Usage
GPModel$margins( variables = NULL, base_categories = NULL, differences = NULL, indices = NULL, ci = 0.95, force = FALSE, ... )
Arguments
variablesA vector specifying the variables marginal effects are desired for; can be an integer vector, giving the column indices of X to get effects for, or a character vector; if NULL (the default), effects are derived for all variables.
base_categoriesIf X contains contrasts, the marginal effects will be the differences between the levels and a base level. By default, the base level is the lowest factor level of the contrast, but you can pass a named list to change the base level for some or all of the variables assigned to a contrast scheme.
differencesA named list of 2-length numeric vectors may be provided giving the low (first element of each vector) and high (second element) values to calculate the effect at for continuous variables. Any elements giving values for binary or categorical variables are ignored (this is meaningless for binary variables as there are only two values to choose from, and categorical variables should be controlled with the option base_categories). If NULL (the default), derivatives are used for marginal effects of continuous predictors.
indicesA numeric vector of indices over which to average marginal effects. If NULL (the default), all observations are used.
ciA numeric vector of length one such that 0 < ci < 1 giving the width of the confidence interval for the average marginal effects. The default is 0.95, corresponding to a 95% confidence interval.
forceA logical vector of length one; should marginal effects for variables who have already had effects calculated be re-calculated? The default is FALSE. (This is useful in case the user has requested effects for continuous variables be calculated as differences but would now prefer derivatives, or vice versa).
...Other arguments affecting the calculations
Details
The derivative of a GP is also a GP, giving us easy access to the
distribution of marginal effects of predictors. The first time this
method is called, it calculates and stores the distribution of
pointwise partial derivatives of f for each specified predictor
in the model (if no predictors are specified, marginal effects are
calculated for all predictors). If a predictor is binary, instead
the distribution of the difference in f between the predictor
taking the binary variable's highest value and its lowest value is
calculated. If a predictor is categorical, a similar calculation is
made, but comparing each of the labels except one to a baseline
label. The user may specify to use a similar calculation for two
user specified values for continuous variables rather than using
the partial derivatives (in some cases this may be more easily
interpretable). Additional calls to margins() may calculate
marginal effects for predictors whose marginal effects had not yet
been requested, but marginal effects for variables that have already
been calculated are not re-calculated unless force = TRUE.
Every time the method is called, it stores a dataframe of average marginal effects; it calculates the mean and variance of the marginal effect of each predictor over all observations, or over specified indices of observations if provided.
Method optimize()
Set hyperparameters by optimizing the log marginal likelihood
Usage
GPModel$optimize( method = "CG", line_search = "Rasmussen", step0 = "rasmussen", ... )
Arguments
methodOptimization method; default is "CG"
line_searchType of line search; default is "Rasmussen"
step0Line search value for the first step; default is "rasmussen"
...Other arguments passed to
mize
Details
The default settings of this function should result in an
optimization routine that is very similar to that employed in the
MATLAB/Octave software GPML. For details on the optimization options,
see mize
Method clone()
The objects of this class are cloneable with this method.
Usage
GPModel$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.