svmDP: Privacy-preserving Support Vector Machine
In DPpack: Differentially Private Statistical Analysis and Machine Learning
svmDP R Documentation
Privacy-preserving Support Vector Machine
Description
This class implements differentially private support vector
machine (SVM) \insertCitechaudhuri2011DPpack. It can be either weighted
\insertCiteYang2005DPpack or unweighted. Either the output or the
objective perturbation method can be used for unweighted SVM, though only
the output perturbation method is currently supported for weighted SVM.
Details
To use this class for SVM, first use the new method to
construct an object of this class with the desired function values and
hyperparameters, including a choice of the desired kernel. After
constructing the object, the fit method can be applied to fit the
model with a provided dataset, data bounds, and optional observation
weights and weight upper bound. In fitting, the model stores a vector of
coefficients coeff which satisfy differential privacy. Additionally,
if a nonlinear kernel is chosen, the models stores a mapping function from
the input data X to a higher dimensional embedding V in the form of a
method XtoV as required \insertCitechaudhuri2011DPpack. These
can be released directly, or used in conjunction with the predict
method to privately predict the label of new datapoints. Note that the
mapping function XtoV is based on an approximation method via
Fourier transforms \insertCiteRahimi2007,Rahimi2008DPpack.
Note that in order to guarantee differential privacy for the SVM model,
certain constraints must be satisfied for the values used to construct the
object, as well as for the data used to fit. These conditions depend on the
chosen perturbation method. First, the loss function is assumed to be
differentiable (and doubly differentiable if the objective perturbation
method is used). The hinge loss, which is typically used for SVM, is not
differentiable at 1. Thus, to satisfy this constraint, this class utilizes
the Huber loss, a smooth approximation to the hinge loss
\insertCiteChapelle2007DPpack. The level of approximation to the hinge
loss is determined by a user-specified constant, h, which defaults to 0.5,
a typical value. Additionally, the regularizer must be 1-strongly convex
and differentiable. It also must be doubly differentiable if objective
perturbation is chosen. If weighted SVM is desired, the provided weights
must be nonnegative and bounded above by a global or public value, which
must also be provided.
Finally, it is assumed that if x represents a single row of the dataset X,
then the l2-norm of x is at most 1 for all x. In order to ensure this
constraint is satisfied, the dataset is preprocessed and scaled, and the
resulting coefficients are postprocessed and un-scaled so that the stored
coefficients correspond to the original data. Due to this constraint on x,
it is best to avoid using a bias term in the model whenever possible. If a
bias term must be used, the issue can be partially circumvented by adding a
constant column to X before fitting the model, which will be scaled along
with the rest of X. The fit method contains functionality to add a
column of constant 1s to X before scaling, if desired.
Super classes
DPpack::EmpiricalRiskMinimizationDP.CMS -> DPpack::WeightedERMDP.CMS -> svmDP
Methods
Public methods
Method new()
Create a new svmDP object.
Usage
svmDP$new(
regularizer,
eps,
gamma,
perturbation.method = "objective",
kernel = "linear",
D = NULL,
kernel.param = NULL,
regularizer.gr = NULL,
huber.h = 0.5
)
Arguments
regularizerString or regularization function. If a string, must be
'l2', indicating to use l2 regularization. If a function, must have form
regularizer(coeff), where coeff is a vector or matrix, and
return the value of the regularizer at coeff. See
regularizer.l2 for an example. Additionally, in order to
ensure differential privacy, the function must be 1-strongly convex and
doubly differentiable.
epsPositive real number defining the epsilon privacy budget. If set
to Inf, runs algorithm without differential privacy.
gammaNonnegative real number representing the regularization
constant.
perturbation.methodString indicating whether to use the 'output' or
the 'objective' perturbation methods \insertCitechaudhuri2011DPpack.
Defaults to 'objective'.
kernelString indicating which kernel to use for SVM. Must be one of
'linear', 'Gaussian'. If 'linear' (default), linear SVM is used. If
'Gaussian,' uses the sampling function corresponding to the Gaussian
(radial) kernel approximation.
DNonnegative integer indicating the dimensionality of the transform
space approximating the kernel if a nonlinear kernel is used. Higher
values of D provide better kernel approximations at a cost of
computational efficiency. This value must be specified if a nonlinear
kernel is used.
kernel.paramPositive real number corresponding to the Gaussian
kernel parameter. Defaults to 1/p, where p is the number of predictors.
regularizer.grOptional function representing the gradient of the
regularization function with respect to coeff and of the form
regularizer.gr(coeff). Should return a vector. See
regularizer.gr.l2 for an example. If regularizer is
given as a string, this value is ignored. If not given and
regularizer is a function, non-gradient based optimization methods
are used to compute the coefficient values in fitting the model.
huber.hPositive real number indicating the degree to which the
Huber loss approximates the hinge loss. Defaults to 0.5
\insertCiteChapelle2007DPpack.
Returns
A new svmDP object.
Method fit()
Fit the differentially private SVM model. This method runs
either the output perturbation or the objective perturbation algorithm
\insertCitechaudhuri2011DPpack, depending on the value of
perturbation.method used to construct the object, to generate an
objective function. A numerical optimization method is then run to find
optimal coefficients for fitting the model given the training data,
weights, and hyperparameters. The built-in optim function
using the "BFGS" optimization method is used. If regularizer is
given as 'l2' or if regularizer.gr is given in the construction of
the object, the gradient of the objective function is utilized by
optim as well. Otherwise, non-gradient based optimization methods
are used. The resulting privacy-preserving coefficients are stored in
coeff.
Usage
svmDP$fit(
X,
y,
upper.bounds,
lower.bounds,
add.bias = FALSE,
weights = NULL,
weights.upper.bound = NULL
)
Arguments
XDataframe of data to be fit.
yVector or matrix of true labels for each row of X.
upper.boundsNumeric vector of length ncol(X) giving upper
bounds on the values in each column of X. The ncol(X) values are
assumed to be in the same order as the corresponding columns of X.
Any value in the columns of X larger than the corresponding upper
bound is clipped at the bound.
lower.boundsNumeric vector of length ncol(X) giving lower
bounds on the values in each column of X. The ncol(X)
values are assumed to be in the same order as the corresponding columns
of X. Any value in the columns of X larger than the
corresponding upper bound is clipped at the bound.
add.biasBoolean indicating whether to add a bias term to X.
Defaults to FALSE.
weightsNumeric vector of observation weights of the same length as
y. If not given, no observation weighting is performed.
weights.upper.boundNumeric value representing the global or public
upper bound on the weights. Required if weights are given.
Method XtoV()
Convert input data X into transformed data V. Uses sampled
pre-filter values and a mapping function based on the chosen kernel to
produce D-dimensional data V on which to train the model or predict
future values. This method is only used if the kernel is nonlinear. See
\insertCitechaudhuri2011;textualDPpack for more details.
Usage
svmDP$XtoV(X)
Arguments
XMatrix corresponding to the original dataset.
Returns
Matrix V of size n by D representing the transformed dataset, where
n is the number of rows of X, and D is the provided transformed space
dimension.
Method predict()
Predict label(s) for given X using the fitted
coefficients.
Usage
svmDP$predict(X, add.bias = FALSE, raw.value = FALSE)
Arguments
XDataframe of data on which to make predictions. Must be of same
form as X used to fit coefficients.
add.biasBoolean indicating whether to add a bias term to X.
Defaults to FALSE. If add.bias was set to TRUE when fitting the
coefficients, add.bias should be set to TRUE for predictions.
raw.valueBoolean indicating whether to return the raw predicted
value or the rounded class label. If FALSE (default), outputs the
predicted labels 0 or 1. If TRUE, returns the raw score from the SVM
model.
Returns
Matrix of predicted labels or scores corresponding to each row of
X.
Method clone()
The objects of this class are cloneable with this method.
Usage
svmDP$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
References
\insertRefchaudhuri2011DPpack
\insertRefYang2005DPpack
\insertRefChapelle2007DPpack
\insertRefRahimi2007DPpack
\insertRefRahimi2008DPpack
Examples
# Build train dataset X and y, and test dataset Xtest and ytest
N <- 400
X <- data.frame()
y <- data.frame()
for (i in (1:N)){
Xtemp <- data.frame(x1 = stats::rnorm(1,sd=.28) , x2 = stats::rnorm(1,sd=.28))
if (sum(Xtemp^2)<.15) ytemp <- data.frame(y=0)
else ytemp <- data.frame(y=1)
X <- rbind(X, Xtemp)
y <- rbind(y, ytemp)
}
Xtest <- X[seq(1,N,10),]
ytest <- y[seq(1,N,10),,drop=FALSE]
X <- X[-seq(1,N,10),]
y <- y[-seq(1,N,10),,drop=FALSE]
# Construct object for SVM
regularizer <- 'l2' # Alternatively, function(coeff) coeff%*%coeff/2
eps <- 1
gamma <- 1
perturbation.method <- 'output'
kernel <- 'Gaussian'
D <- 20
svmdp <- svmDP$new(regularizer, eps, gamma, perturbation.method,
kernel=kernel, D=D)
# Fit with data
# Bounds for X based on construction
upper.bounds <- c( 1, 1)
lower.bounds <- c(-1,-1)
weights <- rep(1, nrow(y)) # Uniform weighting
weights[nrow(y)] <- 0.5 # Half weight for last observation
wub <- 1 # Public upper bound for weights
svmdp$fit(X, y, upper.bounds, lower.bounds, weights=weights,
weights.upper.bound=wub) # No bias term
# Predict new data points
predicted.y <- svmdp$predict(Xtest)
n.errors <- sum(predicted.y!=ytest)
DPpack documentation built on April 8, 2023, 9:09 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| svmDP | R Documentation |
Privacy-preserving Support Vector Machine
Description
This class implements differentially private support vector machine (SVM) \insertCitechaudhuri2011DPpack. It can be either weighted \insertCiteYang2005DPpack or unweighted. Either the output or the objective perturbation method can be used for unweighted SVM, though only the output perturbation method is currently supported for weighted SVM.
Details
To use this class for SVM, first use the new method to
construct an object of this class with the desired function values and
hyperparameters, including a choice of the desired kernel. After
constructing the object, the fit method can be applied to fit the
model with a provided dataset, data bounds, and optional observation
weights and weight upper bound. In fitting, the model stores a vector of
coefficients coeff which satisfy differential privacy. Additionally,
if a nonlinear kernel is chosen, the models stores a mapping function from
the input data X to a higher dimensional embedding V in the form of a
method XtoV as required \insertCitechaudhuri2011DPpack. These
can be released directly, or used in conjunction with the predict
method to privately predict the label of new datapoints. Note that the
mapping function XtoV is based on an approximation method via
Fourier transforms \insertCiteRahimi2007,Rahimi2008DPpack.
Note that in order to guarantee differential privacy for the SVM model, certain constraints must be satisfied for the values used to construct the object, as well as for the data used to fit. These conditions depend on the chosen perturbation method. First, the loss function is assumed to be differentiable (and doubly differentiable if the objective perturbation method is used). The hinge loss, which is typically used for SVM, is not differentiable at 1. Thus, to satisfy this constraint, this class utilizes the Huber loss, a smooth approximation to the hinge loss \insertCiteChapelle2007DPpack. The level of approximation to the hinge loss is determined by a user-specified constant, h, which defaults to 0.5, a typical value. Additionally, the regularizer must be 1-strongly convex and differentiable. It also must be doubly differentiable if objective perturbation is chosen. If weighted SVM is desired, the provided weights must be nonnegative and bounded above by a global or public value, which must also be provided.
Finally, it is assumed that if x represents a single row of the dataset X,
then the l2-norm of x is at most 1 for all x. In order to ensure this
constraint is satisfied, the dataset is preprocessed and scaled, and the
resulting coefficients are postprocessed and un-scaled so that the stored
coefficients correspond to the original data. Due to this constraint on x,
it is best to avoid using a bias term in the model whenever possible. If a
bias term must be used, the issue can be partially circumvented by adding a
constant column to X before fitting the model, which will be scaled along
with the rest of X. The fit method contains functionality to add a
column of constant 1s to X before scaling, if desired.
Super classes
DPpack::EmpiricalRiskMinimizationDP.CMS -> DPpack::WeightedERMDP.CMS -> svmDP
Methods
Public methods
Method new()
Create a new svmDP object.
Usage
svmDP$new( regularizer, eps, gamma, perturbation.method = "objective", kernel = "linear", D = NULL, kernel.param = NULL, regularizer.gr = NULL, huber.h = 0.5 )
Arguments
regularizerString or regularization function. If a string, must be 'l2', indicating to use l2 regularization. If a function, must have form
regularizer(coeff), wherecoeffis a vector or matrix, and return the value of the regularizer atcoeff. Seeregularizer.l2for an example. Additionally, in order to ensure differential privacy, the function must be 1-strongly convex and doubly differentiable.epsPositive real number defining the epsilon privacy budget. If set to Inf, runs algorithm without differential privacy.
gammaNonnegative real number representing the regularization constant.
perturbation.methodString indicating whether to use the 'output' or the 'objective' perturbation methods \insertCitechaudhuri2011DPpack. Defaults to 'objective'.
kernelString indicating which kernel to use for SVM. Must be one of 'linear', 'Gaussian'. If 'linear' (default), linear SVM is used. If 'Gaussian,' uses the sampling function corresponding to the Gaussian (radial) kernel approximation.
DNonnegative integer indicating the dimensionality of the transform space approximating the kernel if a nonlinear kernel is used. Higher values of D provide better kernel approximations at a cost of computational efficiency. This value must be specified if a nonlinear kernel is used.
kernel.paramPositive real number corresponding to the Gaussian kernel parameter. Defaults to 1/p, where p is the number of predictors.
regularizer.grOptional function representing the gradient of the regularization function with respect to
coeffand of the formregularizer.gr(coeff). Should return a vector. Seeregularizer.gr.l2for an example. Ifregularizeris given as a string, this value is ignored. If not given andregularizeris a function, non-gradient based optimization methods are used to compute the coefficient values in fitting the model.huber.hPositive real number indicating the degree to which the Huber loss approximates the hinge loss. Defaults to 0.5 \insertCiteChapelle2007DPpack.
Returns
A new svmDP object.
Method fit()
Fit the differentially private SVM model. This method runs
either the output perturbation or the objective perturbation algorithm
\insertCitechaudhuri2011DPpack, depending on the value of
perturbation.method used to construct the object, to generate an
objective function. A numerical optimization method is then run to find
optimal coefficients for fitting the model given the training data,
weights, and hyperparameters. The built-in optim function
using the "BFGS" optimization method is used. If regularizer is
given as 'l2' or if regularizer.gr is given in the construction of
the object, the gradient of the objective function is utilized by
optim as well. Otherwise, non-gradient based optimization methods
are used. The resulting privacy-preserving coefficients are stored in
coeff.
Usage
svmDP$fit( X, y, upper.bounds, lower.bounds, add.bias = FALSE, weights = NULL, weights.upper.bound = NULL )
Arguments
XDataframe of data to be fit.
yVector or matrix of true labels for each row of
X.upper.boundsNumeric vector of length
ncol(X)giving upper bounds on the values in each column of X. Thencol(X)values are assumed to be in the same order as the corresponding columns ofX. Any value in the columns ofXlarger than the corresponding upper bound is clipped at the bound.lower.boundsNumeric vector of length
ncol(X)giving lower bounds on the values in each column ofX. Thencol(X)values are assumed to be in the same order as the corresponding columns ofX. Any value in the columns ofXlarger than the corresponding upper bound is clipped at the bound.add.biasBoolean indicating whether to add a bias term to
X. Defaults to FALSE.weightsNumeric vector of observation weights of the same length as
y. If not given, no observation weighting is performed.weights.upper.boundNumeric value representing the global or public upper bound on the weights. Required if weights are given.
Method XtoV()
Convert input data X into transformed data V. Uses sampled pre-filter values and a mapping function based on the chosen kernel to produce D-dimensional data V on which to train the model or predict future values. This method is only used if the kernel is nonlinear. See \insertCitechaudhuri2011;textualDPpack for more details.
Usage
svmDP$XtoV(X)
Arguments
XMatrix corresponding to the original dataset.
Returns
Matrix V of size n by D representing the transformed dataset, where n is the number of rows of X, and D is the provided transformed space dimension.
Method predict()
Predict label(s) for given X using the fitted
coefficients.
Usage
svmDP$predict(X, add.bias = FALSE, raw.value = FALSE)
Arguments
XDataframe of data on which to make predictions. Must be of same form as
Xused to fit coefficients.add.biasBoolean indicating whether to add a bias term to
X. Defaults to FALSE. If add.bias was set to TRUE when fitting the coefficients, add.bias should be set to TRUE for predictions.raw.valueBoolean indicating whether to return the raw predicted value or the rounded class label. If FALSE (default), outputs the predicted labels 0 or 1. If TRUE, returns the raw score from the SVM model.
Returns
Matrix of predicted labels or scores corresponding to each row of
X.
Method clone()
The objects of this class are cloneable with this method.
Usage
svmDP$clone(deep = FALSE)
Arguments
deepWhether to make a deep clone.
References
\insertRefchaudhuri2011DPpack
\insertRefYang2005DPpack
\insertRefChapelle2007DPpack
\insertRefRahimi2007DPpack
\insertRefRahimi2008DPpack
Examples
# Build train dataset X and y, and test dataset Xtest and ytest
N <- 400
X <- data.frame()
y <- data.frame()
for (i in (1:N)){
Xtemp <- data.frame(x1 = stats::rnorm(1,sd=.28) , x2 = stats::rnorm(1,sd=.28))
if (sum(Xtemp^2)<.15) ytemp <- data.frame(y=0)
else ytemp <- data.frame(y=1)
X <- rbind(X, Xtemp)
y <- rbind(y, ytemp)
}
Xtest <- X[seq(1,N,10),]
ytest <- y[seq(1,N,10),,drop=FALSE]
X <- X[-seq(1,N,10),]
y <- y[-seq(1,N,10),,drop=FALSE]
# Construct object for SVM
regularizer <- 'l2' # Alternatively, function(coeff) coeff%*%coeff/2
eps <- 1
gamma <- 1
perturbation.method <- 'output'
kernel <- 'Gaussian'
D <- 20
svmdp <- svmDP$new(regularizer, eps, gamma, perturbation.method,
kernel=kernel, D=D)
# Fit with data
# Bounds for X based on construction
upper.bounds <- c( 1, 1)
lower.bounds <- c(-1,-1)
weights <- rep(1, nrow(y)) # Uniform weighting
weights[nrow(y)] <- 0.5 # Half weight for last observation
wub <- 1 # Public upper bound for weights
svmdp$fit(X, y, upper.bounds, lower.bounds, weights=weights,
weights.upper.bound=wub) # No bias term
# Predict new data points
predicted.y <- svmdp$predict(Xtest)
n.errors <- sum(predicted.y!=ytest)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.