# GDHingeAUC: Compute the VB (variational Bayes) approximation of a Gibbs... In PACVB: Variational Bayes (VB) Approximation of Gibbs Posteriors with Hinge Losses

## Description

The function computes a VB approximation of a Gibbs measure with a convexified AUC loss using a gradient descent. The user must specify the design matrix, the response vector and the inverse temperature parameter. The number of iteration of the convex solver is fixed a priori (the default value is 10), the choice can be made using theorem 6.3 in Alquier et al. .

## Usage

 ```1 2``` ```GDHingeAUC(X,Y,lambda,theta=0,K=100,v=10,ls=FALSE, B=0,family="F1",eps=0.05) ```

## Arguments

 `X` Design matrix. The matrix should include a constant column if a bias is to be considered. In addition the Gradient descent has been calibrated considering a centered and scale design matrix. `Y` Response vector. The vector should take values in {-1,1}. `lambda` Inverse temperature of the Gibbs posterior (See Alquier et al. (2015) for guide lines) `theta` Initial value of the gradient descent. In the case of the "F1" family the last value of the vector is the initial log-variance. The vector should be of size p+1, where p is the number of columns of the design matrix. If no initial value are chosen the algorithm is initialized to a Gaussian random vector. `K` Number of iteration of the gradient descent. Default value K=10. An informed choice can be made using theorem 6.3 in Alquier et al. . `v` Prior variance. The prior is taken to be spherical Gaussian, the variance must therefore be specified in the form of a scalar. For default choices see Alquier et al. . The default is arbitrarily set to 10. `ls` Logical value. Indicates if a line search should be used to find an optimal step length. Default value is FALSE. The option is not available for stochastic gradient descent. `B` Batch sizes when considering a stochastic gradient descent. B=0 corresponds to standard gradient descent. `family` Approximate family to consider when implementing VB. Possible values are: "F0" variance is fixed to 1/(sample size) times identity; "F1" spherical Gaussian. (see Alquier et al.  for details) `eps` Probability of the empirical bound of the theoretical risk to be considered. Default is 0.05

## Details

The implementation is based on theorem 6.3 of Alquier et al.  using convex solver presented in Nesterov  (section 3.2.3). The algorithm uses a hinge loss version of AUC and is not explicitly written in Alquier et al. . The calibration depends on an upper bound on the l2 distance between the solution and the initial value. We use an arbitrary value of sqrt (p+1) We also give the possibility to use a line search algorithm satisfying the Wolfe conditions. We also give the possibility to use a line search algorithm satisfying the Wolfe conditions.

## Value

 `m` Mean of the Gaussian approximation `s` Variance of the Gaussian approximation `bound` Empirical bound on the aggregated risk. A negative value indicates that temperaure was taken outside of the admissible intervale.

## Warning

The columns of the design matrix should be centered and scale fo proper behaviour of the algorithm.

James Ridgway

## References

Alquier, P., Ridgway, J., and Chopin, N. On the properties of variational approximations of Gibbs posteriors. arXiv preprint, 2015

Nesterov, Y. Introductory lectures on convex optimization, volume 87. Springer Science and Buisness Media, 2004.

## Examples

 ```1 2 3 4 5 6 7 8``` ```data(Pima.tr) Y<-2*as.matrix(as.numeric(Pima.tr[,8]))-3 X<-data.matrix(Pima.tr[,1:7]) m<-apply(X,2,mean) v<-apply(X,2,sd) X<-t(apply(t(apply(X,1,"-",m)),1,"/",v)) l<-45 Sol<-GDHingeAUC(X,Y,l) ```

PACVB documentation built on Sept. 12, 2016, 8:37 a.m.