condmixt.nll: Negative log-likelihood for conditional mixtures
In condmixt: Conditional Density Estimation with Neural Network Conditional Mixtures

Description Usage Arguments Details Value Author(s) References See Also Examples

Computes negative log-likelihood and gradient for given neural network parameters, numbers of hidden units and of components, the type of components and the presence of a discrete dirac component on a given data set.

condhparetomixt.nll(theta, h, m, x, y)
condhparetomixt.nll.tailpen(theta,h,m,x,y,lambda=0,w=0.2,beta=50,mu=0.2,sigma=0.2)
condhparetomixt.dirac.nll.tailpen(theta,h,m,x,y,lambda=0,w=0.2,beta=50,mu=0.2,sigma=0.2)
condhparetomixt.dirac.nll(theta,h,m,x,y)
condgaussmixt.nll(theta,h,m,x,y)
condgaussmixt.dirac.nll(theta,h,m,x,y)
condlognormixt.nll(theta,h,m,x,y)
condlognormixt.dirac.nll(theta,h,m,x,y)
condbergamixt.nll(theta,h,x,y)

`theta`	Vector of neural network parameters
`h`	Number of hidden units
`m`	Number of components
`x`	Matrix of explanatory (independent) variables of dimension d x n, d is the number of variables and n is the number of examples (patterns)
`y`	Vector of n dependent variables
`lambda`	penalty parameter which controls the trade-off between the penalty and the negative log-likelihood, takes on positive values. If zero, no penalty
`w`	penalty parameter in [0,1] which is the proportion of components with light tails, 1-`w` being the proportion of components with heavy tails
`beta`	positive penalty parameter which indicates the importance of the light tail components (it is the parameter of an exponential which represents the prior over the light tail components)
`mu`	penalty parameter in (0,1) which represents the a priori value for the heavy tail index of the underlying distribution
`sigma`	positive penalty parameter which controls the spread around the a priori value for the heavy tail index of the underlying distribution

condhparetomixt indicates a mixture with hybrid Pareto components, condgaussmixt for Gaussian components, condlognormixt for Log-Normal components, condbergam for a Bernoulli-Gamma two component mixture, tailpen indicates that a penalty is added to the log-likelihood to guide the tail index parameter estimation, dirac indicates that a discrete dirac component at zero is included in the mixture

In order to drive the tail index estimation, a penalty is introduced in the log-likelihood. The goal of the penalty is to include a priori information which in our case is that only a few mixture components have a heavy tail index which should approximate the tail of the underlying distribution while most other mixture components have a light tail and aim at modelling the central part of the underlying distribution.

The penalty term is given by the logarithm of the following two-component mixture, as a function of a tail index parameter xi :

w beta exp(-beta xi) + (1-w) exp(-(xi-mu)^2/(2 sigma^2))/(sqrt(2 pi) sigma)

where the first term is the prior on the light tail component and the second term is the prior on the heavy tail component.

Returns a single value (the negative log-likelihood for given parameters and sample) and a vector, the gradient, which is passed as an attribute.

Julie Carreau

Bishop, C. (1995), Neural Networks for Pattern Recognition, Oxford

Carreau, J. and Bengio, Y. (2009), A Hybrid Pareto Mixture for Conditional Asymmetric Fat-Tailed Distributions, 20, IEEE Transactions on Neural Networks

hparetomixt.negloglike, hparetomixt.negloglike.tailpen, condmixt.init

n <- 200
x <- runif(n) # x is a random uniform variate
# y depends on x through the parameters of the Frechet distribution
y <- rfrechet(n,loc = 3*x+1,scale = 0.5*x+0.001,shape=x+1)
plot(x,y,pch=22)
# 0.99 quantile of the generative distribution
qgen <- qfrechet(0.99,loc = 3*x+1,scale = 0.5*x+0.001,shape=x+1)
points(x,qgen,pch="*",col="orange")

h <- 2 # number of hidden units
m <- 4 # number of components

# initialize a conditional mixture with hybrid Pareto components
thetainit <- condhparetomixt.init(1,h,m,y)

# computes negative log-likelihood and gradient for initial neural network parameters
condhparetomixt.nll(thetainit,h,m,t(x),y)