Estimate the KullbackLeibler divergence of several distributions.
1 2 3 4 
object 
See Methods section below. 
method 
The method to be used; "continuous" determines the KullbackLeibler divergence between the unweighted theoretical component distributions and the unweighted posterior probabilities at the observed points are used by "discrete". 
eps 
Probabilities below this threshold are replaced by this threshold for numerical stability. 
overlap 
Logical, do not determine the KL divergence for
those pairs where for each point at least one of the densities has a
value smaller than 
... 
Passed to the matrix method. 
Estimates
\int f(x) (\log f(x)  \log g(x)) dx
for distributions with densities f() and g().
A matrix of KL divergences where the rows correspond to using the respective distribution as f() in the formula above.
Takes as input a matrix of density values with one row per observation and one column per distribution.
Returns the KullbackLeibler divergence of the mixture components.
The density functions are modified to have equal support.
A weight of at least eps
is given to each
observation point for the modified densities.
Friedrich Leisch and Bettina Gruen
S. Kullback and R. A. Leibler. On information and sufficiency.The Annals of Mathematical Statistics, 22(1), 79–86, 1951.
Friedrich Leisch. Exploring the structure of mixture model components. In Jaromir Antoch, editor, Compstat 2004–Proceedings in Computational Statistics, 1405–1412. Physika Verlag, Heidelberg, Germany, 2004. ISBN 3790815543.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17  ## Gaussian and Student t are much closer to each other than
## to the uniform:
x < seq(3, 3, length = 200)
y < cbind(u = dunif(x), n = dnorm(x), t = dt(x, df = 10))
matplot(x, y, type = "l")
KLdiv(y)
if (require("mlbench")) {
set.seed(2606)
x < mlbench.smiley()$x
model1 < flexmix(x ~ 1, k = 9, model = FLXmclust(diag = FALSE),
control = list(minprior = 0))
plotEll(model1, x)
KLdiv(model1)
}

Questions? Problems? Suggestions? Tweet to @rdrrHQ or email at ian@mutexlabs.com.
All documentation is copyright its authors; we didn't write any of that.