kld: Kullback-Leibler Divergence between Centered Multivariate...

In multvardiv: Multivariate Probability Distributions, Statistical Divergence

Kullback-Leibler Divergence between Centered Multivariate Distributions

Description

Computes the Kullback-Leibler divergence between two random vectors distributed according to centered multivariate distributions:

• multivariate generalized Gaussian distribution (MGGD) with zero mean vector, using the kldggd function

• multivariate Cauchy distribution (MCD) with zero location vector, using the kldcauchy function

• multivariate t distribution (MTD) with zero mean vector, using the kldstudent function

One can also use one of the kldggd, kldcauchy or kldstudent functions, depending on the probability distribution.

Usage

kld(Sigma1, Sigma2, distribution = c("mggd", "mcd", "mtd"),
           beta1 = NULL, beta2 = NULL, nu1 = NULL, nu2 = NULL, eps = 1e-06)

Arguments

Sigma1	symmetric, positive-definite matrix. The scatter matrix of the first distribution.
Sigma2	symmetric, positive-definite matrix. The scatter matrix of the second distribution.
distribution	the probability distribution. It can be "mggd" (multivariate generalized Gaussian distribution) "mcd" (multivariate Cauchy) or "mtd" (multivariate t).
beta1, beta2	numeric. If distribution = "mggd", the shape parameters of the first and second distributions. NULL if distribution is "mcd" or "mtd".
nu1, nu2	numeric. If distribution = "mtd", the degrees of freedom of the first and second distributions. NULL if distribution is "mggd" or "mcd".
eps	numeric. Precision for the computation of the Lauricella D-hypergeometric function if distribution is "mggd" (see kldggd) or of its partial derivative if distribution = "mcd" or distribution = "mtd" (see kldcauchy or kldstudent). Default: 1e-06.

Value

A numeric value: the Kullback-Leibler divergence between the two distributions, with two attributes attr(, "epsilon") (precision of the Lauricella D-hypergeometric function or of its partial derivative) and attr(, "k") (number of iterations).

Author(s)

Pierre Santagostini, Nizar Bouhlel

References

N. Bouhlel, A. Dziri, Kullback-Leibler Divergence Between Multivariate Generalized Gaussian Distributions. IEEE Signal Processing Letters, vol. 26 no. 7, July 2019. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1109/LSP.2019.2915000")}

N. Bouhlel, D. Rousseau, A Generic Formula and Some Special Cases for the Kullback–Leibler Divergence between Central Multivariate Cauchy Distributions. Entropy, 24, 838, July 2022. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.3390/e24060838")}

N. Bouhlel and D. Rousseau (2023), Exact Rényi and Kullback-Leibler Divergences Between Multivariate t-Distributions, IEEE Signal Processing Letters. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1109/LSP.2023.3324594")}

Examples

# Generalized Gaussian distributions
beta1 <- 0.74
beta2 <- 0.55
Sigma1 <- matrix(c(0.8, 0.3, 0.2, 0.3, 0.2, 0.1, 0.2, 0.1, 0.2), nrow = 3)
Sigma2 <- matrix(c(1, 0.3, 0.2, 0.3, 0.5, 0.1, 0.2, 0.1, 0.7), nrow = 3)
# Kullback-Leibler divergence
kl12 <- kld(Sigma1, Sigma2, "mggd", beta1 = beta1, beta2 = beta2)
kl21 <- kld(Sigma2, Sigma1, "mggd", beta1 = beta2, beta2 = beta1)
print(kl12)
print(kl21)
# Distance (symmetrized Kullback-Leibler divergence)
kldist <- as.numeric(kl12) + as.numeric(kl21)
print(kldist)

# Cauchy distributions
Sigma1 <- matrix(c(1, 0.6, 0.2, 0.6, 1, 0.3, 0.2, 0.3, 1), nrow = 3)
Sigma2 <- matrix(c(1, 0.3, 0.1, 0.3, 1, 0.4, 0.1, 0.4, 1), nrow = 3)
kld(Sigma1, Sigma2, "mcd")
kld(Sigma2, Sigma1, "mcd")

Sigma1 <- matrix(c(0.5, 0, 0, 0, 0.4, 0, 0, 0, 0.3), nrow = 3)
Sigma2 <- diag(1, 3)
# Case when all eigenvalues of Sigma1 %*% solve(Sigma2) are < 1
kld(Sigma1, Sigma2, "mcd")
# Case when all eigenvalues of Sigma1 %*% solve(Sigma2) are > 1
kld(Sigma2, Sigma1, "mcd")

# Student distributions
nu1 <- 2
Sigma1 <- matrix(c(2, 1.2, 0.4, 1.2, 2, 0.6, 0.4, 0.6, 2), nrow = 3)
nu2 <- 4
Sigma2 <- matrix(c(1, 0.3, 0.1, 0.3, 1, 0.4, 0.1, 0.4, 1), nrow = 3)
# Kullback-Leibler divergence
kld(Sigma1, Sigma2, "mtd", nu1 = nu1, nu2 = nu2)
kld(Sigma2, Sigma1, "mtd", nu1 = nu2, nu2 = nu1)

multvardiv documentation built on April 3, 2025, 6:08 p.m.