mi.plugin: Plug-In Estimator of Mutual Information and of the...
In entropy: Estimation of Entropy, Mutual Information and Related Quantities

Description Usage Arguments Details Value Author(s) See Also Examples

mi.plugin computes the mutual information of two discrete random variables from the specified joint probability mass function.

chi2indep.plugin computes the chi-squared divergence of independence.

1 2	mi.plugin(freqs2d, unit=c("log", "log2", "log10")) chi2indep.plugin(freqs2d, unit=c("log", "log2", "log10"))

`freqs2d`	matrix of joint bin frequencies (joint probability mass function).
`unit`	the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set `unit="log2"`.

The mutual information of two random variables X and Y is the Kullback-Leibler divergence between the joint density/probability mass function and the product independence density of the marginals.

It can also defined using entropy as MI = H(X) + H(Y) - H(X, Y).

Similarly, the chi-squared divergence of independence is the chi-squared divergence between the joint density and the product density. It is a second-order approximation of twice the mutual information.

mi.plugin returns the mutual information.

chi2indep.plugin returns the chi-squared divergence of independence.

Korbinian Strimmer (https://strimmerlab.github.io).

mi.Dirichlet, mi.shrink, mi.empirical, KL.plugin, discretize2d.

# load entropy library 
library("entropy")

# joint distribution of two discrete variables
freqs2d = rbind( c(0.2, 0.1, 0.15), c(0.1, 0.2, 0.25) )  

# corresponding mutual information
mi.plugin(freqs2d)

# MI computed via entropy
H1 = entropy.plugin(rowSums(freqs2d))
H2 = entropy.plugin(colSums(freqs2d))
H12 = entropy.plugin(freqs2d)
H1+H2-H12

# and corresponding (half) chi-squared divergence of independence
0.5*chi2indep.plugin(freqs2d)