kullback: Calculates the Kullback-Leibler divergence

In nvihrs14/tcherry: Learning the structure of tcherry trees

Description

Calculates the Kullback-Leibler divergence for two discrete probability distributions over the same universe.

Usage

kullback(target_p, approx_p, log_base = 2)

Arguments

target_p	Named array with the target distribution.
approx_p	Named array with the approximate distribution.
log_base	The base used for the logarithm.

Details

The Kullback-Leibler divergence between two probability distributions p and q is calculated by the formula

D(p||q) = ∑ p(x)log(p(x)/q(x))

where the sum is over all possible states of x.

Value

The Kullback-Leibler divergence.

Author(s)

Katrine Kirkeby, [e-mail]

Maria Knudsen, [e-mail]

Ninna Vihrs, [e-mail]

References

\insertRef

TCJTtcherry

Examples

p_target <- array(c(0.1, 0.2, 0.05, 0.4, 0.005, 0.015, 0.13, 0.1),
                  dim = c(2, 2, 2),
                  dimnames = list("V1" = c("a", "b"),
                                  "V2" = c("a", "b"),
                                  "V3" = c("a", "b")))
                                  
p_approx <- array(c(0.01, 0.23, 0.06, 0.4, 0.005, 0.018,
                    0.137, 0.14),
                  dim = c(2, 2, 2),
                  dimnames = list("V1" = c("a", "b"),
                                  "V2" = c("a", "b"),
                                  "V3" = c("a", "b")))
                                  
kullback(p_target, p_approx)

nvihrs14/tcherry documentation built on Aug. 1, 2020, 6:25 p.m.