shannon_relative_entropy: Shannon Relative Entropy

In rinform: An R Wrapper of the 'Inform' C Library for Information Analysis of Complex Systems

Description

Compute the base-b Shannon relative entropy between posterior p and prior q distributions.

Usage

shannon_relative_entropy(p, q, b = 2)

Arguments

p	Dist specifying the posterior distribution.
q	Dist specifying the prior distribution.
b	Numeric giving the base of the logarithm.

Value

Numeric giving the Shannon relative entropy.

Examples

p <- Dist(c(4, 1))
q <- Dist(c(1, 1))
shannon_relative_entropy(p, q)          # 0.2780719
shannon_relative_entropy(q, p)          # 0.3219281
shannon_relative_entropy(p, q, b = 3)   # 0.1754438
shannon_relative_entropy(q, p, b = 3)   # 0.2031140

p <- Dist(c(1, 0))
q <- Dist(c(1, 1))
shannon_relative_entropy(p, q)          # 1.0
shannon_relative_entropy(q, p)          # NaN
shannon_relative_entropy(p, q, b = 3)   # 0.6309298
shannon_relative_entropy(q, p, b = 3)   # NaN

rinform documentation built on April 1, 2018, 12:12 p.m.