mutualInformation: Calculates Mutual Information for a two-way table.

In ralmond/CPTtools: Tools for Creating Conditional Probability Tables

Calculates Mutual Information for a two-way table.

Description

Calculates the mutual information for a two-way table of observed counts or a joint probability distribution. The mutual information is a measure of association between two random variables.

Usage

mutualInformation(table)

Arguments

table

A two way table or probability distribution. Possibly the output of the table command.

Details

This is basically the Kullback-Leibler distance between the joint probability distribution and the probability distribution created by assuming the marginal distributions are independent. This is given in the following formula:

I[X;Y] = \sum_{x}\sum_{y} \Pr(X=x, Y=y) \log \frac{\Pr(X=x, Y=y)}{\Pr(X=x)\Pr(Y=y)}

Note

For square matrixes, the maximum mutual information, which should should correspond to a diagnoal matrix, is log(d) where d is the number of dimensions.

Author(s)

Russell Almond

References

https://en.wikipedia.org/wiki/Mutual_information

Shannon (1948) “A Mathematical Theory of Communication.”

See Also

table, CPF, ewoe.CPF, expTable

Examples

## UCBAdmissions is a three way table, so we need to
## make it a two way table.
mutualInformation(apply(UCBAdmissions,c(1,2),sum))
apply(UCBAdmissions,3,mutualInformation)
apply(UCBAdmissions,2,mutualInformation)
apply(UCBAdmissions,1,mutualInformation)

print(c(mutualInformation(matrix(1,2,2)),0))
print(c(mutualInformation(diag(2)),
        mutualInformation(1-diag(2)), log(2)))
print(c(mutualInformation(diag(3)),log(3)))

ralmond/CPTtools documentation built on Dec. 27, 2024, 7:15 a.m.