# entropy: Estimating Entropy From Observed Counts In entropy: Estimation of Entropy, Mutual Information and Related Quantities

## Description

`entropy` estimates the Shannon entropy H of the random variable Y from the corresponding observed counts `y`.

`freqs` estimates bin frequencies from the counts `y`.

## Usage

 ```1 2 3 4``` ```entropy(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG", "minimax", "CS", "NSB", "shrink"), unit=c("log", "log2", "log10"), verbose=TRUE, ...) freqs(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG", "minimax", "CS", "NSB", "shrink"), verbose=TRUE) ```

## Arguments

 `y` vector of counts. `method` the method employed to estimate entropy (see Details). `unit` the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set `unit="log2"`. `lambda.freqs` shrinkage intensity (for "shrink" option). `verbose` verbose option (for "shrink" option). `...` option passed on to `entropy.NSB`.

## Details

The `entropy` function allows to estimate entropy from observed counts by a variety of methods:

• `method="ML"`:maximum likelihood, see `entropy.empirical`

• `method="MM"`:bias-corrected maximum likelihood, see `entropy.MillerMadow`

• `method="Jeffreys"`:`entropy.Dirichlet` with `a=1/2`

• `method="Laplace"`:`entropy.Dirichlet` with `a=1`

• `method="SG"`:`entropy.Dirichlet` with `a=a=1/length(y)`

• `method="minimax"`:`entropy.Dirichlet` with `a=sqrt(sum(y))/length(y`

• `method="CS"`:see `entropy.ChaoShen`

• `method="NSB"`:see `entropy.NSB`

• `method="shrink"`:see `entropy.shrink`

The `freqs` function estimates the underlying bin frequencies. Note that estimated frequencies are not available for `method="MM"`, `method="CS"` and `method="NSB"`. In these instances a vector containing NAs is returned.

## Value

`entropy` returns an estimate of the Shannon entropy.

`freqs` returns a vector with estimated bin frequencies (if available).

## Author(s)

Korbinian Strimmer (http://strimmerlab.org).

`entropy-package`, `discretize`.

## Examples

 ``` 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15``` ```# load entropy library library("entropy") # observed counts for each bin y = c(4, 2, 3, 0, 2, 4, 0, 0, 2, 1, 1) entropy(y, method="ML") entropy(y, method="MM") entropy(y, method="Jeffreys") entropy(y, method="Laplace") entropy(y, method="SG") entropy(y, method="minimax") entropy(y, method="CS") #entropy(y, method="NSB") entropy(y, method="shrink") ```

### Example output

```[1] 1.968382
[1] 2.152593
[1] 2.17948
[1] 2.257876
[1] 2.036888
[1] 2.154091
[1] 2.201137
Estimating optimal shrinkage intensity lambda.freq (frequencies): 0.7664
[1] 2.379603
attr(,"lambda.freqs")
[1] 0.7663934
```

entropy documentation built on May 29, 2017, 11:20 p.m.