These functions calculate the root-mean-squared-error, the expected log likelihood, and Kullback-Leibler (KL) divergence (a.k.a. distance), between two multivariate normal (MVN) distributions described by their mean vector and covariance matrix
1 2 3
mean vector of first (estimated) MVN
covariance matrix of first (estimated) MVN
mean vector of second (true, baseline, or comparator) MVN
covariance matrix of second (true, baseline, or comparator) MVN
The root-mean-squared-error is calculated between the entries of the mean vectors, and the upper-triangular part of the covariance matrices (including the diagonal).
The KL divergence is given by the formula:
0.5 (log(|S1|/|S2|) + tr(inv(S1)S2) + t(mu1-m2)inv(S2)(mu1-mu2) - N)
where N is
length(mu1), and must agree with
the dimensions of the other parameters. Note that the parameterization
used involves swapped arguments compared to some other references,
e.g., as provided by Wikipedia. See note below.
The expected log likelihood can be formulated in terms of the
KL divergence. That is, the expected log likelihood of data
simulated from the normal distribution with parameters
S2 under the estimated normal with parameters
S1 is given by
-0.5 ln((2 pi e)^N |S2|) - kl.norm(mu1, S1, mu2, S2).
In the case of the expected log likelihood the result is a real number. The RMSE is a positive real number. The KL divergence method returns a positive real number depicting the distance between the two normal distributions
The KL-divergence is not symmetric. Therefore
kl.norm(mu1,S1,mu2,S2) != kl.norm(mu2,S2,mu1,S1).
But a symmetric metric can be constructed from
0.5 * (kl.norm(mu1,S1,mu2,S2) + kl.norm(mu2,S2,mu1,S1))
or by using
symm = TRUE. The arguments are reversed
compared to some other references, like Wikipedia. To match
those versions use
kl.norm(mu2, S2, mu1, s1)
Robert B. Gramacy email@example.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
mu1 <- rnorm(5) s1 <- matrix(rnorm(100), ncol=5) S1 <- t(s1) %*% s1 mu2 <- rnorm(5) s2 <- matrix(rnorm(100), ncol=5) S2 <- t(s2) %*% s2 ## RMSE rmse.muS(mu1, S1, mu2, S2) ## expected log likelihood Ellik.norm(mu1, S1, mu2, S2) ## KL is not symmetric kl.norm(mu1, S1, mu2, S2) kl.norm(mu2, S2, mu1, S1) ## symmetric version kl.norm(mu2, S2, mu1, S1, symm=TRUE)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.