Distributions Provided by the **mniw** Package

In mniw: The Matrix-Normal Inverse-Wishart Distribution

\newcommand{\bm}[1]{\boldsymbol{#1}} \newcommand{\tx}[1]{\textrm{#1}} \newcommand{\tfrac}[2]{\textstyle{\frac{#1}{#2}}} \newcommand{\rv}[3]{#2_{#1},\ldots,#2_{#3}} \newcommand{\iid}{\overset{\;\tx{iid}\;}{\sim}} \newcommand{\ind}{\overset{\:\tx{ind}\:}{\sim}} \newcommand{\XX}{{\bm{X}}} \newcommand{\YY}{{\bm{Y}}} \newcommand{\LL}{{\bm{L}}} \newcommand{\PPs}{{\bm{\Psi}}} \newcommand{\ZZ}{{\bm{Z}}} \newcommand{\N}{\mathcal{N}} \newcommand{\LLa}{{\bm{\Lambda}}} \newcommand{\SSi}{{\bm{\Sigma}}} \newcommand{\GGa}{{\bm{\Gamma}}} \newcommand{\UU}{{\bm{U}}} \newcommand{\VV}{{\bm{V}}} \newcommand{\aa}{{\bm{a}}} \newcommand{\bb}{{\bm{b}}} \newcommand{\xx}{{\bm{x}}} \newcommand{\yy}{{\bm{y}}} \newcommand{\bbe}{{\bm{\beta}}} \newcommand{\mmu}{{\bm{\mu}}} \newcommand{\kka}{{\bm{\kappa}}} \newcommand{\lla}{{\bm{\lambda}}} \newcommand{\TTh}{{\bm{\Theta}}} \newcommand{\GG}{{\bm{G}}} \newcommand{\OOm}{{\bm{\Omega}}} \newcommand{\bz}{{\bm{0}}} \newcommand{\wish}{\tx{Wish}} \newcommand{\iwish}{\tx{InvWish}} \newcommand{\MN}{\tx{MatNorm}} \newcommand{\MT}{\tx{MatT}} \newcommand{\mniw}{\tx{MNIW}} \newcommand{\re}{\tx{RxNorm}}

Wishart Distribution

The Wishart distribution on a random positive-definite matrix $\XX_{q\times q}$ is is denoted $\XX \sim \wish(\PPs, \nu)$, and defined as $\XX = (\LL \ZZ)(\LL \ZZ)'$, where:

• $\PPs_{q\times q} = \LL\LL'$ is the positive-definite matrix scale parameter,
• $\nu > q$ is the shape parameter,

• $\ZZ_{q\times q}$ is a random lower-triangular matrix with elements

  [ Z_{ij} \begin{cases} \iid \N(0,1) & i < j \ \ind \chi^2_{(\nu-i+1)} & i = j \ = 0 & i > j. \end{cases} ]

The log-density of the Wishart distribution is [ \log p(\XX \mid \PPs, \nu) = -\tfrac{1}{2} \left[\mathrm{tr}(\PPs^{-1} \XX) + (q+1-\nu)\log |\XX| + \nu \log |\PPs| + \nu q \log(2) + 2 \log \Gamma_q(\tfrac \nu 2)\right], ] where $\Gamma_n(x)$ is the multivariate Gamma function defined as [ \Gamma_n(x) = \pi^{n(n-1)/4} \prod_{j=1}^n \Gamma\big(x + \tfrac 1 2 (1-j)\big). ]

Inverse-Wishart Distribution

The Inverse-Wishart distribution $\XX \sim \iwish(\PPs, \nu)$ is defined as $\XX^{-1} \sim \wish(\PPs^{-1}, \nu)$. Its log-density is given by [ \log p(\XX \mid \PPs, \nu) = -\tfrac 1 2 \left[\mathrm{tr}(\PPs \XX^{-1}) + (\nu+q+1) \log |\XX| - \nu \log |\PPs| + \nu q \log(2) + 2 \log \Gamma_q(\tfrac \nu 2)\right]. ]

Properties

If $\XX_{q\times q} \sim \wish(\PPs,\nu)$, the for a nonzero vector $\aa \in \mathbb R^q$ we have [ \frac{\aa'\XX\aa}{\aa'\PPs\aa} \sim \chi^2_{(\nu)}. ]

Matrix-Normal Distribution

The Matrix-Normal distribution on a random matrix $\XX_{p \times q}$ is denoted $\XX \sim \MN(\LLa, \SSi_R, \SSi_C)$, and defined as $\XX = \LL\ZZ \UU + \LLa$, where:

• $\LLa_{p \times q}$ is the mean matrix parameter,
• ${\SSi_R}_{p \times p} = \LL\LL'$ is the row-variance matrix parameter,
• ${\SSi_C}_{q \times q} = \UU'\UU$ is the column-variance matrix parameter,
• $\ZZ_{q\times q}$ is a random matrix with $Z_{ij} \iid \N(0,1)$.

The log-density of the Matrix-Normal distribution is [ \log p(\XX \mid \LLa, \SSi_R, \SSi_C) = -\tfrac 1 2 \left[\mathrm{tr}\big(\SSi_C^{-1}(\XX-\LLa)'\SSi_R^{-1}(\XX-\LLa)\big) + \nu q \log(2\pi) + \nu \log |\SSi_C| + q \log |\SSi_R|\right]. ]

Properties

If $\XX_{p \times q} \sim \MN(\LLa, \SSi_R, \SSi_C)$, then for nonzero vectors $\aa \in \mathbb R^p$ and $\bb \in \mathbb R^q$ we have [ \aa' \XX \bb \sim \N(\aa' \LLa \bb, \aa'\SSi_R\aa \cdot \bb'\SSi_C\bb). ]

Matrix-Normal Inverse-Wishart Distribution

The Matrix-Normal Inverse-Wishart Distribution on a random matrix $\XX_{p \times q}$ and random positive-definite matrix $\VV_{q\times q}$ is denoted $(\XX,\VV) \sim \mniw(\LLa, \SSi, \PPs, \nu)$, and defined as [ \begin{aligned} \XX \mid \VV & \sim \MN(\LLa, \SSi, \VV) \ \VV & \sim \iwish(\PPs, \nu). \end{aligned} ]

Properties

The MNIX distribution is conjugate prior for the multivariable response regression model [ \YY_{n \times q} \sim \MN(\XX_{n\times p} \bbe_{p \times q}, \VV, \SSi). ] That is, if $(\bbe, \SSi) \sim \mniw(\LLa, \OOm^{-1}, \PPs, \nu)$, then [ \bbe, \SSi \mid \YY \sim \mniw(\hat \LLa, \hat \OOm^{-1}, \hat \PPs, \hat \nu), ] where [ \begin{aligned} \hat \OOm & = \XX'\VV^{-1}\XX + \OOm & \hat \PPs & = \PPs + \YY'\VV^{-1}\YY + \LLa'\OOm\LLa - \hat \LLa'\hat \OOm \hat \LLa \ \hat \LLa & = \hat \OOm^{-1}(\XX'\VV^{-1}\YY + \OOm\LLa) & \hat \nu & = \nu + n. \end{aligned} ]

Matrix-t Distribution

The Matrix-$t$ distribution on a random matrix $\XX_{p \times q}$ is denoted $\XX \sim \MT(\LLa, \SSi_R, \SSi_C, \nu)$, and defined as the marginal distribution of $\XX$ for $(\XX, \VV) \sim \mniw(\LLa, \SSi_R, \SSi_C, \nu)$. Its log-density is given by [ \begin{aligned} \log p(\XX \mid \LLa, \SSi_R, \SSi_C, \nu) & = -\tfrac 1 2 \Big[(\nu+p+q-1)\log | I + \SSi_R^{-1}(\XX-\LLa)\SSi_C^{-1}(\XX-\LLa)'| \ & \phantom{= -\tfrac 1 2 \Big[} + q \log |\SSi_R| + p \log |\SSi_C| \ & \phantom{= -\tfrac 1 2 \Big[} + pq \log(\pi) - \log \Gamma_q(\tfrac{\nu+p+q-1}{2}) + \log \Gamma_q(\tfrac{\nu+q-1}{2})\Big]. \end{aligned} ]

Properties

If $\XX_{p\times q} \sim \MT(\LLa, \SSi_R, \SSi_C, \nu)$, then for nonzero vectors $\aa \in \mathbb R^p$ and $\bb \in \mathbb R^q$ we have [ \frac{\aa'\XX\bb - \mu}{\sigma} \sim t_{(\nu -q + 1)}, ] where [ \mu = \aa'\LLa\bb, \qquad \sigma^2 = \frac{\aa'\SSi_R\aa \cdot \bb'\SSi_C\bb}{\nu - q + 1}. ]

Random-Effects Normal Distribution

Consider the multivariate normal distribution on $q$-dimensional vectors $\xx$ and $\mmu$ given by [ \begin{aligned} \xx \mid \mmu & \sim \N(\mmu, \VV) \ \mmu & \sim \N(\lla, \SSi). \end{aligned} ] The random-effects normal distribution is defined as the posterior distribution $\mmu \sim p(\mmu \mid \xx)$, which is given by [ \mmu \mid \xx \sim \N\big(\GG(\xx-\lla) + \lla, \GG\VV\big), \qquad \GG = \SSi(\VV + \SSi)^{-1}. ] The notation for this distribution is $\mmu \sim \re(\xx, \VV, \lla, \SSi)$.

Hierarchical Normal-Normal Model

The hierarchical Normal-Normal model is defined as [ \begin{aligned} \yy_i \mid \mmu_i, \bbe, \SSi & \ind \N(\mmu_i, \VV_i) \ \mmu_i \mid \bbe, \SSi & \iid \N(\xx_i'\bbe, \SSi) \ (\bbe, \SSi) & \sim \mniw(\LLa, \Omega^{-1}, \PPs, \nu), \end{aligned} ] where:

• ${\yy_i}_{q\times 1}$ is the response vector for subject $i$,
• ${\mmu_i}_{q\times 1}$ is the random effect for subject $i$,
• ${\VV_i}_{q\times q}$ is the error variance for subject $i$,
• ${\xx_i}_{p\times 1}$ is the covariate vector for subject $i$,
• $\bbe_{p \times q}$ is the random-effects coefficient matrix,
• $\SSi_{q \times q}$ is the random-effects error variance.

Let $\YY_{n\times q} = (\rv 1 \yy n)$, $\XX_{n\times p} = (\rv 1 \xx n)$, and $\TTh_{n \times q} = (\rv 1 \mmu n)$. If interest lies in the posterior distribution $p(\TTh, \bbe, \SSi \mid \YY, \XX)$, then a Gibbs sampler can be used to cycle through the following conditional distributions: [ \begin{aligned} \mmu_i \mid \bbe, \SSi, \YY, \XX & \ind \re(\yy_i, \VV_i, \xx_i'\bbe, \SSi) \ \bbe, \SSi \mid \TTh, \YY, \XX & \sim \mniw(\hat \LLa, \hat \OOm^{-1}, \hat \PPs, \hat \nu), \end{aligned} ] where $\hat \LLa$, $\hat \OOm$, $\hat \PPs$, and $\hat \nu$ are obtained from the MNIW conjugate posterior formula with $\YY \gets \TTh$.

mniw documentation built on Aug. 22, 2022, 5:05 p.m.