innovationVector: Last hidden layer states' deltas update

In mgoulet847/tagi: Tractable Approximate Gaussian Inference in Neural Networks

Description

This function updates hidden layer units' deltas using next hidden layer' deltas. It updates μ_{Z|y} and Σ_{Z|y} from the Z|y distribution.

Usage

innovationVector(SzF, dMz, dSz)

Arguments

SzF	Covariance matrix of units for the next layer Σ_{y}
dMz	Delta of mean vector of units for the next hidden layer μ_{Z}
dSz	Delta of covariance matrix of units for the next hidden layer Σ_{Z}

Details

f(\boldsymbol{z}|\boldsymbol{y}) = \mathcal{N}(\boldsymbol{z};\boldsymbol{μ_{Z|y}},\boldsymbol{Σ_{Z^|y}}) where

\boldsymbol{μ_{{Z}|y}} = \boldsymbol{μ_{Z}} + \boldsymbol{J_{Z}}(\boldsymbol{μ_{Z^{+}|y}} - \boldsymbol{μ_{Z^{+}}})

\boldsymbol{Σ_{{Z}|y}} = \boldsymbol{Σ_{Z}} + \boldsymbol{J_{Z}}(\boldsymbol{Σ_{Z^{+}|y}} - \boldsymbol{Σ_{Z^{+}}})\boldsymbol{J_{Z}^{T}}

\boldsymbol{J_{Z}} = \boldsymbol{Σ_{ZZ^{+}}} \boldsymbol{Σ_{Z^{+}}^{-1}}

Value

• Delta of mean vector of current hidden layer units given y μ_{Z}|y

• Delta of covariance matrix of current hidden layer units given y Σ_{Z}|y

mgoulet847/tagi documentation built on Dec. 21, 2021, 5:10 p.m.