backwardHiddenStateUpdate: Backward hidden states update

In mgoulet847/tagi: Tractable Approximate Gaussian Inference in Neural Networks

Description

This function updates hidden units from responses to input data. It updates μ_{Z|y} and Σ_{Z|y} from the Z|y distribution for a given layer.

Usage

backwardHiddenStateUpdate(mz, Sz, mzF, SzF, SzB, Czz, mzB, idx)

Arguments

mz	Mean vector of units for the current layer μ_{Z}
Sz	Covariance matrix of units for the current layer Σ_{Z}
mzF	Mean vector of units for the next layer μ_{Z^{+}}
SzF	Covariance matrix of units for the next layer Σ_{Z^{+}}
SzB	Covariance matrix of units for the next layer given y Σ_{Z^{+}|y}
Czz	Covariance matrix between units of previous and current layers Σ_{ZZ^{+}}
mzB	Mean vector of units for the next layer given y μ_{Z^{+}|y}
idx	Indices for the hidden state update step of the current layer

Details

f(\boldsymbol{z}|\boldsymbol{y}) = \mathcal{N}(\boldsymbol{z};\boldsymbol{μ_{Z|y}},\boldsymbol{Σ_{Z|y}}) where

\boldsymbol{μ_{Z|y}} = \boldsymbol{μ_{Z}} + \boldsymbol{J_{Z}}(\boldsymbol{μ_{Z^{+}|y}} - \boldsymbol{μ_{Z^{+}}})

\boldsymbol{Σ_{Z|y}} = \boldsymbol{Σ_{Z}} + \boldsymbol{J_{Z}}(\boldsymbol{Σ_{Z^{+}|y}} - \boldsymbol{Σ_{Z^{+}}})\boldsymbol{J_{Z}^{T}}

\boldsymbol{J_{Z}} = \boldsymbol{Σ_{Z Z^{+}}}\boldsymbol{Σ^{-1}_{Z^{+}}}

Value

• Mean vector of units for the current layer given y μ_{Z|y}

• Covariance matrix of units for the current layer given y Σ_{Z|y}

mgoulet847/tagi documentation built on Dec. 21, 2021, 5:10 p.m.