vi_symmetrized_csiszar_function: Symmetrizes a Csiszar-function in log-space

In tfprobability: Interface to 'TensorFlow Probability'

Symmetrizes a Csiszar-function in log-space

Description

A Csiszar-function is a member of F = { f:R_+ to R : f convex }.

Usage

vi_symmetrized_csiszar_function(logu, csiszar_function, name = NULL)

Arguments

logu	float-like Tensor representing log(u) from above.
csiszar_function	function representing a Csiszar-function over log-domain.
name	name prefixed to Ops created by this function.

Details

The symmetrized Csiszar-function is defined as:

f_g(u) = 0.5 g(u) + 0.5 u g (1 / u)

where g is some other Csiszar-function. We say the function is "symmetrized" because:

D_{f_g}[p, q] = D_{f_g}[q, p]

for all p << >> q (i.e., support(p) = support(q)).

There exists alternatives for symmetrizing a Csiszar-function. For example,

f_g(u) = max(f(u), f^*(u)),

where f^* is the dual Csiszar-function, also implies a symmetric f-Divergence.

Example: When either of the following functions are symmetrized, we obtain the Jensen-Shannon Csiszar-function, i.e.,

g(u) = -log(u) - (1 + u) log((1 + u) / 2) + u - 1
h(u) = log(4) + 2 u log(u / (1 + u))

implies,

f_g(u) = f_h(u) = u log(u) - (1 + u) log((1 + u) / 2)
= jensen_shannon(log(u)).

Warning: this function makes non-log-space calculations and may therefore be numerically unstable for |logu| >> 0.

Value

symmetrized_g_of_u: float-like Tensor of the result of applying the symmetrization of g evaluated at u = exp(logu).

See Also

Other vi-functions: vi_amari_alpha(), vi_arithmetic_geometric(), vi_chi_square(), vi_csiszar_vimco(), vi_dual_csiszar_function(), vi_fit_surrogate_posterior(), vi_jeffreys(), vi_jensen_shannon(), vi_kl_forward(), vi_kl_reverse(), vi_log1p_abs(), vi_modified_gan(), vi_monte_carlo_variational_loss(), vi_pearson(), vi_squared_hellinger()

tfprobability documentation built on Sept. 1, 2022, 5:07 p.m.