bmab_gi: Calculate the Gittins index for a single arm (Bernoulli...

In jedwards24/gittins: Calculate Gittins Indices

Calculate the Gittins index for a single arm (Bernoulli rewards)

Description

The two versions of this function differ only in their state parameters:

• bmab_gi() uses Sigma and n parameters.

• bmab_gi_ab() uses alpha and beta parameters

These are related by alpha = Sigma (Bayesian number of successes) and beta = n - Sigma (Bayesian number of failures). Then n is the Bayesian number of observations. Together with gamma (the discount factor for rewards), these define the problem. The remaining arguments are settings for the calculation only (see details).

Usage

bmab_gi(Sigma, n, gamma, N, tol = 5e-04, lb = NA, ub = NA)

bmab_gi_ab(alpha, beta, gamma, N, tol = 5e-04, lb = NA, ub = NA)

Arguments

Sigma	Numeric > 0. Value of Sigma for the arm.
n	Numeric > Sigma > 0. Value of n for the arm.
gamma	Numeric in (0, 1). Reward discount factor.
N	Integer >= 2. Time horizon used in the calculation..
tol	Numeric > 0. Absolute accuracy required.
lb	Optional lower bound for GI.
ub	Optional upper bound for GI.
alpha	Numeric > 0.Value of alpha for the arm.
beta	Numeric > 0. Value of beta for the arm.

Details

The problem has an infinite time horizon, but the dynamic program used to calculate the GI has a finite horizon N. For sufficiently large N, the calculation can be arbitrarily accurate. In practice, a fairly low value of N works well unless gamma is close to 1.

The lb and ub arguments can be used to provide a starting interval for calibration if desired. However, for normal use this is not needed as they will be calculated internally if not supplied. So the initial interval is determined as follows:

• For lower bound, use lb if supplied else use KGI.

• For upper bound, use ub if supplied else use GI+.

Value

A single Gittins index

See Also

For a link to the accompanying paper see gittins-package.

jedwards24/gittins documentation built on Oct. 13, 2023, 4:17 p.m.