nmab_gi: Calculate the Gittins index for a single arm (normal rewards)

In jedwards24/gittins: Calculate Gittins Indices

Calculate the Gittins index for a single arm (normal rewards)

Description

The problem state is given by the Sigma, n, gamma, and tau arguments. The remaining arguments affect how the calculation is done and are chosen based on accuracy and speed requirements (see details).

Usage

nmab_gi(Sigma, n, gamma, tau, N, xi, delta, tol = 5e-04, lb = NA, ub = NA)

Arguments

Sigma	Numeric. Value of Sigma for the arm (Bayesian reward).
n	Numeric > 0. Value of n for the arm (Bayesian number of observation).
gamma	Numeric in (0, 1). Reward discount factor.
tau	Numeric > 0. Observation precision.
N	Integer >= 2. Time horizon used.
xi	Numeric > 0. Value of xi (entent of dynamic program state space).
delta	Numeric > 0. Value of delta (fineness of discretisation in the dynamic program).
tol	Numeric > 0. Absolute accuracy required.
lb	Optional lower bound for GI.
ub	Optional upper bound for GI.

Details

The problem has an infinite continuous state space, but the calculation can only be done for a finite discrete area. N and xi control the finite extent of the state space used (one in each direction). These have a diminishing effect on accuracy and just need to be sufficiently large, beyond which there will be minimal improvement. Each parameter has a linear effect on the state space size and therefore algorithm speed. Larger gamma or smaller tau requires a larger N, but depends only on the least favourable of the two. Often N can be smaller than for similar BMAB problems. The required xi is more robust to changes in the problem setting as adjusts to tau somewhat. Therefore, standard values can be used across problems with less disadvantage. xi is based on standard deviations so 3 or 4 will often be large enough. Using N or xi values that are too low gives an underestimate of the Gittins index.

The discretisation parameter delta is the main limiter to accuracy as it has a non-linear effect on computation time. Accuracy increases as delta gets smaller, with inaccuracies leading to an overestimate of the Gittins index.

The lb and ub arguments can be used to provide a starting interval for calibration if desired. However, for normal use this is not needed as they will be calculated internally if not supplied. So the initial interval is determined as follows:

• For lower bound, use lb if supplied else use KGI.

• For upper bound, use ub if supplied else use GI+.

Value

A vector of GI values.

See Also

For a link to the accompanying paper see gittins-package.

jedwards24/gittins documentation built on Oct. 13, 2023, 4:17 p.m.