bmab_gi_multiple: Calculate Gittins indices for multiple starting states...

In jedwards24/gittins: Calculate Gittins Indices

Calculate Gittins indices for multiple starting states (Bernoulli rewards)

Description

Runs bmab_gi() multiple times for a range of arm starting states. The starting states used are determined by the starting state arguments and num_actions (see details). Other arguments are passed to bmab_gi(). See ?bmab_gi for argument and calculation details.

Usage

bmab_gi_multiple(
  alpha_start = NULL,
  beta_start = NULL,
  num_actions,
  gamma,
  N,
  tol = 5e-04,
  Sigma_start = NULL,
  n_start = NULL
)

Arguments

alpha_start	Lowest value of alpha for the arms.
beta_start	Lowest value of beta for the arms.
num_actions	Determines the number of states GI are calculated for.
gamma	Numeric in (0, 1). Reward discount factor.
N	Integer >= 2. Time horizon used in the calculation..
tol	Numeric > 0. Absolute accuracy required.
Sigma_start	Lowest value of Sigma for the arms.
n_start	Lowest value of n for the arms.

Details

The starting states are either given in alpha/beta or Sigma/n form (strictly one or the other). These are alternative ways of doing the same thing since alpha = Sigma and beta = n - Sigma.

The GI are calculated for arms with the following states:

• alpha = alpha_start : (alpha_start_start + num_actions - 1),

• beta = beta_start : (beta_start + num_actions - 1),

• where alpha + beta <= num_actions + alpha_start + beta_start - 1.

Value

A data frame of starting states and GI values. Extra information is stored in attributes (see examples):

• params: the parameters used.

• gi_matrix: the GI values in a matrix (beta x alpha).

• gi_matrix_ns: the GI values in a matrix (n x Sigma).

Examples

b1 <- bmab_gi_multiple(1, 1, num_actions = 4, gamma = 0.8, N = 20)
b2 <- bmab_gi_multiple(num_actions = 4, gamma = 0.8, N = 20, Sigma_start = 1, n_start = 2)
identical(b1, b2) # TRUE
# View attributes
attr(b1, "params")
attr(b1, "gi_matrix")
attr(b1, "gi_matrix_ns")

jedwards24/gittins documentation built on Oct. 13, 2023, 4:17 p.m.