# Two-Arm Bandit Task

### Description

Hierarchical Bayesian Modeling of the Two-Arm Bandit Task (e.g., Erev et al., 2010; Hertwig et al., 2004) using the following parameters: "A" (learning rate), "tau" (inverse temperature).

**MODEL:**
Rescorla-Wagner (delta) model

### Usage

1 2 3 4 5 | ```
bandit2arm_delta(data = "choose", niter = 4000, nwarmup = 1000,
nchain = 1, ncore = 1, nthin = 1, inits = "random",
indPars = "mean", saveDir = NULL, email = NULL,
modelRegressor = FALSE, adapt_delta = 0.8, stepsize = 1,
max_treedepth = 10)
``` |

### Arguments

`data` |
A .txt file containing the data to be modeled. Data columns should be labelled as follows: "subjID", "choice", and "outcome". See |

`niter` |
Number of iterations, including warm-up. |

`nwarmup` |
Number of iterations used for warm-up only. |

`nchain` |
Number of chains to be run. |

`ncore` |
Integer value specifying how many CPUs to run the MCMC sampling on. Defaults to 1. |

`nthin` |
Every |

`inits` |
Character value specifying how the initial values should be generated. Options are "fixed" or "random" or your own initial values. |

`indPars` |
Character value specifying how to summarize individual parameters. Current options are: "mean", "median", or "mode". |

`saveDir` |
Path to directory where .RData file of model output ( |

`email` |
Character value containing email address to send notification of completion. Leave blank if not interested. |

`modelRegressor` |
Exporting model-based regressors? TRUE or FALSE. Currently not available for this model. |

`adapt_delta` |
Floating point number representing the target acceptance probability of a new sample in the MCMC chain. Must be between 0 and 1. See |

`stepsize` |
Integer value specifying the size of each leapfrog step that the MCMC sampler can take on each new iteration. See |

`max_treedepth` |
Integer value specifying how many leapfrog steps that the MCMC sampler can take on each new iteration. See |

### Details

This section describes some of the function arguments in greater detail.

**data** should be assigned a character value specifying the full path and name of the file, including the file extension
(e.g. ".txt"), that contains the behavioral data of all subjects of interest for the current analysis.
The file should be a text (.txt) file whose rows represent trial-by-trial observations and columns
represent variables. For the Two-Arm Bandit Task, there should be three columns of data
with the labels "subjID", "choice", and "outcome". It is not necessary for the columns to be in this particular order,
however it is necessary that they be labelled correctly and contain the information below:

`"subjID"`

Should contain a unique identifier for each subject within data-set to be analyzed.

`"choice"`

Should contain a integer value representing the chosen choice option within the given trial (e.g., 1 or 2 in 2-arm bandit task).

`"outcome"`

Should contain outcomes within each given trial (e.g., 1 = reward, -1 = loss).

*****Note: The data.txt file may contain other columns of data (e.g. "Reaction_Time", "trial_number", etc.), but only the data with the column
names listed above will be used for analysis/modeling. As long as the columns above are present and labelled correctly,
there is no need to remove other miscellaneous data columns.

**nwarmup** is a numerical value that specifies how many MCMC samples should not be stored upon the
beginning of each chain. For those familiar with Bayesian methods, this value is equivalent to a burn-in sample.
Due to the nature of MCMC sampling, initial values (where the sampling chain begins) can have a heavy influence
on the generated posterior distributions. The `nwarmup`

argument can be set to a high number in order to curb the
effects that initial values have on the resulting posteriors.

**nchain** is a numerical value that specifies how many chains (i.e. independent sampling sequences) should be
used to draw samples from the posterior distribution. Since the posteriors are generated from a sampling
process, it is good practice to run multiple chains to ensure that a representative posterior is attained. When
sampling is completed, the multiple chains may be checked for convergence with the `plot(myModel, type = "trace")`

command. The chains should resemble a "furry caterpillar".

**nthin** is a numerical value that specifies the "skipping" behavior of the MCMC samples being chosen
to generate the posterior distributions. By default, `nthin`

is equal to 1, hence every sample is used to
generate the posterior.

**Contol Parameters:** adapt_delta, stepsize, and max_treedepth are advanced options that give the user more control
over Stan's MCMC sampler. The Stan creators recommend that only advanced users change the default values, as alterations
can profoundly change the sampler's behavior. Refer to Hoffman & Gelman (2014, Journal of Machine Learning Research) for
more information on the functioning of the sampler control parameters. One can also refer to section 58.2 of the
Stan User's Manual for a less technical description of these arguments.

### Value

`modelData`

A class `"hBayesDM"`

object with the following components:

`model`

Character string with the name of the model (

`"bandit2arm_delta"`

).`allIndPars`

`"data.frame"`

containing the summarized parameter values (as specified by`"indPars"`

) for each subject.`parVals`

A

`"list"`

where each element contains posterior samples over different model parameters.`fit`

A class

`"stanfit"`

object containing the fitted model.`rawdata`

`"data.frame"`

containing the raw data used to fit the model, as specified by the user.

### References

Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S. M., Hau, R., et al. (2010). A choice prediction competition: Choices from experience and from description. Journal of Behavioral Decision Making, 23(1), 15-47. http://doi.org/10.1002/bdm.683

Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions From Experience and the Effect of Rare Events in Risky Choice. Psychological Science, 15(8), 534-539. http://doi.org/10.1111/j.0956-7976.2004.00715.x

Hoffman, M. D., & Gelman, A. (2014). The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. The Journal of Machine Learning Research, 15(1), 1593-1623.

### See Also

We refer users to our in-depth tutorial for an example of using hBayesDM: https://rpubs.com/CCSL/hBayesDM

### Examples

1 2 3 4 5 6 7 8 9 10 11 | ```
## Not run:
# Run the model and store results in "output"
output <- bandit2arm_delta("example", 2000, 1000, 3, 3)
# Plot the posterior distributions of the hyper-parameters
plot(output)
# Show the WAIC and LOOIC model fit estimates
printFit(output)
## End(Not run)
``` |

Want to suggest features or report bugs for rdrr.io? Use the GitHub issue tracker. Vote for new features on Trello.