bandit2arm_delta: Two-Arm Bandit Task

Description Usage Arguments Details Value References See Also Examples

View source: R/bandit2arm_delta.R


Hierarchical Bayesian Modeling of the Two-Arm Bandit Task (e.g., Erev et al., 2010; Hertwig et al., 2004) using the following parameters: "A" (learning rate), "tau" (inverse temperature).

MODEL: Rescorla-Wagner (delta) model


bandit2arm_delta(data = "choose", niter = 3000, nwarmup = 1000,
  nchain = 4, ncore = 1, nthin = 1, inits = "random",
  indPars = "mean", saveDir = NULL, email = NULL,
  modelRegressor = FALSE, adapt_delta = 0.95, stepsize = 1,
  max_treedepth = 10)



A .txt file containing the data to be modeled. Data columns should be labelled as follows: "subjID", "choice", and "outcome". See Details below for more information.


Number of iterations, including warm-up.


Number of iterations used for warm-up only.


Number of chains to be run.


Integer value specifying how many CPUs to run the MCMC sampling on. Defaults to 1.


Every i == nthin sample will be used to generate the posterior distribution. Defaults to 1. A higher number can be used when auto-correlation within the MCMC sampling is high.


Character value specifying how the initial values should be generated. Options are "fixed" or "random" or your own initial values.


Character value specifying how to summarize individual parameters. Current options are: "mean", "median", or "mode".


Path to directory where .RData file of model output (modelData) can be saved. Leave blank if not interested.


Character value containing email address to send notification of completion. Leave blank if not interested.


Exporting model-based regressors? TRUE or FALSE. Currently not available for this model.


Floating point number representing the target acceptance probability of a new sample in the MCMC chain. Must be between 0 and 1. See Details below.


Integer value specifying the size of each leapfrog step that the MCMC sampler can take on each new iteration. See Details below.


Integer value specifying how many leapfrog steps that the MCMC sampler can take on each new iteration. See Details below.


This section describes some of the function arguments in greater detail.

data should be assigned a character value specifying the full path and name of the file, including the file extension (e.g. ".txt"), that contains the behavioral data of all subjects of interest for the current analysis. The file should be a tab-delimited text (.txt) file whose rows represent trial-by-trial observations and columns represent variables. For the Two-Arm Bandit Task, there should be three columns of data with the labels "subjID", "choice", and "outcome". It is not necessary for the columns to be in this particular order, however it is necessary that they be labelled correctly and contain the information below:


Should contain a unique identifier for each subject within data-set to be analyzed.


Should contain a integer value representing the chosen choice option within the given trial (e.g., 1 or 2 in 2-arm bandit task).


Should contain outcomes within each given trial (e.g., 1 = reward, -1 = loss).

*Note: The data.txt file may contain other columns of data (e.g. "Reaction_Time", "trial_number", etc.), but only the data with the column names listed above will be used for analysis/modeling. As long as the columns above are present and labelled correctly, there is no need to remove other miscellaneous data columns.

nwarmup is a numerical value that specifies how many MCMC samples should not be stored upon the beginning of each chain. For those familiar with Bayesian methods, this value is equivalent to a burn-in sample. Due to the nature of MCMC sampling, initial values (where the sampling chain begins) can have a heavy influence on the generated posterior distributions. The nwarmup argument can be set to a high number in order to curb the effects that initial values have on the resulting posteriors.

nchain is a numerical value that specifies how many chains (i.e. independent sampling sequences) should be used to draw samples from the posterior distribution. Since the posteriors are generated from a sampling process, it is good practice to run multiple chains to ensure that a representative posterior is attained. When sampling is completed, the multiple chains may be checked for convergence with the plot(myModel, type = "trace") command. The chains should resemble a "furry caterpillar".

nthin is a numerical value that specifies the "skipping" behavior of the MCMC samples being chosen to generate the posterior distributions. By default, nthin is equal to 1, hence every sample is used to generate the posterior.

Contol Parameters: adapt_delta, stepsize, and max_treedepth are advanced options that give the user more control over Stan's MCMC sampler. The Stan creators recommend that only advanced users change the default values, as alterations can profoundly change the sampler's behavior. Refer to Hoffman & Gelman (2014, Journal of Machine Learning Research) for more information on the functioning of the sampler control parameters. One can also refer to section 58.2 of the Stan User's Manual for a less technical description of these arguments.


modelData A class "hBayesDM" object with the following components:


Character string with the name of the model ("bandit2arm_delta").


"data.frame" containing the summarized parameter values (as specified by "indPars") for each subject.


A "list" where each element contains posterior samples over different model parameters.


A class "stanfit" object containing the fitted model.


"data.frame" containing the raw data used to fit the model, as specified by the user.


Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S. M., Hau, R., et al. (2010). A choice prediction competition: Choices from experience and from description. Journal of Behavioral Decision Making, 23(1), 15-47.

Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions From Experience and the Effect of Rare Events in Risky Choice. Psychological Science, 15(8), 534-539.

Hoffman, M. D., & Gelman, A. (2014). The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. The Journal of Machine Learning Research, 15(1), 1593-1623.

See Also

We refer users to our in-depth tutorial for an example of using hBayesDM:


## Not run: 
# Run the model and store results in "output"
output <- bandit2arm_delta(data = "example", niter = 2000, nwarmup = 1000, nchain = 3, ncore = 3)

# Visually check convergence of the sampling chains (should like like 'hairy caterpillars')
plot(output, type = 'trace')

# Check Rhat values (all Rhat values should be less than or equal to 1.1)

# Plot the posterior distributions of the hyper-parameters (distributions should be unimodal)

# Show the WAIC and LOOIC model fit estimates 

## End(Not run)

hBayesDM documentation built on May 24, 2017, 1:01 a.m.

Search within the hBayesDM package
Search all R packages, documentation and source code