README.md

In psuthaharan/bandit2arm: Modeling Behavior from the Two-Armed Bandit Task

A simple tutorial for modeling behavior from a classic reinforcement learning paradigm using the bandit2arm package

See Suthaharan,P., Corlett, P.R., & Ang, Y.S. (2021). Computational modeling of behavioral tasks: An illustration from a classic reinforcement learning paradigm for more detail.

Questions? Contact Praveen Suthaharan ([e-mail]).

—————

The goal of bandit2arm is to educate users on how to simulate behavioral data from the two-armed bandit task and, subsequently, take the simulated data and estimate the behavioral parameters.

Installation

You can install the released version of bandit2arm from CRAN with:

install.packages("bandit2arm")

And the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("psuthaharan/bandit2arm")

Simulation

Simulate behavior:

library(bandit2arm)
# generate 100 individuals who completed the task
# dataset1 contains behavioral data keeping only those selected-choice trials
dataset1 <- simulate_bandit2arm(n_subj = 100, n_tr = 200, trials.unique = TRUE)

# dataset2 contains behavioral data keeping all trials (selected-choice and the non-selected-choice)
dataset2 <- simulate_bandit2arm(n_subj = 100, n_tr = 200, trials.unique = FALSE)

Visualization

Plot behavior:

# for plotting purpose - let's use dataset2
# You randomly select a participant to observe his/her behavior

# Participant 100
# View first 10 rows of data
head(dataset2$bandit2arm[[100]],10)
#>    Trial        Value        Pr Option Action      Reward
#> 1      1  0.011472946 0.5000000      1      1  0.07437611
#> 2      1  0.000000000 0.5000000      2      1  0.07437611
#> 3      2  0.150651408 0.5024829      1      1  0.91373052
#> 4      2  0.000000000 0.4975171      2      1  0.91373052
#> 5      3  0.150651408 0.5325572      1      2 -1.39233659
#> 6      3 -0.214775990 0.4674428      2      2 -1.39233659
#> 7      4  0.028651131 0.5784314      1      1 -0.64024449
#> 8      4 -0.214775990 0.4215686      2      1 -0.64024449
#> 9      5 -0.001389443 0.5524874      1      1 -0.16609406
#> 10     5 -0.214775990 0.4475126      2      1 -0.16609406


# Visualize behavior 
plot_bandit2arm(data = dataset2, subj = 100, colors = c("orange","purple"))

# Visualize behavior - animated 
plot_bandit2arm(data = dataset2, subj = 100, colors = c("orange","purple"), plot.type = "animate")

Estimation

Estimate behavioral parameters from simulated data:

# Run MLE
estimate_bandit2arm(data = dataset1, method = "mle", plot = TRUE)

# Run MAP
estimate_bandit2arm(data = dataset1, method = "map", plot = TRUE)

# Run EML
estimate_bandit2arm(data = dataset1, method = "eml", plot = TRUE)

psuthaharan/bandit2arm documentation built on Jan. 26, 2021, 1:36 a.m.