# choicerules: Choicerule Models (action-selection rules) In JanaJarecki/cogscimodels: Cognitive Models - Estimation, Prediction, and Development of Models for Cognitive Scientists

## Description

Models discrete action selection: applies a choce rule/decision rule/action selection rule to select among values.

• `softmax()` fits a soft-maximum = soft approximation to argmax (Sutton & Barto, 2018).

• `epsilon_greedy()` fits epsilon-greedy.

• `epsilon()` fits probabilistic-epsilon.

• `argmax()` maximizes deterministically.

• `luce()` selects proportionally (Luce's rule aka Luce's axiom, Luce, 1959).

## Usage

 ```1 2 3 4 5 6 7 8 9``` ```softmax(formula, data, fix = list(), options = NULL) epsilon_greedy(formula, data, fix = list(), options = NULL) epsilon(formula, data, fix = list(), options = NULL) luce(formula, data, ...) argmax(formula, data, ...) ```

## Arguments

 `formula` A formula, the variables in `data` to be modeled. For example, `y ~ x1 | x2` models response y as function of two stimuli with values x1 and x2 (respectively). Lines `|` separate stimuli. `data` A data frame, the data to be modeled. `fix` (optional) A list or the string `"start"`, the fixed model parameters, if missing all parameters are estimated. Model parameter names are `tau` (see details - model parameters). `list(tau = 3.85)` sets parameter `tau` equal to 3.85. `"start"` sets all parameters equal to their initial values (estimates none). Useful for building a first test model. `options` (optional) A list, list entries change the modeling procedure. For example, `list(lb = c(k=0))` changes the lower bound of parameter k to 0, or `list(fit_measure = "mse")` changes the goodness of fit measure in parameter estimation to mean-squared error, for all options, see `cm_options()`. `...` other arguments, ignored. `discount` A number, how many initial trials to not use during parameter fitting.

## Details

This is how the model predicts and treats observations:

• For `formula = y ~ x1` and `y ~ x1 | x2` it predicts the probability to select x1, thus `y = 1` must mean select x1.

• For `formula = y ~ x1 | x2 | x3` it predicts three columns, the probabilities to select x1, x2, and x3, respectively.

#### Model Parameters

Most models have no free parameters, except softmax and epsilon greedy which have 1 free parameter each:

• In `softmax()`: `tau`: the softness of the choices, high values cause more equiprobable choices.

• In `epsilon()` and `epsilon_greedy()`: `eps`: the error proportion of the choices, high values cause more errors.

#### Background

`epsilon()` picks action i with probability (1 - ε)*p(i) and with ε it picks randomly from all actions. For ε = 0 it gives p(i), that is the original probabilistic policy.

`epsilon_greedy()` picks the best action with probability 1 - ε, and with ε it picks randomly from all actions, including the best.

`argmax()` picks the highest-valued action with probability 1, and in case of ties it picks equiprobable.

`luce()` picks action i with probability v(i) / ∑ v.

## Value

Returns a cognitive model object, which is an object of class cm. A model, that has been assigned to `m`, can be summarized with `summary(m)` or `anova(m)`. The parameter space can be viewed using `pa. rspace(m)`, constraints can be viewed using `constraints(m)`.

## Author(s)

Jana B. Jarecki, jj@janajarecki.com

## References

Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd Ed.). MIT Press, Cambridge, MA. (http://incompleteideas.net/book/the-book-2nd.html)

Luce, R. D. (1959). On the possible psychophysical laws. Psychological Review, 66(2), 81-95. doi:10.1037/h0043178

Other cognitive models: `baseline_const_c()`, `bayes()`, `cpt`, `ebm()`, `hm1988()`, `shortfall`, `utility`
 ``` 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33``` ```# Make some fake data D <- data.frame(a = c(.3,.8,.5), # value of option A b = c(.7,.2,.5), # value of option B y = c(0,1,1)) # respondent's choice (0=A, 1=B) M <- softmax(y ~ a | b, D, c(tau=1)) # creates soft-max model w tau=1 predict(M) # predict action selection M\$predict() # -- (same) -- summary(M) # summarize anova(M) # anova-like table coef(M) # free parameter (NULL) M\$get_par() # fixed parameter (tau = 1) M\$npar() # 1 parameter M\$MSE() # mean-squared error logLik(M) # log likelihood ### Parameter specification and fitting --------------------------------- softmax(y ~ a | b, D, fix="start") # fix 'tau' to its start value softmax(y ~ a | b, D, fix=c(tau=0.2)) # fix 'tau' to 0.2 softmax(y ~ a | b, D) # fit 'tau' to data y in D ### The different choice rules ------------------------------------------ softmax(y ~ a | b, D, fix=c(tau=0.5)) # fix 'tau' to 0.5 softmax(y ~ a | b, D) # fit 'tau' to y epsilon_greedy(y~a | b, D, c(eps=0.1)) # fix 'eps' to 10 % epsilon_greedy(y~a | b, D ) # fit 'eps' to y epsilon(y ~ a | b, D, c(eps=0.1)) # fix 'eps' to 0.1 epsilon(y ~ a | b, D) # fit 'eps' to y luce(y ~ a | b, D) # Luce's choice rule, 0 parameter argmax(y ~ a | b, D) # Argmax choice rule, 0 parameter ```