choicerules: Choicerule Models (action-selection rules)

In JanaJarecki/cognitivemodels: Cognitive Models - Estimation, Prediction, and Development of Models for Cognitive Scientists

Choicerule Models (action-selection rules)

Description

Models discrete action selection: applies a choce rule/decision rule/action selection rule to select among values.

• softmax() fits a soft-maximum = soft approximation to argmax (Sutton & Barto, 2018).

• epsilon_greedy() fits epsilon-greedy.

• epsilon() fits probabilistic-epsilon.

• argmax() maximizes deterministically.

• luce() selects proportionally (Luce's rule aka Luce's axiom, Luce, 1959).

Usage

softmax(formula, data, fix = list(), options = NULL)

epsilon_greedy(formula, data, fix = list(), options = NULL)

epsilon(formula, data, fix = list(), options = NULL)

luce(formula, data, ...)

argmax(formula, data, ...)

Arguments

formula	A formula, the variables in data to be modeled. For example, y ~ x1 | x2 models response y as function of two stimuli with values x1 and x2 (respectively). Lines | separate stimuli.
data	A data frame, the data to be modeled.
fix	(optional) A list with parameter-value pairs of fixed parameters. If missing all free parameters are estimated. If set to "start" all parameters are fixed to their start values. Model parameter names are tau (see details - model parameters). list(tau = 3.85) sets parameter tau equal to 3.85. "start" sets all parameters equal to their initial values (estimates none). Useful for building a first test model.
options	(optional) A list, list entries change the modeling procedure. For example, list(lb = c(k=0)) changes the lower bound of parameter k to 0, or list(fit_measure = "mse") changes the goodness of fit measure in parameter estimation to mean-squared error, for all options, see cm_options.
...	other arguments, ignored.
discount	A number, how many initial trials to not use during parameter fitting.

Details

This is how the model predicts and treats observations:

• For formula = y ~ x1 and y ~ x1 | x2 it predicts the probability to select x1, thus y = 1 must mean select x1.

• For formula = y ~ x1 | x2 | x3 it predicts three columns, the probabilities to select x1, x2, and x3, respectively.

Model Parameters

Most models have no free parameters, except softmax and epsilon greedy which have 1 free parameter each:

• In softmax(): tau: the softness of the choices, high values cause more equiprobable choices.

• In epsilon() and epsilon_greedy(): eps: the error proportion of the choices, high values cause more errors.

Background

epsilon() picks action i with probability (1 - ε)*p(i) and with ε it picks randomly from all actions. For ε = 0 it gives p(i), that is the original probabilistic policy.

epsilon_greedy() picks the best action with probability 1 - ε, and with ε it picks randomly from all actions, including the best.

argmax() picks the highest-valued action with probability 1, and in case of ties it picks equiprobable.

luce() picks action i with probability v(i) / ∑ v.

Value

Returns a cognitive model object, which is an object of class cm. A model, that has been assigned to m, can be summarized with summary(m) or anova(m). The parameter space can be viewed using pa. rspace(m), constraints can be viewed using constraints(m).

Author(s)

Jana B. Jarecki, [e-mail]

References

Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd Ed.). MIT Press, Cambridge, MA. (http://incompleteideas.net/book/the-book-2nd.html)

Luce, R. D. (1959). On the possible psychophysical laws. Psychological Review, 66(2), 81-95. doi:10.1037/h0043178

See Also

Other cognitive models: baseline_const_c(), bayes(), cpt, ebm(), hm1988(), shift(), shortfall, threshold(), utility

Examples

# Make some fake data
D <- data.frame(a = c(.3,.8,.5),       # value of option A
                b = c(.7,.2,.5),       # value of option B
                y = c(0,1,1))          # respondent's choice (0=A, 1=B)

M <- softmax(y ~ a | b, D, c(tau=1))   # creates soft-max model w tau=1

predict(M)                             # predict action selection
M$predict()                            # -- (same) --
summary(M)                             # summarize
anova(M)                               # anova-like table
coef(M)                                # free parameter (NULL)
M$get_par()                            # fixed parameter (tau = 1)
M$npar()                               # 1 parameter
M$MSE()                                # mean-squared error
logLik(M)                              # log likelihood


### Parameter specification and fitting ---------------------------------
softmax(y ~ a | b, D, fix="start")     # fix 'tau' to its start value
softmax(y ~ a | b, D, fix=c(tau=0.2))  # fix 'tau' to 0.2
softmax(y ~ a | b, D)                  # fit 'tau' to data y in D


### The different choice rules ------------------------------------------
softmax(y ~ a | b, D,  fix=c(tau=0.5)) # fix 'tau' to 0.5
softmax(y ~ a | b, D)                  # fit 'tau' to y
epsilon_greedy(y~a | b, D, c(eps=0.1)) # fix 'eps' to 10 %
epsilon_greedy(y~a | b, D )            # fit 'eps' to y
epsilon(y ~ a | b, D, c(eps=0.1))      # fix 'eps' to 0.1
epsilon(y ~ a | b, D)                  # fit 'eps' to y
luce(y ~ a | b, D)                     # Luce's choice rule, 0 parameter
argmax(y ~ a | b, D)                   # Argmax choice rule, 0 parameter

JanaJarecki/cognitivemodels documentation built on Nov. 5, 2022, 5:30 a.m.