softmax_rl: Softmax reinforcement learning model

In XSLmodels: Run and Optimize Cross-Situational Word Learning Models

Softmax reinforcement learning model

Description

A Q-learning-style reinforcement learning model of cross-situational word learning. Unlike every other model in this package, which updates every word-object pair presented together on a trial, this model makes a discrete choice: for each word heard, it samples a single candidate referent from a softmax policy over its current values for all known objects (its current best guess at that word's meaning), then updates only that guess's value via a standard TD/delta-rule update, using whether the guess was actually present in the scene as the reward (1 = "confirmed", 0 = "disconfirmed"). This is the only feedback available in a task with no trial-by-trial correct answer, and mirrors the propose-and-verify logic of propose_but_verify()/pursuit(), but replaces their threshold-based keep/discard rules with graded value learning and an explicit (softmax) exploration/exploitation tradeoff.

Usage

softmax_rl(alpha, beta)

Arguments

alpha	Learning rate
beta	Inverse temperature: how deterministically the model favors its highest-valued guess (0 = uniformly random guessing, large = always picks the current best guess)

Value

An object of class xslMod

Examples

mod <- softmax_rl(alpha = .3, beta = 3)
xsl_run(mod, get_example_ambiguous_condition())

XSLmodels documentation built on Oct. 18, 2026, 5:08 p.m.