getFirstTransition: Get Transition from First to Second State
In jdtrat/dynaq: Tools to Simulate DynaQ Reinforcement Learning Algorithms

Description Usage Arguments Details Value

getFirstTransition takes in a state (though unnecessary, as the task structure dictates it will always be State "FC") and an action and returns the next state based off them. If this is being called for actual experience, that is all it does. simulated experience, is more complicated and described below.

getFirstTransition(
  state,
  action,
  sim = FALSE,
  tFunction = NULL,
  transDF = NULL
)

`state`	The first state.
`action`	The action chosen in the first state.
`sim`	Logical: TRUE if simulating experience; FALSE if not simulating experience.
`tFunction`	Transition function that has the current probability of transitioning to States "A" or "B".
`transDF`	Transition data from real experience as defined in `simModel`

If simulated experience, the function requires a transition function and transition data from real experience. Using these, it determines the next state based on the supplied transition function, tFunction, and prior experience, transDF.

If this is the first round, as indicated by an empty trans$Table global variable (which tracks the prior state-action-reward information), then the transition is chosen based off whatever was previously experienced during the actual trial. It does this by calling the randomPrevious function, passing in "State2", and observing what was really experienced.

This is to say that an individual cannot experience a state in a simulation that they have not previously visited. Similarly, if this is not the first simulated experience, the function checks for which states have been visited previously. If, for example, the action is right, and the agent has only experienced State "RB," then they will only transition to that state in this simulation. However, if they've experienced both States RA and RB, they could transition to either based on the supplied transition tFunction (i.e. the agent's model of the task structure).

Next state.

jdtrat/dynaq documentation built on July 24, 2020, 7:18 a.m.