Traj2SARS: Convert Trajectory to SARS
In XiaoqiLu/PhD-Thesis: Regularized Q-Learning

Description Usage Arguments Details Value Examples

The function Traj2SARS() converts a trajectory object to a SARS object.

1	Traj2SARS(traj, Interpreter, skip = 0)

`traj`	a trajectory object (`class = "Traj"`).
`Interpreter`	an interpreter function, see details.
`skip`	number of steps to skip over

The interpreter function Interpreter(actions, observations) takes the history information up to certain time step, and outputs a list of (state, reward). Specifically, the history information is partial trajectory:

O_1, A_1, O_2, A_2, …, O_t, A_t, O_{t+1}

a SARS object (class = "SARS")

observations <- list(0, -1, -1, 0, 1, 1)
actions <- list(-1, 1, 1, 0, -1)
traj <- Traj(observations, actions)
Interpreter <- function(actions, observations) {
n <- length(actions)
if (n > 0) {
  state <- c(observations[[n]], observations[[n + 1]])
  reward <- 1 - state[2]^2
} else {
  state <- reward <- NULL
}
return(list(state = state, reward = reward))
}
Traj2SARS(traj, Interpreter, skip = 1)