SARS2Phis: Convert SARS to Basis Representation

In XiaoqiLu/PhD-Thesis: Regularized Q-Learning

Description

The function SARS2Phis() converts a SARS object to basis representation. The motivation is to preprocess data and prepare for faster training.

Usage

SARS2Phis(sars, action_space, Feature)

Arguments

sars	a SARS object (class = "SARS").
action_space	a list of feasible actions.
Feature	feature function, taking states and actions as input, outputs basis.

Value

a list containing the following components:

phi

data matrix of φ(state, action)

phi_next_list

a list of data matrices, each matrix has the same size as phi and represents the next time-step data matrix with each possible action

r

rewards

n

sample size

Examples

states <- matrix(c(1, 2, 3, 4), 2, 2)
actions <- matrix(c(1, 0), 2, 1)
rewards <- matrix(c(1, 2), 2, 1)
states_next <- matrix(c(2, 3, 4, 5), 2, 2)
sars <- SARS(states, actions, rewards, states_next)
Feature <- function(states, actions) {
  phi <- RowWiseKronecker(Poly(states, 2), Poly(actions, 1))
  return(phi)
}
SARS2Phis(sars, list(0, 1), Feature)

XiaoqiLu/PhD-Thesis documentation built on March 1, 2021, 10:49 a.m.