mdp_eval_policy_TD_0: Evaluates a policy using the TD(0) algorithm In MDPtoolbox: Markov Decision Processes Toolbox

Description

Evaluates a policy using the TD(0) algorithm

Usage

 `1` ```mdp_eval_policy_TD_0(P, R, discount, policy, N) ```

Arguments

 `P` transition probability array. P can be a 3 dimensions array [S,S,A] or a list [[A]], each element containing a sparse matrix [S,S]. `R` reward array. R can be a 3 dimensions array [S,S,A] or a list [[A]], each element containing a sparse matrix [S,S] or a 2 dimensional matrix [S,A] possibly sparse. `discount` discount factor. discount is a real number which belongs to [0; 1[. `policy` a policy. policy is a S length vector. Each element is an integer corresponding to an action. `N` (optional) number of iterations to perform. N is an integer greater than the de- fault value. By default, N is set to 10000

Details

mdp_eval_policy_TD_0 evaluates the value fonction associated to a policy using the TD(0) algorithm (Reinforcement Learning).

Value

 `Vpolicy` value fonction. Vpolicy is a length S vector.

Examples

 ``` 1 2 3 4 5 6 7 8 9 10 11 12``` ```# With a non-sparse matrix P <- array(0, c(2,2,2)) P[,,1] <- matrix(c(0.5, 0.5, 0.8, 0.2), 2, 2, byrow=TRUE) P[,,2] <- matrix(c(0, 1, 0.1, 0.9), 2, 2, byrow=TRUE) R <- matrix(c(5, 10, -1, 2), 2, 2, byrow=TRUE) mdp_eval_policy_TD_0(P, R, 0.9, c(1,2)) # With a sparse matrix P <- list() P[[1]] <- Matrix(c(0.5, 0.5, 0.8, 0.2), 2, 2, byrow=TRUE, sparse=TRUE) P[[2]] <- Matrix(c(0, 1, 0.1, 0.9), 2, 2, byrow=TRUE, sparse=TRUE) mdp_eval_policy_TD_0(P, R, 0.9, c(1,2)) ```

MDPtoolbox documentation built on May 30, 2017, 5:15 a.m.