Evaluates a policy using iterations of the Bellman operator

1 | ```
mdp_eval_policy_iterative(P, R, discount, policy, V0, epsilon, max_iter)
``` |

`P` |
transition probability array. P can be a 3 dimensions array [S,S,A] or a list [[A]], each element containing a sparse matrix [S,S]. |

`R` |
reward array. R can be a 3 dimensions array [S,S,A] or a list [[A]], each element containing a sparse matrix [S,S] or a 2 dimensional matrix [S,A] possibly sparse. |

`discount` |
discount factor. discount is a real number which belongs to [0; 1[. |

`policy` |
a policy. policy is a S length vector. Each element is an integer corresponding to an action. |

`V0` |
(optional) starting point. V0 is a S length vector representing an inital guess of the value function. By default, V0 is only composed of 0 elements. |

`epsilon` |
(optional) search for an epsilon-optimal policy. epsilon is a real greater than 0. By default, epsilon = 0.01. |

`max_iter` |
(optional) maximum number of iterations. max_iter is an integer greater than 0. If the value given in argument is greater than a computed bound, a warning informs that the computed bound will be used instead. By default, max_iter = 1000. |

mdp_eval_policy_iterative evaluates the value fonction associated to a policy applying iteratively the Bellman operator.

`Vpolicy` |
value fonction. Vpolicy is a S length vector. |

1 2 3 4 5 6 7 8 9 10 11 12 13 | ```
# With a non-sparse matrix
P <- array(0, c(2,2,2))
P[,,1] <- matrix(c(0.5, 0.5, 0.8, 0.2), 2, 2, byrow=TRUE)
P[,,2] <- matrix(c(0, 1, 0.1, 0.9), 2, 2, byrow=TRUE)
R <- matrix(c(5, 10, -1, 2), 2, 2, byrow=TRUE)
policy <- c(2,1)
mdp_eval_policy_iterative(P, R, 0.8, policy)
# With a sparse matrix
P <- list()
P[[1]] <- Matrix(c(0.5, 0.5, 0.8, 0.2), 2, 2, byrow=TRUE, sparse=TRUE)
P[[2]] <- Matrix(c(0, 1, 0.1, 0.9), 2, 2, byrow=TRUE, sparse=TRUE)
mdp_eval_policy_iterative(P, R, 0.8, policy)
``` |

Questions? Problems? Suggestions? Tweet to @rdrrHQ or email at ian@mutexlabs.com.

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.