MDPtoolbox: Markov Decision Processes Toolbox

The Markov Decision Processes (MDP) toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: finite horizon, value iteration, policy iteration, linear programming algorithms with some variants and also proposes some functions related to Reinforcement Learning.

Install the latest version of this package by entering the following in R:
AuthorIadine Chades, Guillaume Chapron, Marie-Josee Cros, Frederick Garcia, Regis Sabbadin
Date of publication2017-03-03 18:01:56
MaintainerGuillaume Chapron <>
LicenseBSD_3_clause + file LICENSE

View on CRAN

Man pages

mdp_bellman_operator: Applies the Bellman operator

mdp_check: Checks the validity of a MDP

mdp_check_square_stochastic: Checks if a matrix is square and stochastic

mdp_computePpolicyPRpolicy: Computes the transition matrix and the reward matrix for a...

mdp_computePR: Computes a reward matrix for any form of transition and...

mdp_eval_policy_iterative: Evaluates a policy using an iterative method

mdp_eval_policy_matrix: Evaluates a policy using matrix inversion and product

mdp_eval_policy_optimality: Computes sets of 'near optimal' actions for each state

mdp_eval_policy_TD_0: Evaluates a policy using the TD(0) algorithm

mdp_example_forest: Generates a MDP for a simple forest management problem

mdp_example_rand: Generates a random MDP problem

mdp_finite_horizon: Solves finite-horizon MDP using backwards induction algorithm

mdp_LP: Solves discounted MDP using linear programming algorithm

mdp_policy_iteration: Solves discounted MDP using policy iteration algorithm

mdp_policy_iteration_modified: Solves discounted MDP using modified policy iteration...

mdp_Q_learning: Solves discounted MDP using the Q-learning algorithm...

mdp_relative_value_iteration: Solves MDP with average reward using relative value iteration...

mdp_span: Evaluates the span of a vector

MDPtoolbox-package: Markov Decision Processes Toolbox

mdp_value_iteration: Solves discounted MDP using value iteration algorithm

mdp_value_iteration_bound_iter: Computes a bound for the number of iterations for the value...

mdp_value_iterationGS: Solves discounted MDP using Gauss-Seidel's value iteration...


mdp_bellman_operator Man page
mdp_check Man page
mdp_check_square_stochastic Man page
mdp_computePpolicyPRpolicy Man page
mdp_computePR Man page
mdp_eval_policy_iterative Man page
mdp_eval_policy_matrix Man page
mdp_eval_policy_optimality Man page
mdp_eval_policy_TD_0 Man page
mdp_example_forest Man page
mdp_example_rand Man page
mdp_finite_horizon Man page
mdp_LP Man page
mdp_policy_iteration Man page
mdp_policy_iteration_modified Man page
mdp_Q_learning Man page
mdp_relative_value_iteration Man page
mdp_span Man page
MDPtoolbox Man page
MDPtoolbox-package Man page
mdp_value_iteration Man page
mdp_value_iteration_bound_iter Man page
mdp_value_iterationGS Man page

Questions? Problems? Suggestions? or email at

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.