MDP: Markov Decision Processes (MDPs) in R

Using this package you can create and optimize MDPs with discrete time steps and state space. Both hierarchical and ordinary MDPs can be modelled.

AuthorLars Relund Nielsen
Date of publication2015-03-09 09:30:16
MaintainerLars Relund Nielsen <lars@relund.dk>
LicenseGPL (>= 2)
Version1.1

View on R-Forge

Man pages

actionIdxDf: Info about the actions in the HMDP model under consideration.

actionIdxMat: Info about the actions in the HMDP model under consideration.

actionInfo: Info about the actions in the HMDP model under consideration.

actionWeightMat: Info about the weights of the actions in the HMDP model under...

binaryMDPWriter: Function for writing an HMDP model to binary files. The...

calcRPO: Calculate the rentention payoff (RPO) or opportunity cost for...

calcSteadyStatePr: Calculate the steady state transition probabilities for the...

calcWeights: Calculate weights based on current policy. Normally run after...

convertBinary2HMP: Convert a HMDP model stored in binary format to a hmp (xml)...

convertHMP2Binary: Convert a HMDP model stored in a hmp (xml) file to binary...

fixAction: Fix the action of a state. That is, the other actions are...

getActionTransIdS: Get the ids of the transition states of an action.

getActionTransPr: Get the transition probabilities of the transition states of...

getActionW: Get the weights of an action.

getIdS: Return ids for states having index string in idxS.

getIdSStages: Return ids for states in a stage.

getLabel: Return the label of states having id idS.

getPolicy: Get parts of the optimal policy.

getPolicyW: Get parts of the optimal policy weights.

getStrIdxS: Return the index strings for states having id idS.

getTransPr: Get the transition probability matrix P for the founder...

getWIdx: Return the index of a weight in the model. Note that index...

hmpMDPWriter: Function for writing an HMDP model to a hmp file (XML). The...

hypergf: The state-expanded hypergraph as a matrix

info: Information about a state and its corresponding actions

loadMDP: Create the HMDP defined in the binary files. The model are...

MDP: Markov Decision Processes (MDPs) in R

policyIteAve: Perform policy iteration (average criterion) on the MDP.

policyIteDiscount: Perform policy iteration (discount criterion) on the MDP.

removeAction: Remove the action of a state from the HMDP.

resetActions: Reset the actions of a state.

setActionWeight: Set the weight of an action.

setPolicy: Set the current policy.

setPolicyAction: Set the action of a state to be in the current policy.

setStateWeight: Set the weight of a state.

stateIdxDf: Info about the states in the HMDP model under consideration.

stateIdxMat: Info about the states in the HMDP model under consideration.

transProbMat: Info about the transition probabilities in the HMDP model...

valueIte: Perform value iteration on the MDP.

weightNames: Names of weights used in actions.

Functions

actionIdxDf Man page
actionIdxMat Man page
actionInfo Man page
actionWeightMat Man page
binaryMDPWriter Man page
calcRPO Man page
calcSteadyStatePr Man page
calcWeights Man page
convertBinary2HMP Man page
convertHMP2Binary Man page
fixAction Man page
getActionTransIdS Man page
getActionTransPr Man page
getActionW Man page
getIdS Man page
getIdSStages Man page
getLabel Man page
getPolicy Man page
getPolicyW Man page
getStrIdxS Man page
getTransPr Man page
getWIdx Man page
hmpMDPWriter Man page
hypergf Man page
info Man page
loadMDP Man page
MDP Man page
MDP-package Man page
policyIteAve Man page
policyIteDiscount Man page
removeAction Man page
resetActions Man page
setActionWeight Man page
setPolicy Man page
setPolicyAction Man page
setStateWeight Man page
stateIdxDf Man page
stateIdxMat Man page
transProbMat Man page
valueIte Man page
weightNames Man page

Files

MDP/DESCRIPTION
MDP/NAMESPACE
MDP/R
MDP/R/binaryActionWriter.R MDP/R/binaryMDPWriter.R MDP/R/convertBinary2HMP.R MDP/R/convertHMP2Binary.R MDP/R/hmpMDPWriter.R MDP/R/infoMat.R MDP/R/loadMDP.R MDP/R/mdp-package.R MDP/R/zzz.R
MDP/codeblocks
MDP/codeblocks/main.cc
MDP/codeblocks/mdp.cbp
MDP/inst
MDP/inst/doc
MDP/inst/doc/mdp_examples.Rtex
MDP/inst/doc/mdp_examples_files
MDP/inst/doc/mdp_examples_files/cow.csv
MDP/inst/doc/mdp_examples_files/hmdp_index.pdf
MDP/inst/doc/mdp_examples_files/listingsRw.tex
MDP/inst/doc/mdp_examples_files/litt.bib
MDP/inst/doc/mdp_examples_files/state_hgf.pdf
MDP/inst/models
MDP/inst/models/machine1.hmp
MDP/inst/models/machine1_actionIdx.bin
MDP/inst/models/machine1_actionIdxLbl.bin
MDP/inst/models/machine1_actionWeight.bin
MDP/inst/models/machine1_actionWeightLbl.bin
MDP/inst/models/machine1_stateIdx.bin
MDP/inst/models/machine1_stateIdxLbl.bin
MDP/inst/models/machine1_transProb.bin
MDP/inst/models/machine2_actionIdx.bin
MDP/inst/models/machine2_actionIdxLbl.bin
MDP/inst/models/machine2_actionWeight.bin
MDP/inst/models/machine2_actionWeightLbl.bin
MDP/inst/models/machine2_stateIdx.bin
MDP/inst/models/machine2_stateIdxLbl.bin
MDP/inst/models/machine2_transProb.bin
MDP/man
MDP/man/MDP.Rd MDP/man/actionIdxDf.Rd MDP/man/actionIdxMat.Rd MDP/man/actionInfo.Rd MDP/man/actionWeightMat.Rd MDP/man/binaryMDPWriter.Rd MDP/man/calcRPO.Rd MDP/man/calcSteadyStatePr.Rd MDP/man/calcWeights.Rd MDP/man/convertBinary2HMP.Rd MDP/man/convertHMP2Binary.Rd MDP/man/fixAction.Rd MDP/man/getActionTransIdS.Rd MDP/man/getActionTransPr.Rd MDP/man/getActionW.Rd MDP/man/getIdS.Rd MDP/man/getIdSStages.Rd MDP/man/getLabel.Rd MDP/man/getPolicy.Rd MDP/man/getPolicyW.Rd MDP/man/getStrIdxS.Rd MDP/man/getTransPr.Rd MDP/man/getWIdx.Rd MDP/man/hmpMDPWriter.Rd MDP/man/hypergf.Rd MDP/man/info.Rd MDP/man/loadMDP.Rd MDP/man/policyIteAve.Rd MDP/man/policyIteDiscount.Rd MDP/man/removeAction.Rd MDP/man/resetActions.Rd MDP/man/setActionWeight.Rd MDP/man/setPolicy.Rd MDP/man/setPolicyAction.Rd MDP/man/setStateWeight.Rd MDP/man/stateIdxDf.Rd MDP/man/stateIdxMat.Rd MDP/man/transProbMat.Rd MDP/man/valueIte.Rd MDP/man/weightNames.Rd
MDP/src
MDP/src/Makevars
MDP/src/basicdt.hh
MDP/src/debug.h
MDP/src/hgfreader.cc
MDP/src/hgfreader.hh
MDP/src/hmdp.cc
MDP/src/hmdp.hh
MDP/src/hmdpReader.cc
MDP/src/hmdpReader.hh
MDP/src/htacyclic.cc
MDP/src/htacyclic.hh
MDP/src/hypergf.cc
MDP/src/hypergf.hh
MDP/src/matalg.hh
MDP/src/matrix.hh
MDP/src/rmain.cc
MDP/src/time.hh
MDP/tests
MDP/tests/binaryMDPWriter.Rex
MDP/tests/hmpMDPWriter.Rex
MDP/tests/infoMat.Rex
MDP/tests/machine.Rex
MDP/tests/sow.Rex

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

Please suggest features or report bugs with the GitHub issue tracker.

All documentation is copyright its authors; we didn't write any of that.