MDP: Markov Decision Processes (MDPs) in R

Using this package you can create and optimize MDPs with discrete time steps and state space. Both hierarchical and ordinary MDPs can be modelled.

Author
Lars Relund Nielsen
Date of publication
2015-03-09 09:30:16
Maintainer
Lars Relund Nielsen <lars@relund.dk>
License
GPL (>= 2)
Version
1.1

View on R-Forge

Man pages

actionIdxDf
Info about the actions in the HMDP model under consideration.
actionIdxMat
Info about the actions in the HMDP model under consideration.
actionInfo
Info about the actions in the HMDP model under consideration.
actionWeightMat
Info about the weights of the actions in the HMDP model under...
binaryMDPWriter
Function for writing an HMDP model to binary files. The...
calcRPO
Calculate the rentention payoff (RPO) or opportunity cost for...
calcSteadyStatePr
Calculate the steady state transition probabilities for the...
calcWeights
Calculate weights based on current policy. Normally run after...
convertBinary2HMP
Convert a HMDP model stored in binary format to a hmp (xml)...
convertHMP2Binary
Convert a HMDP model stored in a hmp (xml) file to binary...
fixAction
Fix the action of a state. That is, the other actions are...
getActionTransIdS
Get the ids of the transition states of an action.
getActionTransPr
Get the transition probabilities of the transition states of...
getActionW
Get the weights of an action.
getIdS
Return ids for states having index string in idxS.
getIdSStages
Return ids for states in a stage.
getLabel
Return the label of states having id idS.
getPolicy
Get parts of the optimal policy.
getPolicyW
Get parts of the optimal policy weights.
getStrIdxS
Return the index strings for states having id idS.
getTransPr
Get the transition probability matrix P for the founder...
getWIdx
Return the index of a weight in the model. Note that index...
hmpMDPWriter
Function for writing an HMDP model to a hmp file (XML). The...
hypergf
The state-expanded hypergraph as a matrix
info
Information about a state and its corresponding actions
loadMDP
Create the HMDP defined in the binary files. The model are...
MDP
Markov Decision Processes (MDPs) in R
policyIteAve
Perform policy iteration (average criterion) on the MDP.
policyIteDiscount
Perform policy iteration (discount criterion) on the MDP.
removeAction
Remove the action of a state from the HMDP.
resetActions
Reset the actions of a state.
setActionWeight
Set the weight of an action.
setPolicy
Set the current policy.
setPolicyAction
Set the action of a state to be in the current policy.
setStateWeight
Set the weight of a state.
stateIdxDf
Info about the states in the HMDP model under consideration.
stateIdxMat
Info about the states in the HMDP model under consideration.
transProbMat
Info about the transition probabilities in the HMDP model...
valueIte
Perform value iteration on the MDP.
weightNames
Names of weights used in actions.

Files in this package

MDP/DESCRIPTION
MDP/NAMESPACE
MDP/R
MDP/R/binaryActionWriter.R
MDP/R/binaryMDPWriter.R
MDP/R/convertBinary2HMP.R
MDP/R/convertHMP2Binary.R
MDP/R/hmpMDPWriter.R
MDP/R/infoMat.R
MDP/R/loadMDP.R
MDP/R/mdp-package.R
MDP/R/zzz.R
MDP/codeblocks
MDP/codeblocks/main.cc
MDP/codeblocks/mdp.cbp
MDP/inst
MDP/inst/doc
MDP/inst/doc/mdp_examples.Rtex
MDP/inst/doc/mdp_examples_files
MDP/inst/doc/mdp_examples_files/cow.csv
MDP/inst/doc/mdp_examples_files/hmdp_index.pdf
MDP/inst/doc/mdp_examples_files/listingsRw.tex
MDP/inst/doc/mdp_examples_files/litt.bib
MDP/inst/doc/mdp_examples_files/state_hgf.pdf
MDP/inst/models
MDP/inst/models/machine1.hmp
MDP/inst/models/machine1_actionIdx.bin
MDP/inst/models/machine1_actionIdxLbl.bin
MDP/inst/models/machine1_actionWeight.bin
MDP/inst/models/machine1_actionWeightLbl.bin
MDP/inst/models/machine1_stateIdx.bin
MDP/inst/models/machine1_stateIdxLbl.bin
MDP/inst/models/machine1_transProb.bin
MDP/inst/models/machine2_actionIdx.bin
MDP/inst/models/machine2_actionIdxLbl.bin
MDP/inst/models/machine2_actionWeight.bin
MDP/inst/models/machine2_actionWeightLbl.bin
MDP/inst/models/machine2_stateIdx.bin
MDP/inst/models/machine2_stateIdxLbl.bin
MDP/inst/models/machine2_transProb.bin
MDP/man
MDP/man/MDP.Rd
MDP/man/actionIdxDf.Rd
MDP/man/actionIdxMat.Rd
MDP/man/actionInfo.Rd
MDP/man/actionWeightMat.Rd
MDP/man/binaryMDPWriter.Rd
MDP/man/calcRPO.Rd
MDP/man/calcSteadyStatePr.Rd
MDP/man/calcWeights.Rd
MDP/man/convertBinary2HMP.Rd
MDP/man/convertHMP2Binary.Rd
MDP/man/fixAction.Rd
MDP/man/getActionTransIdS.Rd
MDP/man/getActionTransPr.Rd
MDP/man/getActionW.Rd
MDP/man/getIdS.Rd
MDP/man/getIdSStages.Rd
MDP/man/getLabel.Rd
MDP/man/getPolicy.Rd
MDP/man/getPolicyW.Rd
MDP/man/getStrIdxS.Rd
MDP/man/getTransPr.Rd
MDP/man/getWIdx.Rd
MDP/man/hmpMDPWriter.Rd
MDP/man/hypergf.Rd
MDP/man/info.Rd
MDP/man/loadMDP.Rd
MDP/man/policyIteAve.Rd
MDP/man/policyIteDiscount.Rd
MDP/man/removeAction.Rd
MDP/man/resetActions.Rd
MDP/man/setActionWeight.Rd
MDP/man/setPolicy.Rd
MDP/man/setPolicyAction.Rd
MDP/man/setStateWeight.Rd
MDP/man/stateIdxDf.Rd
MDP/man/stateIdxMat.Rd
MDP/man/transProbMat.Rd
MDP/man/valueIte.Rd
MDP/man/weightNames.Rd
MDP/src
MDP/src/Makevars
MDP/src/basicdt.hh
MDP/src/debug.h
MDP/src/hgfreader.cc
MDP/src/hgfreader.hh
MDP/src/hmdp.cc
MDP/src/hmdp.hh
MDP/src/hmdpReader.cc
MDP/src/hmdpReader.hh
MDP/src/htacyclic.cc
MDP/src/htacyclic.hh
MDP/src/hypergf.cc
MDP/src/hypergf.hh
MDP/src/matalg.hh
MDP/src/matrix.hh
MDP/src/rmain.cc
MDP/src/time.hh
MDP/tests
MDP/tests/binaryMDPWriter.Rex
MDP/tests/hmpMDPWriter.Rex
MDP/tests/infoMat.Rex
MDP/tests/machine.Rex
MDP/tests/sow.Rex