MDP: Markov Decision Processes (MDPs) in R

Using this package you can create and optimize MDPs with discrete time steps and state space. Both hierarchical and ordinary MDPs can be modelled.

AuthorLars Relund Nielsen
Date of publication2015-03-09 09:30:16
MaintainerLars Relund Nielsen <lars@relund.dk>
LicenseGPL (>= 2)
Version1.1

View on R-Forge

Man pages

actionIdxDf: Info about the actions in the HMDP model under consideration.

actionIdxMat: Info about the actions in the HMDP model under consideration.

actionInfo: Info about the actions in the HMDP model under consideration.

actionWeightMat: Info about the weights of the actions in the HMDP model under...

binaryMDPWriter: Function for writing an HMDP model to binary files. The...

calcRPO: Calculate the rentention payoff (RPO) or opportunity cost for...

calcSteadyStatePr: Calculate the steady state transition probabilities for the...

calcWeights: Calculate weights based on current policy. Normally run after...

convertBinary2HMP: Convert a HMDP model stored in binary format to a hmp (xml)...

convertHMP2Binary: Convert a HMDP model stored in a hmp (xml) file to binary...

fixAction: Fix the action of a state. That is, the other actions are...

getActionTransIdS: Get the ids of the transition states of an action.

getActionTransPr: Get the transition probabilities of the transition states of...

getActionW: Get the weights of an action.

getIdS: Return ids for states having index string in idxS.

getIdSStages: Return ids for states in a stage.

getLabel: Return the label of states having id idS.

getPolicy: Get parts of the optimal policy.

getPolicyW: Get parts of the optimal policy weights.

getStrIdxS: Return the index strings for states having id idS.

getTransPr: Get the transition probability matrix P for the founder...

getWIdx: Return the index of a weight in the model. Note that index...

hmpMDPWriter: Function for writing an HMDP model to a hmp file (XML). The...

hypergf: The state-expanded hypergraph as a matrix

info: Information about a state and its corresponding actions

loadMDP: Create the HMDP defined in the binary files. The model are...

MDP: Markov Decision Processes (MDPs) in R

policyIteAve: Perform policy iteration (average criterion) on the MDP.

policyIteDiscount: Perform policy iteration (discount criterion) on the MDP.

removeAction: Remove the action of a state from the HMDP.

resetActions: Reset the actions of a state.

setActionWeight: Set the weight of an action.

setPolicy: Set the current policy.

setPolicyAction: Set the action of a state to be in the current policy.

setStateWeight: Set the weight of a state.

stateIdxDf: Info about the states in the HMDP model under consideration.

stateIdxMat: Info about the states in the HMDP model under consideration.

transProbMat: Info about the transition probabilities in the HMDP model...

valueIte: Perform value iteration on the MDP.

weightNames: Names of weights used in actions.

Files in this package

MDP/DESCRIPTION
MDP/NAMESPACE
MDP/R
MDP/R/binaryActionWriter.R MDP/R/binaryMDPWriter.R MDP/R/convertBinary2HMP.R MDP/R/convertHMP2Binary.R MDP/R/hmpMDPWriter.R MDP/R/infoMat.R MDP/R/loadMDP.R MDP/R/mdp-package.R MDP/R/zzz.R
MDP/codeblocks
MDP/codeblocks/main.cc
MDP/codeblocks/mdp.cbp
MDP/inst
MDP/inst/doc
MDP/inst/doc/mdp_examples.Rtex
MDP/inst/doc/mdp_examples_files
MDP/inst/doc/mdp_examples_files/cow.csv
MDP/inst/doc/mdp_examples_files/hmdp_index.pdf
MDP/inst/doc/mdp_examples_files/listingsRw.tex
MDP/inst/doc/mdp_examples_files/litt.bib
MDP/inst/doc/mdp_examples_files/state_hgf.pdf
MDP/inst/models
MDP/inst/models/machine1.hmp
MDP/inst/models/machine1_actionIdx.bin
MDP/inst/models/machine1_actionIdxLbl.bin
MDP/inst/models/machine1_actionWeight.bin
MDP/inst/models/machine1_actionWeightLbl.bin
MDP/inst/models/machine1_stateIdx.bin
MDP/inst/models/machine1_stateIdxLbl.bin
MDP/inst/models/machine1_transProb.bin
MDP/inst/models/machine2_actionIdx.bin
MDP/inst/models/machine2_actionIdxLbl.bin
MDP/inst/models/machine2_actionWeight.bin
MDP/inst/models/machine2_actionWeightLbl.bin
MDP/inst/models/machine2_stateIdx.bin
MDP/inst/models/machine2_stateIdxLbl.bin
MDP/inst/models/machine2_transProb.bin
MDP/man
MDP/man/MDP.Rd MDP/man/actionIdxDf.Rd MDP/man/actionIdxMat.Rd MDP/man/actionInfo.Rd MDP/man/actionWeightMat.Rd MDP/man/binaryMDPWriter.Rd MDP/man/calcRPO.Rd MDP/man/calcSteadyStatePr.Rd MDP/man/calcWeights.Rd MDP/man/convertBinary2HMP.Rd MDP/man/convertHMP2Binary.Rd MDP/man/fixAction.Rd MDP/man/getActionTransIdS.Rd MDP/man/getActionTransPr.Rd MDP/man/getActionW.Rd MDP/man/getIdS.Rd MDP/man/getIdSStages.Rd MDP/man/getLabel.Rd MDP/man/getPolicy.Rd MDP/man/getPolicyW.Rd MDP/man/getStrIdxS.Rd MDP/man/getTransPr.Rd MDP/man/getWIdx.Rd MDP/man/hmpMDPWriter.Rd MDP/man/hypergf.Rd MDP/man/info.Rd MDP/man/loadMDP.Rd MDP/man/policyIteAve.Rd MDP/man/policyIteDiscount.Rd MDP/man/removeAction.Rd MDP/man/resetActions.Rd MDP/man/setActionWeight.Rd MDP/man/setPolicy.Rd MDP/man/setPolicyAction.Rd MDP/man/setStateWeight.Rd MDP/man/stateIdxDf.Rd MDP/man/stateIdxMat.Rd MDP/man/transProbMat.Rd MDP/man/valueIte.Rd MDP/man/weightNames.Rd
MDP/src
MDP/src/Makevars
MDP/src/basicdt.hh
MDP/src/debug.h
MDP/src/hgfreader.cc
MDP/src/hgfreader.hh
MDP/src/hmdp.cc
MDP/src/hmdp.hh
MDP/src/hmdpReader.cc
MDP/src/hmdpReader.hh
MDP/src/htacyclic.cc
MDP/src/htacyclic.hh
MDP/src/hypergf.cc
MDP/src/hypergf.hh
MDP/src/matalg.hh
MDP/src/matrix.hh
MDP/src/rmain.cc
MDP/src/time.hh
MDP/tests
MDP/tests/binaryMDPWriter.Rex
MDP/tests/hmpMDPWriter.Rex
MDP/tests/infoMat.Rex
MDP/tests/machine.Rex
MDP/tests/sow.Rex

Questions? Problems? Suggestions? or email at ian@mutexlabs.com.

All documentation is copyright its authors; we didn't write any of that.