binaryMDPWriter: Function for writing an HMDP model to binary files. The...

In MDP: Markov Decision Processes (MDPs) in R

Description

Binary files are efficent for storing large models. Compared to the HMP (XML) format the binary files use less storage space and loading the model is faster.

Usage

  binaryMDPWriter(prefix="",
    binNames=c("stateIdx.bin","stateIdxLbl.bin","actionIdx.bin",
    "actionIdxLbl.bin","actionWeight.bin","actionWeightLbl.bin","transProb.bin"))

Arguments

prefix	A character string with the prefix added to binNames.
binNames	A character vector of length 7 giving the names of the binary files storing the model.

Details

The functions which can be used are:

• setWeights(labels, ...): Set the labels of the weights used in the actions. labels is a vector of label names, ... are not used. The function must be called before starting building the model.

• process(): Starts a (sub)process.

• endProcess(): Ends a (sub)process.

• stage(label=NULL): Starts a stage. Currently label are not used in the binary format.

• endStage(): Ends a (sub)process.

• state(label=NULL): Starts a state. Returns (invisible) the states index number sIdx.

• endState(): Ends a stage.

• action(label=NULL, weights, prob, ...): Starts an action. Parameter weights must be a vector of action weights, prob must contain triples of (scope,idx,pr) (see the description of actionIdx.bin below), ... is currently not used.

• endAction(): Ends an action.

• closeWriter(): Close the writer. Must be called when the model description has finished.

Seven binary files are created using the following format:

• stateIdx.bin: File of integers containing the indexes defining all states in the format "n0 s0 -1 n0 s0 a0 n1 s1 -1 n0 s0 a0 n1 s1 a1 n2 s2 -1 n0 s0 ...". Here -1 is used to indicate that a new state is considered (new line).

• stateIdxLbl.bin: File of characters in the format "sIdx label sIdx label ..." Here sIdx corresponds to the index/line number in stateIdxLbl.bin (index starts from 0). Note no delimiter is used.

• actionIdx.bin: File of integers containing the indexes defining all actions in the format "sIdx scope idx scope idx scope idx -1 sIdx scope idx scope idx -1 sIdx scope -1 ...". sIdx corresponds to the index/line number in stateIdx.bin (index starts from 0). Next pairs (scope idx) will follow indicating the possible transitions. Scope can be 4 values: 2 - A transition to a child process (stage zero in the child process), 1 - A transition to next stage in the current process, 0 - A transition to the next stage in the father process. Here idx in the pair denote the index of the state at the stage considered, e.g. if scope=1 and idx=2 we consider state number 3 at next stage in the current process. Finally, if scope = 3 then a transition to a state specified by it's state sIdx is given. That is, if scope=3 and idx=5 then we have a transition to the state specified at line 6 in stateIdxLbl.bin. This is usefull when considering shared child processes.

• actionIdxLbl.bin: File of characters in the format "aIdx label aIdx label ..." Here aIdx corresponds to the index/line number in actionIdx.bin (index starts from 0). Note no delimiter is used.

• actionWeight.bin: File of doubles containing the weights of the actions in the format "c1 c2 c3 c1 c2 c3 ..." assuming three weights for each action.

• actionWeightLbl.bin: File of characters containing the labels of the weights in the format "lable1 label2 label3" assuming three weights for each action.

• transProb.bin: File of doubles containing the probabilities of the transitions defined in actions in actionIdx.bin. The format is "p1 p2 p3 -1 p1 -1 p1 p2 -1 ...". Here -1 is used to indicate that a new action is considered (new line).

Value

A list of functions.

Note

Note all indexes are starting from zero (C/C++ style).

Author(s)

Lars Relund [e-mail]

Examples

# Create a small HMDP with two levels
w<-binaryMDPWriter()
w$setWeights(c("Duration","Net reward","Items"))
w$process()
	w$stage()
		w$state(label="M0")
			w$action(label="A0",weights=c(0,0,0),prob=c(2,0,1))
				w$process()
					w$stage()
						w$state(label="D")
							w$action(label="A0",weights=c(0,0,1),prob=c(1,0,0.5,1,1,0.5))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(0,0,0),prob=c(1,0,1))
							w$endAction()
							w$action(label="A1",weights=c(1,2,1),prob=c(1,0,0.5,1,1,0.5))
							w$endAction()
						w$endState()
						w$state(label="C1")
							w$action(label="A0",weights=c(0,0,0),prob=c(1,0,1))
							w$endAction()
							w$action(label="A1",weights=c(1,2,1),prob=c(1,0,0.5,1,1,0.5))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
						w$endState()
						w$state(label="C1")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
						w$endState()
					w$endStage()
				w$endProcess()
			w$endAction()
			w$action(label="A1",weights=c(0,0,0),prob=c(2,0,1))
				w$process()
					w$stage()
						w$state(label="D")
							w$action(label="A0",weights=c(0,0,1),prob=c(1,0,1))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(0,0,0),prob=c(1,0,1))
							w$endAction()
							w$action(label="A1",weights=c(1,2,1),prob=c(1,0,0.5,1,1,0.5))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
						w$endState()
						w$state(label="C1")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
							w$action(label="A1",weights=c(0,10,5),prob=c(0,0,0.5,0,1,0.5))
							w$endAction()
						w$endState()
					w$endStage()
				w$endProcess()
			w$endAction()
		w$endState()
		w$state(label="M1")
			w$action(label="A0",weights=c(0,0,0),prob=c(2,0,1))
				w$process()
					w$stage()
						w$state(label="D")
							w$action(label="A0",weights=c(0,0,1),prob=c(1,0,0.5,1,1,0.5))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(0,0,0),prob=c(1,0,1))
							w$endAction()
						w$endState()
						w$state(label="C1")
							w$action(label="A0",weights=c(0,0,0),prob=c(1,0,1))
							w$endAction()
						w$endState()
					w$endStage()
					w$stage()
						w$state(label="C0")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
						w$endState()
						w$state(label="C1")
							w$action(label="A0",weights=c(1,4,0),prob=c(0,0,1))
							w$endAction()
						w$endState()
					w$endStage()
				w$endProcess()
			w$endAction()
		w$endState()
	w$endStage()
w$endProcess()
w$closeWriter()

stateIdxDf()
actionIdxDf()
actionWeightMat()
transProbMat()
a<-actionInfo()
a                   # ordered by action id
a[order(a$sId),]    # ordered by state id

MDP documentation built on May 2, 2019, 6:48 p.m.