projection: Defining a Belief Space Projection

In pomdp: Infrastructure for Partially Observable Markov Decision Processes (POMDP)

Defining a Belief Space Projection

Description

High dimensional belief spaces can be projected to lower dimension. This is useful for visualization and to analyze the belief space and value functions. This definition is used by functions like plot_belief_space(), plot_value_function(), and sample_belief_space().

Usage

projection(x = NULL, model)

Arguments

x	specification of the projection (see Details section).
model	a POMDP.

Details

The belief space is $n-1$ dimensional, were $n$ is the number of states. Note: it is n-1 dimensional since the probabilities need to add up to 1. A projection fixes the belief value for a set of states. For example, for a 4-state POMDP (s1, s2, s3, s4), we can project the belief space on s1 and s2 by holding s3 and s4 constant which is represented by the vector c(s1 = NA, s2 = NA, s3 = 0, s4 = .1). We use NA to represent that the values are not fixed and the value that the other dimensions are held constant.

We provide several ways to specify a projection:

• A vector with values for all dimensions. NAs are used for the dimension projected on. This is the canonical form used in this package. Example: c(NA, NA, 0, .1)

• A named vector with just the dimensions held constant. Example: c(s3 = 0, s4 = .1)

• A vector of state names to project on. All other dimensions are held constant at 0. Example: c("s1", "s2")

• A vector with indices of the states to project on. All other dimensions are held constant at 0. Example: c(1, 2)

Value

a canonical description of the projection.

Author(s)

Michael Hahsler

See Also

Other policy: estimate_belief_for_nodes(), optimal_action(), plot_belief_space(), plot_policy_graph(), policy(), policy_graph(), reward(), solve_POMDP(), solve_SARSOP(), value_function()

Other POMDP: MDP2POMDP, POMDP(), accessors, actions(), add_policy(), plot_belief_space(), reachable_and_absorbing, regret(), sample_belief_space(), simulate_POMDP(), solve_POMDP(), solve_SARSOP(), transition_graph(), update_belief(), value_function(), write_POMDP()

Examples

model <- POMDP(
 states = 4,
 actions = 2,
 observations = 2,
 transition_prob = list("identity","identity"),
 observation_prob = list("uniform","uniform"),
 reward = rbind(R_(value = 1))
)

projection(NULL, model = model)
projection(1:2, model = model)
projection(c("s2", "s3"), model = model)
projection(c(1,4), model = model)
projection(c(s2 = .4, s3 = .2), model = model)
projection(c(s1 = .1, s2 = NA, s3 = NA, s4 = .3), model = model)

pomdp documentation built on May 29, 2024, 2:04 a.m.