makeEnvironment: Create reinforcement learning environment.

Description Usage Arguments Details Value Methods Environments Examples

Description

This function creates an environment for reinforcement learning.

Usage

1
makeEnvironment(class = "custom", discount = 1, ...)

Arguments

class

[character(1)]
Class of environment. One of c("custom", "mdp", "gym", "gridworld").

discount

[numeric(1) in (0, 1)]
Discount factor.

...

[any]
Arguments passed on to the specific environment.

Details

Use the step method to interact with the environment.

Note that all states and actions are numerated starting with 0!

For a detailed explanation and more examples have a look at the vignette "How to create an environment?".

Value

R6 class of class Environment.

Methods

Environments

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
step = function(self, action) {
  state = list(mean = action + rnorm(1), sd = runif(1))
  reward = rnorm(1, state[[1]], state[[2]])
  done = FALSE
  list(state, reward, done)
}

reset = function(self) {
  state = list(mean = 0, sd = 1)
  state
}

env = makeEnvironment(step = step, reset = reset, discount = 0.9)
env$reset()
env$step(100)

# Create a Markov Decision Process.
P = array(0, c(2, 2, 2))
P[, , 1] = matrix(c(0.5, 0.5, 0, 1), 2, 2, byrow = TRUE)
P[, , 2] = matrix(c(0, 1, 0, 1), 2, 2, byrow = TRUE)
R = matrix(c(5, 10, -1, 2), 2, 2, byrow = TRUE)
env = makeEnvironment("mdp", transitions = P, rewards = R)

env$reset()
env$step(1L)

# Create a Gridworld.
grid = makeEnvironment("gridworld", shape = c(4, 4),
  goal.states = 15, initial.state = 0)
grid$visualize()

## Not run: 
# Create an OpenAI Gym environment.
# Make sure you have Python, gym and reticulate installed.
env = makeEnvironment("gym", gym.name = "MountainCar-v0")

# Take random actions for 200 steps.
env$reset()
for (i in 1:200) {
  action = sample(env$actions, 1)
  env$step(action)
  env$visualize()
}
env$close()

## End(Not run)

markusdumke/reinforcelearn documentation built on May 31, 2019, 8:48 p.m.