qlearningupdate: qlearningupdate
In QLearning: Reinforcement Learning using the Q Learning Algorithm

Description Usage Arguments Details Value Note Author(s) References Examples

This repository implements Q-Learning, a model-free form of reinforcement learning in R.

1 2	qlearningupdate(q, currentstate, currentaction, currentreward, nextstate=NULL, rewardcount=.5, gamma=.25)

`q`	Input state/action matrix.
`currentstate`	Current state of the game. Does not have to match any of the state for q.
`currentaction`	Action to take.
`currentreward`	Reward for currentaction in current iteration.
`nextstate`	State that the game is in after taking currentaction.
`rewardcount`	Regularization constant for reward.
`gamma`	Learning rate constant for Q-Learning.

For internal use for qlearn.

An updated state/action matrix.

Contact at liam.bressler@yale.edu

Liam Bressler

http://labressler.github.io/analytics

cardgame <- function()
{
  playercards <- sample(1:8,4) #distribute the cards, we're player one
  ourcard <- playercards[1] #our card
  playertotals <- rep(-1,4) #including the antes
  playersinpot <- vector()
  for (player in 2:4) #other 3 players go first
  {
    if (playercards[player]>=2)
    {
      playertotals[player] <- (-3)
      playersinpot <- append(playersinpot,player)
    }
  }
  #the next line is where we want to choose our action
  player1 <- 'Choose'
  if (player1=="Call")
  {
    playertotals[1] <- (-3)
    playersinpot <- append(playersinpot,1)
  }
  potsize <- -1*(sum(playertotals)) #the amount in the pot is how much the players put in
  playercards[!(1:4 %in% playersinpot)] <- 0 #get rid of everyone who folded
  winner <- which.max(playercards) #winner is the person with the highest card who didn't fold
  playertotals[winner] <- playertotals[winner]+potsize
  return(playertotals[1]) #return how much we won
}

strat <- qlearn(game="cardgame",statevars="ourcard",possibleactions=c("Call","Fold"),
  playername="player1",numiter=25000) #make sure each function and variable name is a string

strat <- qlearningupdate(strat,currentstate=7,currentaction="Call",currentreward=5)
#Update the matrix after an example when we call with the 7 card as our state, winning 5 chips