Q-learning: Recommend stage 1 treatment

Share:

Description

Recommends the Q-learning estimated optimal first-stage treatment for a given stage 1 history, h1.

Usage

1
qLearnQ1(object, h1q)

Arguments

object

object of type qLearnS1

h1q

vector of observed first-stage main effects corresponding to the variables in H1q used in qLearnS1()

Details

Use the estimated optimal first-stage decision rule from qLearnS1() to recommend the best stage 1 treatment for a patient presenting with history h1q. It is essential that h1q include the same variables and ordering as H1q. If a formula was used to fit qLearnS1(), we recommend checking summary(<qLearnS1 object>) for the correct order of h1q.

Value

q1Pos

estimated value of the first-stage Q-function when H1=h1 and A1=1

q1Neg

estimated value of the first-stage Q-function when H1=h1 and A1=-1

q1opt

estimated optimal first-stage treatment for a patient presenting with h1

Author(s)

Kristin A. Linn <kalinn@ncsu.edu>, Eric B. Laber, Leonard A. Stefanski

References

Linn, K. A., Laber, E. B., Stefanski, L. A. (2015) "iqLearn: Interactive Q-Learning in R", Journal of Statistical Software, 64(1), 1–25.

Laber, E. B., Linn, K. A., and Stefanski, L. A. (2014) "Interactive model building for Q-learning", Biometrika, 101(4), 831-847.

See Also

qLearnS1, summary.qLearnS1, plot.qLearnS1

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
## load in two-stage BMI data
data (bmiData)
bmiData$A1[which (bmiData$A1=="MR")] = 1
bmiData$A1[which (bmiData$A1=="CD")] = -1
bmiData$A2[which (bmiData$A2=="MR")] = 1
bmiData$A2[which (bmiData$A2=="CD")] = -1
bmiData$A1 = as.numeric (bmiData$A1)
bmiData$A2 = as.numeric (bmiData$A2)
s1vars = bmiData[,1:4]
s2vars = bmiData[,c (1, 3, 5)]
a1 = bmiData[,7]
a2 = bmiData[,8]
## define response y to be the negative 12 month change in BMI from
## baseline 
y = -(bmiData[,6] - bmiData[,4])/bmiData[,4]
## second-stage regression
fitQ2 = qLearnS2 (y ~ gender + parent_BMI + month4_BMI +
  A2*(parent_BMI + month4_BMI), data=bmiData, "A2", c("parent_BMI",
  "month4_BMI"))                                 
## first-stage regression                                   
fitQ1 = qLearnS1 (~ gender + race + parent_BMI + baseline_BMI +
  A1*(gender + parent_BMI), data=bmiData, "A1", c ("gender",
                              "parent_BMI"), fitQ2)
summary (fitQ1)

h1q = c (1, 1, 35, 45);
optQ1 = qLearnQ1 (fitQ1, h1q);
optQ1$q1opt

Want to suggest features or report bugs for rdrr.io? Use the GitHub issue tracker.