Q-learning: first-stage regression

Description

Regresses the predicted future outcome maximized over a2 on first-stage history and treatment to estimate the optimal first-stage decision rule using Q-learning.

Usage

1
2
3
4
5
6
7
qLearnS1(object, ...)

## S3 method for class 'formula'
qLearnS1(formula, data, treatName, intNames,
qS2object, ...)
## Default S3 method:
qLearnS1(object, H1q, A1, s1ints, ...)

Arguments

formula

right-hand sided stage 1 regression formula

data

data frame containing variables used in formula

treatName

character string indicating the stage 1 treatment name

intNames

vector of characters indicating the names of the variables that interact with the stage 1 treatment in the regression model

qS2object

object of type qLearnS2

object

object of type qLearnS2

H1q

matrix or data frame of first-stage covariates to include as main effects in the linear model

A1

vector of first-stage randomized treatments

s1ints

indices pointing to columns of H1q that should be included as treatment interaction effects in the linear model

...

other arguments to be passed to lm()

Details

Fits a model of the form

E (Ytilde | H1, A1) = H10^Tβ10 + A1*H11^Tβ11,

where H10 and H11 are summaries of H1. For an object of type qLearnS1, summary(object) and plot(object) can be used for evaluating model fit.

Value

betaHat10

estimated main effect coeffients, beginning with the intercept

betaHat11

estimated treatment interaction coefficients, beginning with the main effect of treatment

optA1

vector of estimated optimal first-stage treatments for the patients in the training data

s1Fit

lm() object of the first-stage regression fit

s1ints

indices of variables in H1q included as treatment interactions in the model; same as input s1ints

Author(s)

Kristin A. Linn <kalinn@ncsu.edu>, Eric B. Laber, Leonard A. Stefanski

References

Linn, K. A., Laber, E. B., Stefanski, L. A. (2015) "iqLearn: Interactive Q-Learning in R", Journal of Statistical Software, 64(1), 1–25.

Laber, E. B., Linn, K. A., and Stefanski, L. A. (2014) "Interactive model building for Q-learning", Biometrika, 101(4), 831-847.

See Also

summary.qLearnS2, plot.qLearnS2

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
## load in two-stage BMI data
data (bmiData)
bmiData$A1[which (bmiData$A1=="MR")] = 1
bmiData$A1[which (bmiData$A1=="CD")] = -1
bmiData$A2[which (bmiData$A2=="MR")] = 1
bmiData$A2[which (bmiData$A2=="CD")] = -1
bmiData$A1 = as.numeric (bmiData$A1)
bmiData$A2 = as.numeric (bmiData$A2)
s1vars = bmiData[,1:4]
s2vars = bmiData[,c (1, 3, 5)]
a1 = bmiData[,7]
a2 = bmiData[,8]
## define response y to be the negative 12 month change in BMI from
## baseline 
y = -(bmiData[,6] - bmiData[,4])/bmiData[,4]
s2ints = c (2, 3)
## second-stage regression
fitQ2 = qLearnS2 (s2vars, y, a2, s2ints)
fitQ2 = qLearnS2 (y ~ gender + parent_BMI + month4_BMI +
  A2*(parent_BMI + month4_BMI), data=bmiData, "A2", c("parent_BMI",
                                  "month4_BMI")) 
## first-stage regression                                   
fitQ1 = qLearnS1 (fitQ2, s1vars, a1, c(3,4))
fitQ1 = qLearnS1 (~ gender + race + parent_BMI + baseline_BMI +
  A1*(gender + parent_BMI), data=bmiData, "A1", c ("gender",
                              "parent_BMI"), fitQ2)

Want to suggest features or report bugs for rdrr.io? Use the GitHub issue tracker.