qLearnS1: Q-learning: first-stage regression

In iqLearn: Interactive Q-Learning

Description

Regresses the predicted future outcome maximized over a2 on first-stage history and treatment to estimate the optimal first-stage decision rule using Q-learning.

Usage

qLearnS1(object, ...)

## S3 method for class 'formula'
qLearnS1(formula, data, treatName, intNames,
qS2object, ...)
## Default S3 method:
qLearnS1(object, H1q, A1, s1ints, ...)

Arguments

formula	right-hand sided stage 1 regression formula
data	data frame containing variables used in formula
treatName	character string indicating the stage 1 treatment name
intNames	vector of characters indicating the names of the variables that interact with the stage 1 treatment in the regression model
qS2object	object of type qLearnS2
object	object of type qLearnS2
H1q	matrix or data frame of first-stage covariates to include as main effects in the linear model
A1	vector of first-stage randomized treatments
s1ints	indices pointing to columns of H1q that should be included as treatment interaction effects in the linear model
...	other arguments to be passed to lm()

Details

Fits a model of the form

E (Ytilde | H1, A1) = H10^Tβ10 + A1*H11^Tβ11,

where H10 and H11 are summaries of H1. For an object of type qLearnS1, summary(object) and plot(object) can be used for evaluating model fit.

Value

betaHat10	estimated main effect coeffients, beginning with the intercept
betaHat11	estimated treatment interaction coefficients, beginning with the main effect of treatment
optA1	vector of estimated optimal first-stage treatments for the patients in the training data
s1Fit	lm() object of the first-stage regression fit
s1ints	indices of variables in H1q included as treatment interactions in the model; same as input s1ints

Author(s)

Kristin A. Linn <[e-mail]>, Eric B. Laber, Leonard A. Stefanski

References

Linn, K. A., Laber, E. B., Stefanski, L. A. (2015) "iqLearn: Interactive Q-Learning in R", Journal of Statistical Software, 64(1), 1–25.

Laber, E. B., Linn, K. A., and Stefanski, L. A. (2014) "Interactive model building for Q-learning", Biometrika, 101(4), 831-847.

See Also

summary.qLearnS2, plot.qLearnS2

Examples

## load in two-stage BMI data
data (bmiData)
bmiData$A1[which (bmiData$A1=="MR")] = 1
bmiData$A1[which (bmiData$A1=="CD")] = -1
bmiData$A2[which (bmiData$A2=="MR")] = 1
bmiData$A2[which (bmiData$A2=="CD")] = -1
bmiData$A1 = as.numeric (bmiData$A1)
bmiData$A2 = as.numeric (bmiData$A2)
s1vars = bmiData[,1:4]
s2vars = bmiData[,c (1, 3, 5)]
a1 = bmiData[,7]
a2 = bmiData[,8]
## define response y to be the negative 12 month change in BMI from
## baseline 
y = -(bmiData[,6] - bmiData[,4])/bmiData[,4]
s2ints = c (2, 3)
## second-stage regression
fitQ2 = qLearnS2 (s2vars, y, a2, s2ints)
fitQ2 = qLearnS2 (y ~ gender + parent_BMI + month4_BMI +
  A2*(parent_BMI + month4_BMI), data=bmiData, "A2", c("parent_BMI",
                                  "month4_BMI")) 
## first-stage regression                                   
fitQ1 = qLearnS1 (fitQ2, s1vars, a1, c(3,4))
fitQ1 = qLearnS1 (~ gender + race + parent_BMI + baseline_BMI +
  A1*(gender + parent_BMI), data=bmiData, "A1", c ("gender",
                              "parent_BMI"), fitQ2)

iqLearn documentation built on May 2, 2019, 6:44 a.m.