qLearnS2: Q-learning: second-stage regression

In kalinn/iqLearn: Interactive Q-Learning

Q-learning: second-stage regression

Description

Same as learnIQ2. Estimates the optimal second-stage decision rule using a linear regression of the response on second-stage history and treatment.

Usage

qLearnS2(H2, ...)

## S3 method for class 'formula'
qLearnS2(formula, data, treatName, intNames, ...)
## Default S3 method:
qLearnS2(H2, Y, A2, s2ints, ...)

Arguments

formula	stage 2 regression formula
data	data frame containing variables used in formula
treatName	character string indicating the stage 2 treatment name
intNames	vector of characters indicating the names of the variables that interact with the stage 2 treatment in the regression model
H2	matrix or data frame of second-stage covariates to include as main effects in the linear model
Y	response vector
A2	vector of second-stage randomized treatments
s2ints	indices pointing to columns of H2 that should be included as treatment interaction effects in the linear model
...	other arguments to be passed to lm()

Details

Fits a model of the form

E (Y | H2, A2) = H20^Tβ20 + A2*H21^Tβ21,

where H20 and H21 are summaries of H2. For an object of type qLearnS2, summary(object) and plot(object) can be used for evaluating model fit.

Value

betaHat20	estimated main effect coefficients; first is the intercept
betaHat21	estimated treatment interaction coefficients; first is the main effect of the second-stage treatment
Ytilde	Q2 function maximized over treatment a2; the predicted future outcome assuming optimal treatment is given at the second stage to be used in the next step of the Q-learning algorithm
optA2	vector of estimated optimal second-stage treatments for the patients in the training data
s2Fit	lm() object of the second-stage regression fit
s2ints	indicies of variables in H2 included as treatment interactions in the model; same as input s2ints

Author(s)

Kristin A. Linn <[e-mail]>, Eric B. Laber, Leonard A. Stefanski

References

Linn, K. A., Laber, E. B., Stefanski, L. A. (2015) "iqLearn: Interactive Q-Learning in R", Journal of Statistical Software, 64(1), 1–25.

Laber, E. B., Linn, K. A., and Stefanski, L. A. (2014) "Interactive model building for Q-learning", Biometrika, 101(4), 831-847.

See Also

summary.qLearnS2, plot.qLearnS2

Examples

## load in two-stage BMI data
data (bmiData)
bmiData$A1[which (bmiData$A1=="MR")] = 1
bmiData$A1[which (bmiData$A1=="CD")] = -1
bmiData$A2[which (bmiData$A2=="MR")] = 1
bmiData$A2[which (bmiData$A2=="CD")] = -1
bmiData$A1 = as.numeric (bmiData$A1)
bmiData$A2 = as.numeric (bmiData$A2)
s1vars = bmiData[,1:4]
s2vars = bmiData[,c (1, 3, 5)]
a1 = bmiData[,7]
a2 = bmiData[,8]
## define response y to be the negative 12 month change in BMI from
## baseline 
y = -(bmiData[,6] - bmiData[,4])/bmiData[,4]
s2ints = c (2, 3)
## second-stage regression
fitQ2 = qLearnS2 (s2vars, y, a2, s2ints)
fitQ2 = qLearnS2 (y ~ gender + parent_BMI + month4_BMI +
  A2*(parent_BMI + month4_BMI), data=bmiData, "A2", c("parent_BMI",
                                  "month4_BMI"))
                                   
summary (fitQ2)
plot (fitQ2)

kalinn/iqLearn documentation built on Aug. 4, 2022, 10:15 p.m.