estimate_counterfactual_ml: Estimate counterfactual performance of another algorithm

In factoryofthesun/r-IVaps: Treatment Effect Estimation Using Data From Algorithm-Based Recommendations

Description

Estimate counterfactual performance of another algorithm

Usage

estimate_counterfactual_ml(
  data = NULL,
  aps = NULL,
  Y = NULL,
  Z = NULL,
  ml = NULL,
  mlnew = NULL,
  degen = c(0, 1),
  apslab = "APS",
  ylab = "Y",
  zlab = "Z",
  mllab = "ML1",
  mlnewlab = "ML2",
  verbose = T
)

Arguments

data	Dataset containing APS, Y, Z, the original ML algorithm predictions, and the counterfactual ML' algorithm predictions.
aps	Vector of APS values.
Y	Vector of outcome values.
Z	Vector of ML recommendation values.
ml	Vector of ML predictions.
mlnew	Vector of counterfactual ML predictions.
degen	Vector of values for which APS is degenerate. Defaults to c(0,1).
apslab	Column name of APS variable. Defaults to "APS".
ylab	Column name of Y variable. Defaults to "Y".
zlab	Column name of Z variable. Defaults to "Z".
mllab	Column name of original ML prediction variable. Defaults to "ML1".
mlnewlab	Column name of counterfactual ML prediction variable. Defaults to "ML2".
verbose	Boolean indicator for whether to print summary output of estimation. Defaults to True.

Details

The process of estimating counterfactual value works as follows. First we we fit the below OLS regression using historical recommendations and outcome Z and Y.

Y_i = β_0 + β_1 Z_i + β_2 p^s(X_i; δ) + ε_i

where β_1 is our estimated effect of treatment recommendation.

Then we take the original algorithm output ml and the counterfactual algorithm output mlnew and estimate the below value equation.

\hat{V}(ML') = \frac{1}{n} ∑_{i=1}^n (Y_i + \hat{β_{ols}}(ML'(X_i) - ML(X_i)))

Value

List containing counterfactual predictions and fitted lm model object.

Examples

library(mlr3)
data("iris")
assign_cutoff <- function(X, cutoff){
  ret <- as.integer(X > cutoff)
  return(ret)
}
test_data <- iris
model <- lm(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data=test_data)
ml <- predict(model, test_data)
task_iris <- TaskRegr$new(id="iris", backend=test_data[,1:4], target="Sepal.Length")
learner = lrn("regr.rpart")
learner$train(task_iris)
mlnew <- predict(learner, test_data)
aps <- estimate_aps(test_data, model, xc = names(test_data)[2:4],
           infer=FALSE, s=400, delta=0.8, fcn=assign_cutoff, cutoff = 6)
# Can send counterfactual estimation inputs in two different ways
Y_val <- test_data$Sepal.Length
Z_val <- assign_cutoff(iris$Sepal.Length, 6)
test_data[, Y := Y_val]
test_data[, Z := Z_val]
test_data[, ml := ml]
test_data[, ml_new := mlnew]
test_data[, APS := aps]
estimate_counterfactual_ml(aps = aps, Y = Y_val, Z = Z_val, ml = ml, mlnew = mlnew)
estimate_counterfactual_ml(test_data, apslab = "APS", ylab = "Y", zlab = "Z",
                      mllab = "ml", mlnewlab = "ml_new")

factoryofthesun/r-IVaps documentation built on Dec. 20, 2021, 7:41 a.m.