eval.pact.cv: Evaluation functions for cross-validated predictions
In pact: Predictive Analysis of Clinical Trials

Description Usage Arguments Details Value Author(s) Examples

Methods for the evaluation of the cross-validated predictive scores obtained from pact.cv

1 2	eval.pact.cv(out.cv, method = c("discrete", "continuous"), g = log(1), plot.score = TRUE, plot.time = NULL, perm.test = FALSE, nperm = 100)

`out.cv`	The object from `pact.cv`
`method`	The evaluation method. Currently two options, `method='discrete'` or `method='continuous'`, are available. See 'Details'.
`g`	The cut-point for grouping scores into subsets 'benefit' and 'no benefit' from new treatment. Ignored for `method='continuous'`.
`plot.score`	Used only for plots if `method='continuous'` is chosen for survival response. Logical representing whether survival curves at specific quantiles of cross-validated scores are to be drawn. See 'Details'.
`plot.time`	Used only for plots if `method='continuous'` is chosen for survival response. Probability of survival greater than `plot.time` is plotted as a function of cross-validated score and Treatment. See 'Details'.
`perm.test`	Logical. If `perm.test=TRUE`, a permutation based test of significance is conducted for statistics computed from the cross-validated scores. See 'Value' and the package vignette and for more details on the permutation tests.
`nperm`	The number of permutations for the permutation test. Ignored if `perm.test=FALSE`

Currently two methods are defined for the evaluation of the scores obtained from pact.cv. In method='discrete' a user specified cut-off score is used to classify the subjects into groups 'benefit' or 'do not benefit' from new treatment. In each of the 'benefit' and 'do not benefit' groups the actual responses in the control (C) and the experimental (E) groups are compared. For the 'cox' family, the 'score' for a subject represents the predicted change in the log hazard when the subject is treated with E as against C (with lower values denoting benefit with E). In the case of the 'binomial' family, the 'score' represents the predicted change in the log odds of a response when the subject is treated with E as against C (with higher values denoting benefit with E). For the 'cox' family, examples of the cut-point g could be g=log(1) with score < g meaning benefit with E. Or one could be more stringent and have g correspond to a 30% reduction in hazard (g=log(0.70)). For the 'binomial' family, g=log(1.20) with score > g meaning sensitive to E would mean that subjects predicted to receive at least 20% increase in odds of response with E are classified as benefitting from E.

In method='continuous' no cut-off is applied to the cross-validated scores. A Cox proportional hazards (PH) regression or a logistic regression model (respectively for 'survival' and 'binary' response) is then developed that includes the main effect of treatment, main effect of cross-validated score, and treatment*score interaction. For survival response, this model is used to generate the Kaplan Meier survival curves for each treatment at the at 20th, 40th, 60th and 80th percentiles of predictive scores (plot.score = TRUE). The model is also used to compute the estimated probability of surviving beyond a landmark time specified in plot.time as a function of treatment and (cross-validated) score (if plot.time = NULL, this plot is not produced). For binary response, the output from evaluation is a plot of the probability of response as a functions of the predictive score and Treatment.

If perm.test=TRUE, permutation based significance tests are performed on appropriate test statistics and p-values are computed. See 'Value' and the package vignette and for more details on the permutation tests.

The return object is of class eval.cv and is a list whose components depend on the family ('cox' or 'binomial') and the chosen evaluation method ('continuous' or 'discrete')

`LR.Benefit`	For `family='cox'` and `method='discrete'`. The log-rank statistic for the survival difference between E and C for the 'benefit' from E group.
`LR.NoBenefit`	For `family='cox'` and `method='discrete'`. The log-rank statistic for the survival difference between E and C for the 'do not benefit' from E group.
`RR.T.Benefit`	For `family='binomial'` and `method='discrete'`. The response rate for subjects getting E in the 'benefit' from E group.
`RR.C.Benefit`	For `family='binomial'` and `method='discrete'`. The response rate for subjects getting C in the 'benefit' from E group
`RR.T.NoBenefit`	For `family='binomial'` and `method='discrete'`. The response rate for subjects getting E in the 'do not benefit' from E group
`RR.C.NoBenefit`	For `family='binomial'` and `method='discrete'`. The response rate for subjects getting C in the 'do not benefit' from E group

`pval.Benefit`	If `perm.test=TRUE`, p-value from permutation test. For `family='cox'` and `method='discrete'`, permutation based p-value for LR.Benefit. For `family='binomial'` and `method='discrete'`, permutation based p-value for difference in response rates for E and C for the subset predicted 'benefit' from E.
`pval.NoBenefit`	If `perm.test=TRUE`, p-value from permutation test. For `family='cox'` and `method='discrete'`, permutation based p-value for LR.NoBenefit. For `family='binomial'` and `method='discrete'`, permutation based p-value for difference in response rates for E and C for the subset predicted 'no benefit' from E.
`reg`	For `method='continuous'`, the regression model with treatment, predictive score and treatment x predictive score interaction
`pval.twosided`	For `method='continuous'`. Two-sided (non-directional) permutation based p-value for the treatment x predictive score interaction coefficient
`pval.onesided`	For `method='continuous'`. One-sided (directional, greater) permutation based p-value for the treatment x predictive score interaction coefficient
`call`	The function call

Additional plots for both method='discrete' as well as method='continuous'. print method is available for a nice display of objects of class eval.cv. See package vignette.

Jyothi Subramanian and Richard Simon
Maintainer: Jyothi Subramanian <subramanianj01@gmail.com>

### Survival response
data(prostateCancer)
Y <- prostateCancer[,3:4]
Xf <- prostateCancer[,7:8]
Xv <- prostateCancer[,c(5:6,9)]
Treatment <- as.factor(prostateCancer[,2])
p <- pact.fit(Y=Y, Xf=Xf, Xv=Xv, Treatment=Treatment, family="cox", varSelect="univar")
cv <- pact.cv(p, nfold=5)
## Not run: eval.pact.cv(cv, method="discrete", g=log(0.80), perm.test=TRUE, nperm=500)  ## At least 20% predicted reduction in HR classified as 'sensitive'
eval.pact.cv(cv, method="continuous", plot.score=TRUE, perm.test=FALSE)

### Binary response
data(EORTC10994)
Y <- as.factor(EORTC10994[,4])
Xv <- EORTC10994[,c(2,5:7)]
Treatment <- as.factor(EORTC10994[,3])
p <- pact.fit(Y=Y,Xv=Xv,Treatment=Treatment,family="binomial", varSelect="univar")
cv <- pact.cv(p, nfold=5)
## Not run: eval.pact.cv(cv, method="discrete", g=log(1), perm.test=TRUE, nperm=500)