auroc: Area Under the Curve (AUC) and Receiver Operating...
In mixOmics: Omics Data Integration Project

Description Usage Arguments Details Value Author(s) See Also Examples

Calculates the AUC and plots ROC for supervised objects from s/plsda, mint.s/plsda and block.plsda, block.splsda or wrapper.sgccda.

## S3 method for class 'plsda'
auroc(object, newdata = object$input.X, outcome.test = as.factor(object$Y),
multilevel = NULL, plot = TRUE, roc.comp = 1, ...)

## S3 method for class 'splsda'
auroc(object, newdata = object$input.X, outcome.test = as.factor(object$Y),
multilevel = NULL, plot = TRUE, roc.comp = 1, ...)

## S3 method for class 'mint.plsda'
auroc(object, newdata = object$X, outcome.test = as.factor(object$Y),
study.test = object$study, multilevel = NULL,plot = TRUE, roc.comp = 1,
roc.study = "global", ...)

## S3 method for class 'mint.splsda'
auroc(object, newdata = object$X, outcome.test = as.factor(object$Y),
study.test = object$study, multilevel = NULL,plot = TRUE, roc.comp = 1,
roc.study = "global", ...)

## S3 method for class 'sgccda'
auroc(object, newdata = object$X, outcome.test = as.factor(object$Y),
multilevel = NULL,plot = TRUE, roc.block = 1, roc.comp = 1, ...)

`object`	Object from one of the following supervised analysis class: "plsda", "splsda", "mint.plsda", "mint.splsda", "block.splsda" or "wrapper.sgccda"
`newdata`	numeric matrix of predictors, by default set to the training data set (see details).
`outcome.test`	Either a factor or a class vector for the discrete outcome, by default set to the outcome vector from the training set (see details).
`study.test`	For MINT objects, grouping factor indicating which samples of `newdata` are from the same study. Overlap with `object$study` are allowed.
`multilevel`	Sample information when a newdata matrix is input and when multilevel decomposition for repeated measurements is required. A numeric matrix or data frame indicating the repeated measures on each individual, i.e. the individuals ID. See examples in `splsda`.
`plot`	Whether the ROC curves should be plotted, by default set to TRUE (see details).
`roc.comp`	Specify the component (integer) for which the ROC will be plotted from the multivariate model, default to 1.
`roc.block`	Specify the block number (integer) or the name of the block (set of characters) for which the ROC will be plotted for a block.plsda or block.splsda object, default to 1.
`roc.study`	Specify the study for which the ROC will be plotted for a mint.plsda or mint.splsda object, default to "global".
`...`	external optional arguments for plotting

For more than two classes in the categorical outcome Y, the AUC is calculated as one class vs. the other and the ROC curves one class vs. the others are output.

The ROC and AUC are calculated based on the predicted scores obtained from the predict function applied to the multivariate methods (predict(object)$predict). Our multivariate supervised methods already use a prediction threshold based on distances (see predict) that optimally determine class membership of the samples tested. As such AUC and ROC are not needed to estimate the performance of the model (see perf, tune that report classification error rates). We provide those outputs as complementary performance measures.

The pvalue is from a Wilcoxon test between the predicted scores between one class vs the others.

External independent data set (newdata) and outcome (outcome.test) can be input to calculate AUROC. The external data set must have the same variables as the training data set (object$X).

If newdata is not provided, AUROC is calculated from the training data set, and may result in overfitting (too optimistic results).

Note that for mint.plsda and mint.splsda objects, if roc.study is different from "global", then newdata), outcome.test and sstudy.test are not used.

Depending on the type of object used, a list that contains: The AUC and Wilcoxon test pvalue for each 'one vs other' classes comparison performed, either per component (splsda, plsda, mint.plsda, mint.splsda), or per block and per component (wrapper.sgccda, block.plsda, blocksplsda). The ggplot output that can be changed by users by adding ggplot layers.

Benoit Gautier, Francois Bartolo, Florian Rohart

tune, perf, and http://www.mixOmics.org for more details.

## example with PLSDA, 2 classes
# ----------------
data(breast.tumors)
X <- breast.tumors$gene.exp
Y <- breast.tumors$sample$treatment

plsda.breast <- plsda(X, Y, ncomp = 2)
auc.plsda.breast = auroc(plsda.breast, ncomp = 1)

# changing plot title
p = auc.plsda.breast$graph.Comp1 #saving the ggplot graph in p
p+ggtitle("Something witty")

## example with sPLSDA
# -----------------
splsda.breast <- splsda(X, Y, ncomp = 2, keepX = c(25, 25))
auroc(plsda.breast, plot = FALSE)

## Not run: 

## example with sPLSDA with 4 classes
# -----------------
data(liver.toxicity)
X <- as.matrix(liver.toxicity$gene)
# Y will be transformed as a factor in the function,
# but we set it as a factor to set up the colors.
Y <- as.factor(liver.toxicity$treatment[, 4])

splsda.liver <- splsda(X, Y, ncomp = 2, keepX = c(20, 20))
auc.splsda.liver = auroc(splsda.liver, ncomp = 1)


## example with mint.plsda
# -----------------
data(stemcells)

res = mint.plsda(X = stemcells$gene, Y = stemcells$celltype, ncomp = 3,
study = stemcells$study)
auc.mint.pslda = auroc(res, plot = FALSE)

## example with mint.splsda
# -----------------
res = mint.splsda(X = stemcells$gene, Y = stemcells$celltype, ncomp = 3, keepX = c(10, 5, 15),
study = stemcells$study)
auc.mint.spslda = auroc(res, plot = TRUE, roc.comp = 3)


## example with block.plsda
# ------------------
data(nutrimouse)
data = list(gene = nutrimouse$gene, lipid = nutrimouse$lipid)
# with this design, all blocks are connected
design = matrix(c(0,1,1,0), ncol = 2, nrow = 2,
byrow = TRUE, dimnames = list(names(data), names(data)))

block.plsda.nutri = block.plsda(X = data, Y = nutrimouse$diet)
auc.block.plsda.nutri = auroc(block.plsda.nutri, block = 'lipid')

## example with block.splsda
# ---------------
list.keepX = list(gene = rep(10, 2), lipid = rep(5,2))
block.splsda.nutri = block.splsda(X = data, Y = nutrimouse$diet, keepX = list.keepX)
auc.block.splsda.nutri = auroc(block.splsda.nutri, block = 1)

## End(Not run)

Loading required package: MASS
Loading required package: lattice
Loading required package: ggplot2

Loaded mixOmics 6.3.2

Thank you for using mixOmics!

How to apply our methods: http://www.mixOmics.org for some examples.
Questions or comments: email us at mixomics[at]math.univ-toulouse.fr  
Any bugs? https://bitbucket.org/klecao/package-mixomics/issues
Cite us:  citation('mixOmics')
Warning messages:
1: In rgl.init(initValue, onlyNULL) : RGL: unable to open X11 display
2: 'rgl_init' failed, running with rgl.useNULL = TRUE 
3: .onUnload failed in unloadNamespace() for 'rgl', details:
  call: fun(...)
  error: object 'rgl_quit' not found 
$Comp1
           AUC   p-value
AF vs BE 0.863 2.473e-05

$Comp2
            AUC   p-value
AF vs BE 0.9981 7.124e-09

$Comp1
           AUC   p-value
AF vs BE 0.863 2.473e-05

$Comp2
            AUC   p-value
AF vs BE 0.9981 7.124e-09

$Comp1
                  AUC   p-value
6 vs Other(s)  0.8268 9.959e-05
18 vs Other(s) 0.8711 9.928e-06
24 vs Other(s) 0.8372 5.929e-05
48 vs Other(s) 0.7930 4.858e-04

$Comp2
                  AUC   p-value
6 vs Other(s)  0.9062 1.316e-06
18 vs Other(s) 0.9909 5.061e-09
24 vs Other(s) 0.9284 3.380e-07
48 vs Other(s) 0.9974 3.168e-09

$Comp1
                          AUC   p-value
Fibroblast vs Other(s) 0.9930 4.441e-16
hESC vs Other(s)       0.6361 1.658e-02
hiPS vs Other(s)       0.8258 3.671e-10

$Comp2
                          AUC   p-value
Fibroblast vs Other(s) 0.9979 2.220e-16
hESC vs Other(s)       0.8977 2.491e-12
hiPS vs Other(s)       0.9352 0.000e+00

$Comp3
                          AUC   p-value
Fibroblast vs Other(s) 0.9930 4.441e-16
hESC vs Other(s)       0.9493 2.442e-15
hiPS vs Other(s)       0.9655 0.000e+00

$Comp1
                          AUC   p-value
Fibroblast vs Other(s) 1.0000 2.220e-16
hESC vs Other(s)       0.7844 5.496e-07
hiPS vs Other(s)       0.8075 3.300e-09

$Comp2
                          AUC   p-value
Fibroblast vs Other(s) 1.0000 2.220e-16
hESC vs Other(s)       0.8096 4.992e-08
hiPS vs Other(s)       0.8024 5.994e-09

$Comp3
                          AUC   p-value
Fibroblast vs Other(s) 1.0000 2.220e-16
hESC vs Other(s)       0.8750 4.013e-11
hiPS vs Other(s)       0.8711 9.421e-13

$gene
$gene$comp1
                    AUC p-value
coc vs Other(s)  0.7461 0.03316
fish vs Other(s) 0.5586 0.61200
lin vs Other(s)  0.4688 0.78680
ref vs Other(s)  0.5742 0.52060
sun vs Other(s)  0.6445 0.21090

$gene$comp2
                    AUC p-value
coc vs Other(s)  0.7461 0.03316
fish vs Other(s) 0.7383 0.03915
lin vs Other(s)  0.4961 0.97300
ref vs Other(s)  0.5820 0.47770
sun vs Other(s)  0.7500 0.03046


$lipid
$lipid$comp1
                    AUC   p-value
coc vs Other(s)  1.0000 1.505e-05
fish vs Other(s) 0.4570 7.099e-01
lin vs Other(s)  0.4336 5.654e-01
ref vs Other(s)  0.6094 3.438e-01
sun vs Other(s)  1.0000 1.505e-05

$lipid$comp2
                    AUC   p-value
coc vs Other(s)  0.9922 2.041e-05
fish vs Other(s) 0.7422 3.605e-02
lin vs Other(s)  0.7305 4.605e-02
ref vs Other(s)  0.6602 1.656e-01
sun vs Other(s)  1.0000 1.505e-05


Warning messages:
1: In data.row.names(row.names, rowsi, i) :
  some row.names duplicated: 41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80 --> row.names NOT used
2: In data.row.names(row.names, rowsi, i) :
  some row.names duplicated: 41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80 --> row.names NOT used
$gene
$gene$comp1
                    AUC   p-value
coc vs Other(s)  0.9727 4.289e-05
fish vs Other(s) 0.7227 5.394e-02
lin vs Other(s)  0.5625 5.885e-01
ref vs Other(s)  0.5391 7.353e-01
sun vs Other(s)  0.6484 1.988e-01

$gene$comp2
                    AUC   p-value
coc vs Other(s)  0.9727 4.289e-05
fish vs Other(s) 0.8125 6.830e-03
lin vs Other(s)  0.6719 1.368e-01
ref vs Other(s)  0.5508 6.603e-01
sun vs Other(s)  0.9258 2.282e-04


$lipid
$lipid$comp1
                    AUC   p-value
coc vs Other(s)  0.9805 3.197e-05
fish vs Other(s) 0.7109 6.787e-02
lin vs Other(s)  0.7305 4.605e-02
ref vs Other(s)  0.4531 6.849e-01
sun vs Other(s)  0.5859 4.569e-01

$lipid$comp2
                    AUC   p-value
coc vs Other(s)  1.0000 1.505e-05
fish vs Other(s) 0.7422 3.605e-02
lin vs Other(s)  0.8750 1.170e-03
ref vs Other(s)  0.7461 3.316e-02
sun vs Other(s)  1.0000 1.505e-05


Warning messages:
1: In data.row.names(row.names, rowsi, i) :
  some row.names duplicated: 41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80 --> row.names NOT used
2: In data.row.names(row.names, rowsi, i) :
  some row.names duplicated: 41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80 --> row.names NOT used