TOC: Theoretical FDR and sensitivity as a function of cutoff level

In OCplus: Operating characteristics plus sample size and local fdr for microarray experiments

Description

Computes and plots the operating characteristics for a two group microarray experiment based on a theoretical model. The false discovery rate (FDR) is plotted against the cutoff level on the t-statistic. Optionally, curves for the the classical significance level and sensitivity can be added. Different curves for different proportions of non-differentially expressed genes can be compared in the same plot, and the sample size per group can be varied between plots.

Usage

TOC(n = 10, p0 = 0.95, sigma = 1, D, F0, F1, n1 = n, n2 = n, paired = FALSE,
    plot = TRUE, local.show=FALSE, alpha.show = TRUE, sensitivity.show = TRUE,
	nplot = 100, xlim, ylim = c(0, 1), main, legend.show = FALSE, ...)

Arguments

n, n1, n2	number of samples per group, by default equal and specified via n, but can be set to different values via n1 and n2.
p0	the proportion of not differentially expressed genes, may be vector valued
sigma	the standard deviation for the log expression values
D	assumed average log fold change (in units of sigma), by default 1; this is a shortcut for specifying a simple symmetrical alternative hypothesis through F1.
F0	the distribution of the log2 expression values under the null hypothesis; by default, this is normal with mean zero and standard deviation sigma, but mixtures of normals can be specified, see Details and Examples.
F1	the distribution of the log2 expression values under the alternative hypothesis; by default, this is an equal mixture of two normals with means D and -D and standard deviation sigma; mixture of normals are again possible, see Details and Examples.
paired	logical value indicating whether two distinct groups of observations or one group of paired observations are studied.
plot	logical value indicating whether the results should be plotted.
local.show	logical value indicating whether to show local or global false discovery rate (default: global).
alpha.show	logical value indicating whether to show the classical significance level for testing one hypothesis as a function of the cutoff level.
sensitivity.show	logical value indicating whether to show the classical sensitivity for testing one hypothesis as a function of the cutoff level.
nplot	number of points that are evaluated for the curves
xlim	the usual limits on the horizontal axis
ylim	the usual limits on the vertical axis
main	the main title of the plot
legend.show	logical value indicating whether to show a legend for the different types of curves in the plot.
...	the usual graphical parameters, passed to plot

Details

This function plots the FDR as a function of the cutoff level when comparing the expression of multiple genes between two groups of subjects. We study a gene selection mechanism that declares all genes to be differentially expressed whose t-statistics have an absolute value greater than a specified cutoff value. The comparison is based on a two-sample t-statistic for equal variances, for either paired or unpaired observations.

The underlying model assumes that a proportion p0 of genes are not differentially expressed between groups, and that 1-p0 are. The logarithmized gene expression values are assumed to be generated by mixtures of normal distributions. Both null and alternative hypothesis are specified through the means of the respective mixture components; these means can be interpreted as average log2 fold changes in units of the standard deviation sigma.

Note that the model does not assume that all genes have the same standard deviation sigma, only that the mean log2 fold change for all regulated genes is proportional to their individual variability (standard deviation). sigma generally does not need to be specified explicitly and can be left at its default value of one, so that D can be interpreted straightforward as log2 fold change between groups.

The default null distribution of the log2 expression values is a single normal distribution with mean zero (and standard deviation sigma); the default alternative distribution is is an equal mixture of two normals with means D and -D (and again standard deviation sigma). However, general mixtures of normals can be specified for both null and alternative distribution through F0 and F1, respectively: both are lists with two elements:

• D is the vector of means (i.e. log2 fold changes),

• p is the vector of mixing proportions for the means.

If present, p must be the same length as D; its elements do not need to be normalized, i.e. sum to one; if absent, equal mixing is assumed, see Examples. A wide (mixture) null hypothesis, or an empirical null hypothesis as outlined by Efron (2004), can be used if genes with log fold changes close to zero are thought to be of no biological interest, and are counted as effectively not regulated. Similarly, the alternative hypothesis can be any mixture of large and small effects, symmetric or non-symmetric, depending on the expected regulation patterns, see Examples.

As a consequence, both the null distribution of the t-statistics (for the unregulated genes) and their alternative distribution (for the regulated genes) are mixtures of (generally non-central) t-distributions, see FDR.

Sample size n and standard deviation sigma are atomic values, but multiple p0 can be specified, resulting in multiple curves. Additionally, the usual significance level and sensitivity for a classical one-hypothesis can be displayed.

Value

This function returns invisibly a data frame with nplot rows whose columns contain the information for the individual curves. The number of columns and their names will depend on the number and value of the p0 specified, and whether alpha and sensitivity are displayed. Additionally, the returned data frame has an attribute param, which is a list with all the non-plotting arguments to the function.

Note

Both the curve labels and the legend may be squashed if the plotting device is too small. Increasing the size of the device and re-plotting should improve readability.

Author(s)

Y. Pawitan and A. Ploner

References

Pawitan Y, Michiels S, Koscielny S, Gusnanto A, Ploner A. (2005) False Discovery Rate, Sensitivity and Sample Size for Microarray Studies. Bioinformatics, 21, 3017-3024.

Efron, B. (2004) Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis. JASA, 99, 96-104.

See Also

FDR, samplesize, EOC

Examples

# Default null and alternative distributions, assuming different proportions
# of regulated genes
TOC(p0=c(0.90, 0.95, 0.99), legend.show=TRUE)

# The effect of sample size and effect size
par(mfrow=c(2,2))
TOC(p0=c(0.90, 0.95, 0.99), n=5, D=1)
TOC(p0=c(0.90, 0.95, 0.99), n=30, D=1)
TOC(p0=c(0.90, 0.95, 0.99), n=5, D=2)
TOC(p0=c(0.90, 0.95, 0.99), n=30, D=2)

# A wide null distribution that allows to disregard genes of small effect
# unspecified p means equal mixing proportions
ret = TOC(F0=list(D=c(-0.25,0,0.25)), main="Wide F0") 
attr(ret,"param")$F0 # the null hypothesis

# An extended (and unsymmetric) alternative
ret = TOC(F1=list(D=c(-2,-1,1), p=c(1,2,2)), p0=0.95, main="Unsymmetric F1")
attr(ret,"param")$F1 # F1$p is normalized

# Unequal sample sizes
TOC(n1=10, n2=30)

# Curves for a paired t-test
TOC(paired=TRUE)

# The output contains all the x- and y-coordinates
ret = TOC(p0=c(0.90, 0.95, 0.99), main="Default settings")
dim(ret)
colnames(ret)
ret[1:10,]
# Additionally, the list of arguments that determine the experiment
attr(ret,"param")

Example output

Warning messages:
1: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
4: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
5: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
6: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
7: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
8: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
9: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
There were 50 or more warnings (use warnings() to see the first 50)
Warning messages:
1: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
4: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
5: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
6: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
There were 50 or more warnings (use warnings() to see the first 50)
Warning messages:
1: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
$D
[1] -0.25  0.00  0.25

$p
[1] 0.3333333 0.3333333 0.3333333

Warning messages:
1: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
$D
[1] -2 -1  1

$p
[1] 0.2 0.4 0.4

There were 50 or more warnings (use warnings() to see the first 50)
Warning messages:
1: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
4: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
5: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
6: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
7: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
8: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
9: In dt(x, df = n - 1, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
Warning messages:
1: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
2: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
3: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
4: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
5: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
6: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
7: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
8: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
9: In dt(x, df = n1 + n2 - 2, ncp = ncp) :
  full precision may not have been achieved in 'pnt{final}'
[1] 100   9
[1] "stat"        "FDR0.90"     "FDR0.95"     "FDR0.99"     "fdr0.90"    
[6] "fdr0.95"     "fdr0.99"     "alpha"       "sensitivity"
         stat   FDR0.90   FDR0.95   FDR0.99   fdr0.90   fdr0.95   fdr0.99
1  0.00000000 0.9000000 0.9500000 0.9900000 0.9909619 0.9956983 0.9991715
2  0.06060606 0.8958858 0.9478236 0.9895456 0.9908749 0.9956567 0.9991635
3  0.12121212 0.8914073 0.9454432 0.9890466 0.9906125 0.9955312 0.9991392
4  0.18181818 0.8865404 0.9428428 0.9884992 0.9901700 0.9953195 0.9990983
5  0.24242424 0.8812597 0.9400052 0.9878992 0.9895395 0.9950176 0.9990399
6  0.30303030 0.8755388 0.9369121 0.9872418 0.9887099 0.9946201 0.9989630
7  0.36363636 0.8693504 0.9335436 0.9865219 0.9876668 0.9941198 0.9988661
8  0.42424242 0.8626666 0.9298789 0.9857341 0.9863923 0.9935077 0.9987474
9  0.48484848 0.8554591 0.9258958 0.9848721 0.9848650 0.9927732 0.9986049
10 0.54545455 0.8476995 0.9215709 0.9839295 0.9830600 0.9919036 0.9984359
       alpha sensitivity
1  1.0000000   1.0000000
2  0.9523408   0.9960752
3  0.9048659   0.9920898
4  0.8577573   0.9879832
5  0.8111922   0.9836950
6  0.7653411   0.9791656
7  0.7203655   0.9743357
8  0.6764162   0.9691471
9  0.6336316   0.9635427
10 0.5921367   0.9574669
$p0
[1] 0.90 0.95 0.99

$n1
[1] 10

$n2
[1] 10

$sigma
[1] 1

$F0
$F0$D
[1] 0

$F0$p
[1] 1


$F1
$F1$D
[1] -1  1

$F1$p
[1] 0.5 0.5


$paired
[1] FALSE

OCplus documentation built on Nov. 8, 2020, 5:20 p.m.