# confSAM: Permutation-based confidence bounds for the false discovery... In confSAM: Estimates and Bounds for the False Discovery Proportion, by Permutation

## Description

Computes a confidence upper bound for the False Discovery Proportion (FDP). The input required is a matrix containing test statistics (or p-values) for (randomly) permuted versions of the data.

## Usage

 ```1 2``` ``` confSAM(p, PM, includes.id=TRUE, cutoff=0.01, reject="small", alpha=0.05, method="simple", ncombs=1000) ```

## Arguments

 `p` A vector containing the p-values for the original (unpermuted) data. `PM` A matrix (with `length(p)` columns) containing for each permutation the p-values corresponding to the permuted version of the data. If `PM` contains the original values `p`, then they should be in the first row of PM. `includes.id` Set this to `FALSE` if PM does not contain the original p-values `p`. `cutoff` A number or a vector of length `length(p)`. In the first case all hypotheses with test statistics exceeding `cutoff` are rejected. In the second case there is a separate cut-off for each hypothesis. `reject` If `reject="small"`, then all hypotheses with test statistics (p-values) smaller than `cutoff` are rejected. If `reject="larger"`, then all hypotheses with test statistics larger than `cutoff` are rejected. If `reject="absolute"`, then all hypotheses with test statistics with absolute value larger than `cutoff` are rejected. `alpha` 1-alpha is the desired confidence level of the bounds. `method` If `method="simple"`, then a basic (fast) bound for V (the number of false positives) is computed. If `method="full"`, then a (computationally intensive) closed testing-based bound for V is computed. This is usually infeasible when the number of rejections is large. If `method="approx"`, then an approximation of the closed testing-based bound for V is computed. The resulting bound may be anti-conservative if `ncombs` is too small. `ncombs` Only applies when `method="approx"`. It is the number of random combinations that the approximation method checks. Larger values of ncombs give more reliable results.

## Value

A vector with three values is returned. The first value is the number if rejections. The second value is a basic median unbiased estimate of the number of false positives V. This estimate coincides with the simple upper bound for alpha=0.5. The third value is a (1-alpha)-confidence upper bound for V (it depends on the argument `method` which bound this is.)

## Examples

 ``` 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39``` ```#This is a fast example. It is recommended to take w and ncombs larger in practice. set.seed(423) m <- 100 #number of hypotheses n <- 10 #the amount of subjects is 2n (n cases, n controls). w <- 50 #number of random permutations. Here we take w small for computational speed X <- matrix(rnorm((2*n)*m), 2*n, m) X[1:n,1:50] <- X[1:n,1:50]+1.5 # the first 50 hypotheses are false #(increased mean for the first n individuals). y <- c(numeric(n)+1,numeric(n)-1) Y <- t(replicate(w, sample(y, size=2*n, replace=FALSE))) Y[1,] <- y #add identity permutation pvalues <- matrix(nrow=w,ncol=m) for(j in 1:w){ for(i in 1:m){ pvalues[j,i] <- t.test( X[Y[j,]==1,i], X[Y[j,]==-1,i] , alternative="two.sided" )\$p.value } } ## number of rejections: confSAM(p=pvalues[1,], PM=pvalues, cutoff=0.05, alpha=0.1, method="simple")[1] ## basic median unbiased estimate of #false positives: confSAM(p=pvalues[1,], PM=pvalues, cutoff=0.05, alpha=0.1, method="simple")[2] ## basic (1-alpha)-upper bound for #false positives: confSAM(p=pvalues[1,], PM=pvalues, cutoff=0.05, alpha=0.1, method="simple")[3] ## potentially smaller (1-alpha)-upper bound for #false positives: ## (taking 'ncombs' much larger recommended) confSAM(p=pvalues[1,], PM=pvalues, cutoff=0.05, alpha=0.1, method="approx", ncombs=50)[3] ## actual number of false positives: sum(pvalues[1,51:100]<0.05) ```

confSAM documentation built on Feb. 19, 2018, 5:01 p.m.