shortcuts: Confidence limits for the number of true hypotheses, with...
In cherry: Multiple Testing Methods for Exploratory Research

Description Usage Arguments Details Value Author(s) References Examples

Calculates confidence limits for the number of true hypotheses on the basis of the closed testing procedure using specific local tests that allow shortcuts.

pickFisher (p, select = seq_along(p), alpha=0.05, silent=FALSE)

curveFisher (p, select = seq_along(p), order, alpha=0.05, plot = TRUE)

pickSimes (hommel, select, alpha=0.05, silent=FALSE)

curveSimes (hommel, select, order, alpha=0.05, plot = TRUE)

pickMeinshausen (p, PM, select = seq_along(p), alpha=0.05, silent=FALSE)

curveMeinshausen (p, PM, select = seq_along(p), order, alpha=0.05, plot = TRUE)

`p`	The vector of p-values for all tested hypotheses.
`hommel`	The hommel object, obtained from the hommelFast function.
`PM`	A matrix of permutation p-values. Rows are hypotheses; columns are permutations.
`select`	The indexing vector of the p-values of the hypotheses to be selected. May be any type of appropriate indexing vector (integers, logical, or character).
`order`	The indexing vector specifying the order in which p-values of the hypotheses are to be rejected. May be integer or character.
`alpha`	The significance level of the test procedure.
`silent`	If `FALSE`, prints verbose result to the screen.
`plot`	If `TRUE`, plots the curve of correct rejections versus total rejections.

The results of the pickFisher, pickSimes and pickMeinshausen functions are identical to applying closed and pick, for specific choices of the local test, but are computationally more efficient. pickFisher uses local tests based on Fisher combinations. This local test is only valid if p-values of true hypotheses are independent. pickSimes uses a local test based on Simes' inequality. It is valid if p-values of true hypotheses are independent but also under some forms of positive correlations. The Hommel variant of the Simes local test is valid under any dependence structure of the p-values. pickMeinshausen is a permutation-based variant of pickSimes. See the reference below.

In the curve functions, the user may specify either select or order. Specifying order fixes the precise order in which hypotheses are selected, whereas specifying select only specifies which hypotheses are candidates for selection, leaving the order to be chosen by the function to maximize the number of correct rejections.

For pickFisher, pickSimes and pickMeinshausen, the function returns the lower confidence limit for the number of false hypotheses (correct rejection) among the set reject. The upper confidence limit is always equal to the number of rejections made. curveFisher and curveSimes return the same confidence limit, but for selecting only the first 1,2,3,... hypotheses from the selected set.

Jelle Goeman: j.j.goeman@lumc.nl; Aldo Solari

Goeman and Solari (2011) Statistical Science 26 (4) 584-597.

Meinshausen (2006) Scandinavian Journal of Statistics 33 (2), 227-237.

  # Fisher's method
  data(NAEP)
  pickFisher(NAEP, c("NH", "NC", "IA"))
  pickFisher(NAEP, 1:7)
  curveFisher(NAEP)
  curveFisher(NAEP, order=7:1)


  # Simes method
  hom <- hommelFast(NAEP)
  pickSimes(hom, c("NH", "NC", "IA"))
  pickSimes(hom, 1:7)
  curveSimes(hom)
  curveSimes(hom, select=1:7)
  
  # Meinshausen's method
  # This example uses data from the multtest package on bioconductor
  if(require("multtest")) {
    data(golub)
    smallglb <- golub[1:500,]
    TM<-sapply(1:nrow(smallglb), function(i) {
      mt.sample.teststat(smallglb[i,], golub.cl,test="t.equalvar",B=500)
    })
    PM<-2*(1-pt(abs(TM),df=length(golub.cl)-2)) # permutation matrix
    
    # p-values
    p<-apply(smallglb,1, function(z) t.test(z[golub.cl==0],z[golub.cl==1],var.equal=TRUE)$p.value) 
    
    pickMeinshausen(p, PM, select=1:100)
    pickMeinshausen(p, PM, select=sort.list(p)[1:100])
    curveMeinshausen(p,PM, select=1:200)
    curveMeinshausen(p,PM, order=1:200)
    
  }