getSigTests: Extract Significant Test Results for a Given Alpha or...

View source: R/getSigTests.R

getSigTestsR Documentation

Extract Significant Test Results for a Given Alpha or Different Multiple Testing Corrections.


This function finds the p-values less than or equal to alpha. Alpha is as given or corresponds to an adjusted alpha. Different multiple testing correction methods to adjust alpha are available.


  getSigTests(pvalues, alpha=0.05, method="plain")



A matrix, vector or an object of class gmw.


Significance level.


Method of multiple testing adjustment.


The argument pvalues is the result of a gmw test run or a matrix/vector of p-values. If pvalues is in matrix shape (several test methods applied to the same data), then each row corresponds to a different test method and the columns to different variables. The option method specifies the method of multiple testing correction. Typical options are "plain" for no correction, "bonferroni" for a Bonferroni correction, "simes" for an improved Bonferroni correction and "BH" for a Benjamini-Hochberg correction. Please note that the Simes method and the Benjamini-Hochberg correction lead to the same results. For permutation test results there is also the Westfall and Young method "maxT" available. In order to perform this correction the option keepPM=TRUE has to be set in the gmw call in order to keep the required permutation matrix.

Additional there are the two options "csD" and "csR". Those calculate for each alpha between 0 and alpha the difference (="csD") or the ratio (="csR") between observed and expected rejections and report an optimal alpha (and corresponding test rejections) for which these criteria are maximal. Please keep in mind that this method might return unreasonable large 'optimal' cut-off points.

See also the function rejectionPlot for more details.


A list object of class 're' with the values: (In case pvalues is a matrix the output is a list with length nrow(X) and each list item contains a single re object)


Position of the significant tests.


P-values of the significant tests.


The original pvalues object.


Chosen method.


Chosen alpha.


Adjusted / Optimal alpha.


Rows of pvalues.


Daniel Fischer


Benjamini, Y. and Hochberg, Y. (1995): Controlling the false discovery rate: a practical and powerful approach to multiple testing. J.R. Statist. Soc. B, 57(1):289 - 300.

Simes, R. J. (1986): An improved bonferroni procedure for multiple tests of significance. Biometrika, 73:751 - 754.

Westfall, P.H. and Young, S.S. (1993): Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. Wiley, New York.

See Also

rejectionPlot, gmw


  X <- matrix(c(rnorm(500,2,1),rnorm(600,2,1),rnorm(400,2.2,1)),byrow=TRUE, ncol=10)
  colnames(X) <- letters[1:10]
  g <- c(rep(1,50),rep(2,60),rep(3,40))
  test <- gmw(X,g,test="kw",type="external")
  cs1 <- getSigTests(test)

  simData <- runif(1000,0,1)
  simData <- c(simData,runif(200,0,0.01))
  simData2 <- runif(1000,0,1)
  simData2 <- c(simData2,runif(200,0,0.01))

  simDataMat <- rbind(simData,simData2)

  getSigTests(simDataMat, method="bon")
  getSigTests(simData, method="bon")
  getSigTests(simData2, method="bon")
  getSigTests(simData, method="sim")
  getSigTests(pvalues=simData, method="bh",alpha=0.05)
  getSigTests(pvalues=simData, method="csD")
  getSigTests(pvalues=simData, method="csR")

  X <- matrix(c(rnorm(50,2,1),rnorm(60,2,1),rnorm(40,2.2,1)),byrow=TRUE, ncol=10)
  colnames(X) <- letters[1:10]
  g <- c(rep(1,5),rep(2,6),rep(3,4))
  X[12:15,1] <- X[12:15,1] + 5

  # Keep the permutation matrix in order to perform the W&Y adjustment
    testPM <- gmw(X,g,test="kw",type="perm",keepPM=TRUE)
  # Apply the Westfall& Young adjustment

gMWT documentation built on Oct. 29, 2022, 1:14 a.m.