In bramburger/colvano: Provide Intervals for Volcano Plots

Introduction

Volcano plots give information about two aspects of differential abundance. On the x-axis (log~2~) fold change is displayed, and on the y-axis (log~10~) p-values usually from (modified) t-tests. Thus, the x-axis shows the size of the effect, and the y-axis a measure of the certainty of there being an effect. Each protein is displayed as a single point, and while magnitude and variability are kind of represented in the the two axes, their relation is not entirely obviously clear.

If the p-value was calculated for the fold change, then a p-value of 0.05 would mean that one endpoint of the 95% CI for the fold change would be exactly 0. Unfortunately, the p-value usually comes from a modified t-test, and the property above holds for the 95% CI on the test statistic. (which is not the same as the fold change) To calculate p-values and/or confidence intervals for fold changes, one would run bootstrap or permutation tests.

As is mentioned in many places, p-values are a pretty suboptimal way of decision making, and are often misunderstood [@Greenland2016]. Especially when comparing proteins with differing numbers of missing values. What we would usually like to know is how likely it is that the next experiment (for example a validation experiment) would result in a significant result. For this we can calculate prediction intervals for p-values [@Vsevolozhskaya2017; @Boos2011]. These intervals assume a new experiment identical to the current one, although sample sizes can be adjusted.

One has to keep in mind, though, that by definition there is a 50% chance that the replication p-value is higher (lower) than the original p-value [@Senn2002]. The width of the interval can be interesting information, though. The parametric version of the prediction intervals gives a standard transformation of p-value to interval width. The bootstrap version does not necessarily give the same interval width for the same p-value with different underlying data.