sexit: Sequential Effect eXistence and sIgnificance Testing (SEXIT)

Description Usage Arguments Details Value References Examples

View source: R/sexit.R

Description

The SEXIT is a new framework to describe Bayesian effects, guiding which indices to use. Accordingly, the sexit() function returns the minimal (and optimal) required information to describe models' parameters under a Bayesian framework. It includes the following indices:

Usage

1
sexit(x, significant = "default", large = "default", ci = 0.95, ...)

Arguments

x

Vector representing a posterior distribution. Can also be a Bayesian model (stanreg, brmsfit or BayesFactor).

significant, large

The threshold values to use for significant and large probabilities. If left to 'default', will be selected through sexit_thresholds(). See the details section below.

ci

Value or vector of probability of the (credible) interval - CI (between 0 and 1) to be estimated. Default to .95 (95%).

...

Currently not used.

Details

Rationale

The assessment of "significance" (in its broadest meaning) is a pervasive issue in science, and its historical index, the p-value, has been strongly criticized and deemed to have played an important role in the replicability crisis. In reaction, more and more scientists have tuned to Bayesian methods, offering an alternative set of tools to answer their questions. However, the Bayesian framework offers a wide variety of possible indices related to "significance", and the debate has been raging about which index is the best, and which one to report.

This situation can lead to the mindless reporting of all possible indices (with the hopes that with that the reader will be satisfied), but often without having the writer understanding and interpreting them. It is indeed complicated to juggle between many indices with complicated definitions and subtle differences.

SEXIT aims at offering a practical framework for Bayesian effects reporting, in which the focus is put on intuitiveness, explicitness and usefulness of the indices' interpretation. To that end, we suggest a system of description of parameters that would be intuitive, easy to learn and apply, mathematically accurate and useful for taking decision.

Once the thresholds for significance (i.e., the ROPE) and the one for a "large" effect are explicitly defined, the SEXIT framework does not make any interpretation, i.e., it does not label the effects, but just sequentially gives 3 probabilities (of direction, of significance and of being large, respectively) as-is on top of the characteristics of the posterior (using the median and HDI for centrality and uncertainty description). Thus, it provides a lot of information about the posterior distribution (through the mass of different 'sections' of the posterior) in a clear and meaningful way.

Threshold selection

One of the most important thing about the SEXIT framework is that it relies on two "arbitrary" thresholds (i.e., that have no absolute meaning). They are the ones related to effect size (an inherently subjective notion), namely the thresholds for significant and large effects. They are set, by default, to 0.05 and 0.3 of the standard deviation of the outcome variable (tiny and large effect sizes for correlations according to Funder \& Ozer, 2019). However, these defaults were chosen by lack of a better option, and might not be adapted to your case. Thus, they are to be handled with care, and the chosen thresholds should always be explicitly reported and justified.

Examples

The three values for existence, significance and size provide a useful description of the posterior distribution of the effects. Some possible scenarios include:

Value

A dataframe and text as attribute.

References

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
## Not run: 
library(bayestestR)

s <- sexit(rnorm(1000, -1, 1))
s
print(s, summary = TRUE)

s <- sexit(iris)
s
print(s, summary = TRUE)

if (require("rstanarm")) {
  model <- rstanarm::stan_glm(mpg ~ wt * cyl,
    data = mtcars,
    iter = 400, refresh = 0
  )
  s <- sexit(model)
  s
  print(s, summary = TRUE)
}

## End(Not run)

DominiqueMakowski/bayestestR documentation built on July 27, 2021, 4:12 p.m.