prospective: Prospective Design Analysis

In ClaudioZandonella/prova_pck: Conduct a Prospective or Retrospective Design Analysis

Description

Given the hypothetical population effect size and the required power level, the function prospective() performs a prospective design analysis for Cohen's d (t-test comparing group means) or Pearson's correlation test between two variables. According to the defined alternative hypothesis and significance level, the required sample size is computed together with the associated Type-M error, Type-S error, and the the critical correlation value (i.e., the minimum absolute effect size value that would result significant).

Usage

prospective(
  effect_size,
  power,
  ratio_n2 = 1,
  effect_type = c("cohen_d", "correlation"),
  alternative = c("two.sided", "less", "greater"),
  sig_level = 0.05,
  B = 10000,
  seed = NULL,
  tl = -Inf,
  tu = Inf,
  B_effect = 250,
  sample_range = c(2, 1000),
  tol = 0.01,
  display_message = FALSE,
  ...
)

Arguments

effect_size	a numeric value or function (see details) indicating the hypothetical population effect size.
power	a numeric value indicating the required power level.
ratio_n2	a numeric value indicating the ratio between sample_n1 and sample_n2.
effect_type	a character string specifying the effect type, must be "cohen_d" (default, Cohen's d standardised means difference) or "pearson" (Pearson's correlation). You can specify just the initial letter.
alternative	a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter.
sig_level	a numeric value indicating the significance level on which the alternative hypothesis is evaluated.
B	a numeric value indicating the number of iterations. Increase the number of iterations to obtain more stable results.
seed	a numeric value indicating the seed for random number generation. Set the seed to obtain results reproducible.
tl	optional value indicating the lower truncation point if effect_size is defined as a function.
tu	optional value indicating the upper truncation point if effect_size is defined as a function.
B_effect	a numeric value indicating the number of sampled effect size if effect_size is defined as a function. Increase the number to obtain more stable results.
sample_range	a length-2 numeric vector indicating the minimum and maximum sample size of the first group (sample_n1).
tol	a numeric value indicating the tolerance of required power level.
display_message	a logical variable indicating whether to display or not the information about computational steps.
...	further arguments to be passed to or from methods.

Details

Conduct a prospective design analysis to define the required sample size and the associated inferential risks according to study design. A general overview is provided in the vignette(todo).

Population effect size

The hypothetical population effect size (effect_size) can be set to a single value or a function that allows to sample values from a given distribution. The function has to be defined as function(x) my_function(x, ...), with only one single variable x that represent the number of samples (e.g., function(x) rnorm(x, mean = 0, sd = 1); function(x) sample(c(.1,.3,.5), x, replace = TRUE)). This allows users to define hypothetical effect size distribution according to their needs.

Argument B_effect allows to define the number of sampled effect size. Users can access sampled effects in the effect_info list included in the output to evaluate if sample is representative of their specification. Increase the number to obtain more accurate results but it will require more computational time (default is 250).

Optional arguments tl and tu allow to truncate the sampling distribution specifying the lower truncation point and upper truncation point respectively. Note that if effect_type = "correlation", distribution is automatically truncated between -1 and 1.

Effect type options

The effect_type argument can be set to "cohen_d" (default) for standardized mean difference or "correlation" if Pearson's correlation is evaluated.

In the case of "cohen_d" one-sample or two-samples t-test are considered following same options specification of basic function t.test(), note that default options of t.test() are paired = FALSE and var.equal = FALSE. For one-sample t-test ratio_n = NULL is required. For paired t-test ratio_n = 1 and option paired = TRUE are required. For two-samples t-test ratio_n can be specified according to user needs and option var.equal = TRUE is required. For Welch t-test, ratio_n can be specified according to user needs (default option is var.equal = FALSE).

In the case of "correlation", only Pearson's correlation between two variables is available and ratio_n is set to 1 (default). The Kendall's tau or Spearman's rho are not implemented.

Study design

Study design can be further defined according to statistical test directionality and required α-level using the arguments alternative and sig_level respectively.

Value

A list with class "design_analysis" containing the following components:

design_analysis	a character string indicating the type of design analysis: "prospective".
call_arguments	a list with all the arguments passed to the function.
effect_info	a list with all the information regarding the considered hypothetical population effect size. The list includes: effect_type indicating the type of effect; effect_function indicating the function from which effect are sampled or the string "single_value" if single value was provided; effect_summary summary of the sampled effects; effect_samples vector with the sampled effects (or unique value in the case of single value); if relevant tl and tu specifying the lower upper truncation point respectively.
test_info	a list with all the information regarding the test performed. The list includes: test_method character sting indicating the test method (e.g., "pearson", "one-sample", "paired", "two-samples", or "welch"); the required sample size (sample_n1 and if relevant sample_n2), alternative hypothesis (alternative), significance level (sig_level) and degrees of freedom (df) of the statistical test; critical_effect the minimum absolute effect value that would result significant. Note that critical_effect in the case of alternative = "two.sided" is the absolute value and both positive and negative values should be considered.
prospective_res	a data frame with the resulting inferential errors. Columns names are power, typeM, and typeS.

References

Altoè, G., Bertoldo, G., Zandonella Callegher, C., Toffalini, E., Calcagnì, A., Finos, L., & Pastore, M. (2020). Enhancing Statistical Inference in Psychological Research via Prospective and Retrospective Design Analysis. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.02893

Gelman, A., & Carlin, J. (2014). Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors. Perspectives on Psychological Science, 9(6), 641–651. https://doi.org/10.1177/1745691614551642

Bertoldo, G., Altoè, G., & Zandonella Callegher, C. (2020, June 15). Designing Studies and Evaluating Research Results: Type M and Type S Errors for Pearson Correlation Coefficient. Retrieved from https://psyarxiv.com/q9f86/

Examples

# One-sample t-test
prospective(effect_size = .3, power = .8, ratio_n = NULL,
            effect_type = "cohen_d", seed = 2020, B = 1e3)
# Paired t-test
prospective(effect_size = .3, power = .8, ratio_n = 1,
            effect_type = "cohen_d", paired = TRUE, seed = 2020, B = 1e3)
# Two-samples t-test
prospective(effect_size = .3, power = .8, ratio_n = 1.5,
            effect_type ="cohen_d", var.equal = TRUE, seed = 2020, B = 1e3)
# Welch t-test
prospective(effect_size = .3, power = .8, ratio_n = 2,
            effect_type ="cohen_d", seed = 2020, B = 1e3)

# Pearson's correlation
prospective(effect_size = .3, power = .8, effect_type = "correlation",
            seed = 2020, B = 1e3)

## Not run: 
# Define effect_size using functions (long computational time)
prospective(effect_size = function(x) rnorm(x, .3, .1), power = .8,
              effect_type = "correlation", seed = 2020)
prospective(effect_size = function(x) rnorm(x, .3, .1), power = .8,
              effect_type = "cohen_d", tl = .2, tu = .4, seed = 2020)

## End(Not run)

ClaudioZandonella/prova_pck documentation built on June 17, 2020, 12:29 a.m.