calculate4GBias: calculate4GBias

In reproducer: Reproduce Statistical Analyses and Meta-Analyses

calculate4GBias

Description

The function simulates four-group experiments and estimates of the power, individual estimate error and small sample bias obtained from a set of simulated experiments. The function produces three set of simulations obtained using three different values of the mean difference between the treatment and control groups as specified by the parameter "diff". The power is estimated as the percentage of simulated experiments for which the mean of the experiment was significantly different from zero using one-sided tests. The experiment data may be one of four different type: Normal, Log-normal, Gamma or Laplace. The output is a table of values identifying the observed values of three effect sizes: Cliff's d, PHat and StdMD, their relted estimate error, small sample bias and power for each set of simulated experiments. This function supports the production of the values reported in data tables in the paper "Recommendations for Analyzing Small Sample Size" and its Supplementary Material.

Usage

calculate4GBias(
  mean = 0,
  sd = 1,
  N,
  reps,
  diff = c(0.2, 0.5, 0.8),
  Expected.StdMD = c(0.2, 0.5, 0.8),
  Expected.PHat = c(0.556, 0.638, 0.714),
  type = "n",
  seed = 223,
  StdAdj = 0,
  Blockmean = 0
)

Arguments

mean	This is the mean value of the control group(s) used in the simulations of each experiment for simulations of a specified mean difference (default 0).
sd	This is the standard deviation value of the control group(s) and treatment group(s) used in the simulations of each experiment of each family for simulations of a specified sample size (default 1).
N	This specifies the sample size per group that will be used in each set of simulations.
reps	The number of families simulated for each sample size.
diff	This specifies the difference between the control and treatment that will be used in each set of simulations. It must always have three values representing small, medium and large mean differences (default c(0.2, 0.5, 0.8)).
Expected.StdMD	This defines the expected value of the overall average StdMD for each mean difference (default c(0.2, 0.5, 0.8)).
Expected.PHat	This defines the expected population value of the overall average Phat for each mean difference (default c(0.556,0.638,0.714)).
type	This specifies the distribution of the data samples that will be simulated. Options ae "n" for Normal, "l", for Log-normal,'g" for Gamma, "lap" for LaPlace (default "n").
seed	A seed for the simulations (default 123).
StdAdj	Used to introduce variance heterogeneity for Laplace and Normal samples (default 0).
Blockmean	Specifies he value of the block effect (default 0).

Value

Design. Specifies the type of experiment 2G, the sample distribution (n,l,g,lap), and whether variance heterogeneity was added (het)

BEIncluded. Specifies whether or not a block effect was introduced.

GrpSize. Specifies the size of each group in the individual experiments.

Diff. The size of the difference between the control and treatment converted to an ordinal scale (Small, Medium, Large)

NPBias The relative difference between the average of the observed values of either Cliff's d or centralised PHat and the population value

StdMDBias. The relative difference between the average of the observed values of StdMDBias and the theoretical value

NPMdMRE The median of the absolute relative difference between the observed values of either Cliff's d or centralised PHat and the theoretical value for each experiment.

StdMDMdMRE The median of the relative difference between the observed values of StdMD and the population value for each experiment.

ObsPHat. The average Phat value found for each simulation.

ObsCliffd. The average Cliffd value found for each simulation.

ObsStdES. The average of StdMD calculated for each simulation.

PHatPower. The proportion of the simulations, for a given mean difference, for which the Phat estimate was significantly different from zero at the 0.05 alpha level based on one-sided tests.

CliffdPower. The proportion of the simulations, for a given mean difference, for which the Cliff's d estimate was significantly different from zero at the 0.05 alpha level based on one-sided tests.

StdMDPower. The proportion of the simulations, for a given mean difference, for which the StdMD estimate was significantly different from zero at the 0.05 alpha level based on one-sided tests.

Author(s)

Barbara Kitchenham and Lech Madeyski

Examples

#as.data.frame(calculate4GBias(mean=0,sd=1,diff=c(0.266,0.72375,1.43633),
#  Expected.StdMD=c(0.2,0.5,0.8),Expected.PHat=c(0.575,0.696,0.845),N=10,reps=200,type="l",
#  seed=17+1823,StdAdj=0,Blockmean=0))
#  Design BEIncluded GrpSize   Diff      NPBias StdMDBias   NPMdMRE StdMDMdMRE  ObsPHat ObsCliffd.
#  1 4G_l         No      10  Small -0.05933333 0.1247408 0.8666667  1.2047848 0.570550   0.1411..
#  2 4G_l         No      10 Medium -0.01760204 0.1565643 0.3112245  0.4426859 0.692550   0.3851..
#  3 4G_l         No      10  Large -0.00326087 0.2273638 0.1594203  0.2924361 0.843875   0.6877..
as.data.frame(calculate4GBias(mean=1,sd=3,diff=c(0.1225,0.3415,0.6224),
 Expected.StdMD=c(-0.208,-0.52,-0.833),Expected.PHat=c(0.444,0.360,0.277),N=20,reps=30,type="g",
 seed=17+977,StdAdj=0 ,Blockmean=0.5))
# Results for reps=200:
#  Design BEIncluded GrpSize   Diff     NPBias  StdMDBias   NPMdMRE StdMDMdMRE   ObsPHat  ObsCli..
#1   4G_g        Yes      20  Small 0.04274554 0.02242895 0.8370536  0.7960052 0.4416062 -0.1167..
#2   4G_g        Yes      20 Medium 0.01959821 0.01585829 0.3348214  0.3210435 0.3572562 -0.2854..
#3   4G_g        Yes      20  Large 0.01303251 0.01515967 0.1905830  0.1871956 0.2740938 -0.4518..

reproducer documentation built on Oct. 18, 2023, 5:10 p.m.