# eff_size: Calculate effect sizes and confidence bounds thereof In emmeans: Estimated Marginal Means, aka Least-Squares Means

 eff_size R Documentation

## Calculate effect sizes and confidence bounds thereof

### Description

Standardized effect sizes are typically calculated using pairwise differences of estimates, divided by the SD of the population providing the context for those effects. This function calculates effect sizes from an `emmGrid` object, and confidence intervals for them, accounting for uncertainty in both the estimated effects and the population SD.

### Usage

```eff_size(object, sigma, edf, method = "pairwise", ...)
```

### Arguments

 `object` an `emmGrid` object, typically one defining the EMMs to be contrasted. If instead, `class(object) == "emm_list"`, such as is produced by `emmeans(model, pairwise ~ treatment)`, a message is displayed; the contrasts already therein are used; and `method` is replaced by `"identity"`. `sigma` numeric scalar, value of the population SD. `edf` numeric scalar that specifies the equivalent degrees of freedom for the `sigma`. This is a way of specifying the uncertainty in `sigma`, in that we regard our estimate of `sigma^2` as being proportional to a chi-square random variable with `edf` degrees of freedom. (`edf` should not be confused with the `df` argument that may be passed via `...` to specify the degrees of freedom to use in t statistics and confidence intervals.) `method` the contrast method to use to define the effects. This is passed to `contrast` after the elements of `object` are scaled. `...` Additional arguments passed to `contrast`

### Details

Any `by` variables specified in `object` will remain in force in the returned effects, unless overridden in the optional arguments.

For models having a single random effect, such as those fitted using `lm`; in that case, the `stats::sigma` and `stats::df.residual` functions may be useful for specifying `sigma` and `edf`. For models with more than one random effect, `sigma` may be based on some combination of the random-effect variances.

Specifying `edf` can be rather unintuitive but is also relatively uncritical; but the smaller the value, the wider the confidence intervals for effect size. The value of `sqrt(2/edf)` can be interpreted as the relative accuracy of `sigma`; for example, with `edf = 50`, √(2/50) = 0.2, meaning that `sigma` is accurate to plus or minus 20 percent. Note in an example below, we tried two different `edf` values as kind of a bracketing/sensitivity-analysis strategy. A value of `Inf` is allowable, in which case you are assuming that `sigma` is known exactly. Obviously, this narrows the confidence intervals for the effect sizes – unrealistically if in fact `sigma` is unknown.

### Value

an `emmGrid` object containing the effect sizes

### Computation

This function uses calls to `regrid` to put the estimated marginal means (EMMs) on the log scale. Then an extra element is added to this grid for the log of `sigma` and its standard error (where we assume that `sigma` is uncorrelated with the log EMMs). Then a call to `contrast` subtracts `log{sigma}` from each of the log EMMs, yielding values of `log(EMM/sigma)`. Finally, the results are re-gridded back to the original scale and the desired contrasts are computed using `method`. In the log-scaling part, we actually rescale the absolute values and keep track of the signs.

### Note

The effects are always computed on the scale of the linear-predictor; any response transformation or link function is completely ignored. If you wish to base the effect sizes on the response scale, it is not enough to replace `object` with `regrid(object)`, because this back-transformation changes the SD required to compute effect sizes.

Disclaimer: There is substantial disagreement among practitioners on what is the appropriate `sigma` to use in computing effect sizes; or, indeed, whether any effect-size measure is appropriate for some situations. The user is completely responsible for specifying appropriate parameters (or for failing to do so).

The examples here illustrate a sobering message that effect sizes are often not nearly as accurate as you may think.

### Examples

```fiber.lm <- lm(strength ~ diameter + machine, data = fiber)

emm <- emmeans(fiber.lm, "machine")
eff_size(emm, sigma = sigma(fiber.lm), edf = df.residual(fiber.lm))

# or equivalently:
eff_size(pairs(emm), sigma(fiber.lm), df.residual(fiber.lm), method = "identity")

### Mixed model example:
if (require(nlme)) withAutoprint({
Oats.lme <- lme(yield ~ Variety + factor(nitro),
random = ~ 1 | Block / Variety,
data = Oats)
# Combine variance estimates
VarCorr(Oats.lme)
(totSD <- sqrt(214.4724 + 109.6931 + 162.5590))
# I figure edf is somewhere between 5 (Blocks df) and 51 (Resid df)
emmV <- emmeans(Oats.lme, ~ Variety)
eff_size(emmV, sigma = totSD, edf = 5)
eff_size(emmV, sigma = totSD, edf = 51)
}, spaced = TRUE)

# Multivariate model for the same data:
MOats.lm <- lm(yield ~ Variety, data = MOats)
eff_size(emmeans(MOats.lm, "Variety"),
sigma = sqrt(mean(sigma(MOats.lm)^2)),   # RMS of sigma()
edf = df.residual(MOats.lm))
```

emmeans documentation built on May 15, 2022, 9:05 a.m.