# Ordinal Dominance Statistics

### Description

Returns an array of ordinal dominance statistics based on the input of two 1-column matrices as an alternative to independent or paired group mean comparisons (especially for Cliff's delta statistics).

### Usage

1 2 3 |

### Arguments

`x` |
A 1-column matrix with optional column name containing all If x is a vector, a default column name is assigned. |

`y` |
A 1-column matrix with optional column name containing all |

`alpha` |
Significance or |

`paired` |
By default, independence of the two groups or data sets is assumed. If the number of cases in x and y are equal and paired (e.g. pre-post) comparisons, this should be set to TRUE to return the full array of within, between, combined and metric delta statistics. |

`outputfile` |
If a a detailed report of the ordinal dominance analysis is wanted, a filename should be given here. The report as standard text file is written to the current working directory. |

`studdist` |
By default, it is assumed that small samples are being examined. In this case, z-values based on Student's t-distribution are used for estimating upper and lower limits of the confidence intervals (CI) as well as z-probabilities. If larger sample sizes are used, these values approximate estimates based on normally distributed z-values. In this case or if comparing with estimates calculated with orddom versions <1.5 (where z-values based on the Standard Normal Distributions were used), this parameter may be set to FALSE. |

`symmetric` |
By default, asymmetric confidence intervals (CI) are being calculated to compensate for positive correlations between the samples as generally recommended by the literature on the delta statistics. To increase power in certain cases, however - e.g. in small paired samples (cf. Cliff 1996, p. 165) or fur purposes of evaluating the CIs of a combined delta estimate in the paired case - symmetric CIs may also be obtained by setting this argument to TRUE. |

`onetailed` |
By default, calculation of p values and confidence intervals (CI) assumes two-sided testing against the null hypothesis. Set to TRUE if the alternative hypothesis targets at one-tailed testing. |

`t.welch` |
By default, for calculation of the t-test scores and metric p and df values, the Welch approximation is used. If set to FALSE, equal variances are assumed for groups X and Y and a pooled variance is being calculated. |

`x.name` |
By default, the label of group x (i.e. 1st or control or pretest group) is taken from the column name of the x input matrix. This argument allows for assigning an alternative label. |

`y.name` |
This argument allows for assigning an alternative label for the y input matrix or group y (i.e. 2nd or experimental or posttest group). |

`description` |
This argument allows for assigning a string (as title or description) for the ordinal comparison outputs. |

### Value

**INDEPENDENT GROUPS** (*paired* argument set to FALSE)

In the case of independent groups or data sets X and Y (e.g. comparison group X vs. treatment group Y), a 2-column-matrix containing 29 rows with values is returned.

The ordinal statistics can be retrieved from the first column (named "ordinal") while the second column (named "metric") contains metric comparison data where appropriate.

`[1 ` |
Label assigned to group x (x.name or column name of the x input matrix) or a default "1st var (x)". |

`[2 ` |
Label assigned to group x (x.name or column name of the x input matrix) or a default "2nd var (y)". |

`[3 ` |
Column 1: Returns type of the comparison, in this case "indep". |

`[4 ` |
Number of cases in x (i.e. group X sample size). |

`[5 ` |
Number of cases in y (i.e. group Y sample size). |

`[6 ` |
Number of occurences of an observation from group y having a higher value than an observation from group x when comparing all x scores with all y scores: |

`[7 ` |
Number of occurences of an observation from group y having the same value as an observation from group x: |

`[8 ` |
Number of occurences of an observation from group y having a smaller value than an observation from group x: |

`[9 ` |
Common Language CL effect size or Probability of Superiority (PS) of X over Y, see below. |

`[10 ` |
Column 1: Discrete case Common Language CL effect size or Probability of Superiority (PS) of Y over X, |

`[11 ` |
Vargha and Delaney's A as stochastic superiority of X over Y, calculated as
(cf. Vargha & Delaney, 1998, 2000,Delaney & Vargha, 2002). This modified probability of superiority effect size has also been called area under the the receiver operating characteristic curve or AUC by Kraemer and Kupfer (2006). If one sampled one single case or subject from group Y and one from group X, respectively, A or AUC is the probability that the sample taken from group Y has a higher score or value than the one sampled from X (given the toss of a coin to break any ties). See also codedmes of this package. |

`[12 ` |
Vargha and Delaney's A as stochastic superiority of Y over X. |

`[13 ` |
For column 1 ("ordinal"): Cliff's delta for independent groups (Cliff, 1996,Long et al., 2003):
where |

`[14 ` |
Significance or |

`[15 ` |
Unless the default
with t-values at the given
where |

`[16 ` |
Confidence interval upper boundary estimate of delta or mean difference. |

`[17 ` |
Unbiased sample estimate of the delta standard deviation in column 1. |

`[18 ` |
Column 1: Variance of delta (unbiased sample estimate), calculated as
or, using the partial variances
which can also alternatively be put as
(For differences to Cliff's (1996, p. 138) formula see notes to Row 28 ("var dij") below.) |

`[19 ` |
Column 2 only: metric Standard error of mean difference: |

`[20 ` |
Column 1: z score of delta on the of the respective |

`[21 ` |
Equals |

`[22 ` |
Probability of z/t score (1-sided or 2-sided comparison as shown in row 21). |

`[23 ` |
Cohen's |

`[24 ` |
Column 1: Cohen's
. |

`[25 ` |
Column 1:Cohen's |

`[26 ` |
Row variance of dominance/difference matrix, calculated as |

`[27 ` |
Column variance of dominance/difference matrix, calculated as |

`[28 ` |
Variance of dominance/difference matrix as sample estimate according to Long et al. (2003, section 3.3 before eqn. 67):
thus avoiding Cliff's original (1996, p. 138) suggestion to use |

`[29 ` |
If the |

`[30 ` |
The
as suggested by Kraemer & Kupfer, 2006, p. 994. |

**DEPENDENT/PAIRED GROUPS** (*paired* argument set to TRUE)

In the case of paired data (e.g. pretest-posttest comparisons of the *n_x=n_y* same subjects), a 4-column-matrix containing 29 rows with values is returned.

The ordinal statistics for *d_{ij}* can be retrieved from the first three columns (named

`within [.,1]` |
for the |

`between [.,2]` |
for the overall distribution changes, based on all |

`combined [.,3]` |
for combined inferences |

Here, the fourth column (named "metric") contains metric comparison data.

`[1 ` |
Original column name of the x (or pretest) input matrix. |

`[2 ` |
Original column name of the y (or posttest) input matrix. |

`[3 ` |
Columns 1-3: Return type of the comparison, in this case "paired". |

`[4 ` |
Number of occurences (\#) of a posttest observation |

`[5 ` |
Number of occurences of a posttest observation having the same value as a pretest observation, limited to the respective pairs under observation in |

`[6 ` |
Number of occurences of a posttest observation having a smaller value than a pretest observation, limited to the respective pairs under observation in |

`[7 ` |
Common Language CL effect size or Probability of Superiority (PS) of X over Y (Grissom, 1994,Grissom & Kim, 2005) (limited to the respective pairs under observation in
. This effect size reflects the probability that a subject or case randomly chosen from the X- or pre-test-scores under observation has a higher score than than a randomly chosen case from the respective Y- or post-test-subsample (cf. Acion et al., 2006). |

`[8 ` |
Common Language CL effect size or Probability of Superiority (PS) of Y over X (Grissom, 1994,Grissom & Kim, 2005) (limited to the respective pairs under observation in |

`[9 ` |
Vargha and Delaney's A as stochastic superiority of X over Y, limited to the respective pairs under observation in |

`[10 ` |
Vargha and Delaney's A as stochastic superiority of Y over X, limited to the respective pairs under observation in |

`[11 ` |
For columns 1 to 3 ("ordinal"), the respective delta for dependent groups (Cliff, 1996,Long et al., 2003,Feng, 2007) is reported. With
where
where |

`[12 ` |
Significance or |

`[13 ` |
Confidence interval (CI) lower boundary estimate. Unless the default
with t-values at the respective significance level based on either Student's t or on z-values from the Standard Normal Distribution, depending on the
where |

`[14 ` |
Confidence interval upper boundary estimate (see row 13). |

`[15 ` |
Estimated standard deviation of the respective delta statistic. Column 4 reports the metric standard deviation of the paired (within) differences. |

`[16 ` |
Unbiased estimates of the variances of the respective delta statistic.
Please note that in various pieces of the available research literature (e.g. Cliff, 1996, eq. 6.8, p. 161), |

`[17 ` |
z score of delta. In column 4 ("metric") equal to the t-test score (assuming equal variances). |

`[18 ` |
Equals |

`[19 ` |
Probability of z-score (1 or 2-tailed comparison as shown in row 18). |

`[20 ` |
Cohen's |

`[21 ` |
Column 1 and 2: Cohen's
. |

`[22 ` |
Cohen's |

`[23,3] ` |
Component of |

`[24,3] ` |
Component of |

`[25,3] ` |
Component of |

`[26,3] ` |
Component of |

`[27,3] ` |
Component of |

`[28,3] ` |
Estimated covariance between |

`[29 ` |
Unless the |

`[30 ` |
In column 1 and 2, the
as suggested by Kraemer & Kupfer, 2006, p. 994. (Column 3 is empty.). |

### Author(s)

Jens J. Rogmann, University of Hamburg, Department of Psychology,

Hamburg, Germany (Jens.Rogmann@uni-hamburg.de)

### References

Acion, L., Peterson, J.J., Temple, S., & Arndt, S. (2006). Probabilistic index: an intuitive non-parametric approach to measuring the size of treatment effects. *Statistics in Medicine, 25*, 591 - 602.

Cliff, N. (1996). *Ordinal Methods for Behavioral Data Analysis*. Mahwah, NJ: Lawrence Erlbaum.

Cohen, J. (1988). *Statistical power analysis for the behavioral sciences (2nd edition)*. New York: Academic Press.

Cook, R.J. & Sackett, D.L. (1995). The number needed to treat: A clinically useful measure of treatment effect. *British Medical Journal, 310*, 452 - 454.

Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. *Psychological Methods, 1*, 170 - 177.

Feng, D. (2007). Robustness and Power of Ordinal d for Paired Data. In Shlomo S. Sawilowsky (Ed.), *Real Data Analysis* (pp. 163-183). Greenwich, CT : Information Age Publishing.

Feng, D., & Cliff, N. (2004). Monte Carlo Evaluation of Ordinal d with Improved Confidence Interval. *Journal of Modern Applied Statistical Methods, 3*(2), 322-332.

Long, J. D., Feng, D., & Cliff, N. (2003). Ordinal analysis of behavioral data. In J. Schinka & W. F. Velicer (eds.), *Research Methods in Psychology. Volume 2 of Handbook of Psychology* (I. B. Weiner, Editor-in-Chief). New York: John Wiley & Sons.

Grissom, R.J. (1994). Probability of the superior outcome of one treatment over another. *Journal of Applied Psychology, 79*, 314-316.

Grissom, R.J. & Kim, J.J. (2005). *Effect sizes for research. A broad practical approach*. Mahwah, NJ, USA: Erlbaum.

Hedges, L.V. & Olkin, I. (1985). *Statistical methods for meta-analysis*. San Diego, CA, USA: Academic Press.

Kraemer, H.C. & Kupfer, D.J. (2006). Size of Treatment Effects and Their Importance to Clinical Research and Practice. *Biological Psychiatry, 59*, 990-996.

McGraw, K.O. & Wong, S.P. (1992). A common language effect size statistic. *Psychological Bulletin, 111*, 361-365.

Romano, J., Kromrey, J. D., Coraggio, J., & Skowronek, J. (2006). *Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen's d for evaluating group differences on the NSSE and other surveys?* Paper presented at the annual meeting of the Florida Association of Institutional Research, Feb. 1-3, 2006, Cocoa Beach, Florida. Last retrieved January 2, 2012 from www.florida-air.org/romano06.pdf

### See Also

`orddom_f`

and `orddom_p`

.

### Examples

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | ```
## Not run:
#Independent Samples (Data taken from Long et al. (2003), Table 3
## End(Not run)
x<-t(matrix(c(3,3,3,4,5,6,12,12,13,14,15,15,15,15,15,16,18,18,18,23,23,27,28,28,43),1))
colnames(x)<-c("Nonalcohol.")
y<-t(matrix(c(1,4,6,7,7,14,14,18,19,20,21,24,25,26,26,26,27,28,28,30,33,33,44,45,50),1))
colnames(y)<-c("Alcoholic")
orddom(x,y,paired=FALSE,outputfile="tmp_r.txt")
## Not run:
#Paired Comparison with data written to file (Data taken from Long et al. (2003), Table 4
## End(Not run)
x<-t(matrix(c(2,6,6,7,7,8,8,9,9,9,10,10,10,11,11,12,13,14,15,16),1))
colnames(x)<-c("Incidental")
y<-t(matrix(c(4,11,8,9,10,11,11,5,14,12,13,10,14,16,14,13,15,15,16,10),1))
colnames(y)<-c("Intentional")
orddom_f(y,x,paired=TRUE,symmetric=FALSE)
## Not run:
#Directly returns d_b of the paired comparison
## End(Not run)
orddom(x,y,,TRUE,,,)[11,2]
``` |