comparability: Split-half factor comparability

In ackwards: Bass-Ackwards Hierarchical Structural Analysis

Split-half factor comparability

Description

Measures how well each factor at each level of a bass-ackwards hierarchy replicates across random split-halves of the sample – Everett's (1983) factor comparability coefficients, reviving the split-half replication gate of the research program that produced the bass-ackwards method: Saucier (1997) screened factor solutions by their split-half stability, and Saucier, Georgiades, Tsaousis, and Goldberg (2005) chose the optimal hierarchical level by requiring split-half replication above a .90 threshold. It is also the direct instrument for the overextraction caution in suggest_k(): non-replicable structure concentrates in the deeper levels of an overextracted hierarchy (Forbes, 2023).

Usage

comparability(
  data,
  k_max,
  engine = "pca",
  cor = "pearson",
  fm = "minres",
  n_splits = 10L,
  seed = NULL,
  ...
)

Arguments

data	A data frame or numeric matrix of observed variables (items in columns, observations in rows). Raw data only – splitting needs rows, so a correlation matrix is not accepted. Missing values are handled pairwise throughout (as in ackwards()'s default).
k_max	Maximum number of factors/components to evaluate – normally the same value (or one or two above it) you intend to pass to ackwards(). Required.
engine	Extraction engine: "pca" (default) or "efa". "esem" is not yet supported here (fitting 2 * n_splits lavaan hierarchies needs its own performance treatment); for ESEM workflows, run comparability() with engine = "efa" as a structural screen.
cor	Correlation basis: "pearson" (default) or "spearman". As with suggest_k(), "polychoric" is not supported (estimating polychoric matrices in every half-sample is slow and unstable); users analysing ordinal data should screen replicability on the Pearson basis and fit the final model with cor = "polychoric" in ackwards().
fm	Factor extraction method passed to psych::fa(); only used when engine = "efa". One of "minres" (default), "ml", or "pa".
n_splits	Number of random split-half replicates. Default 10L. The published precedents used a single split (Saucier et al., 2005); repeating the split guards against the luck of one draw and is this implementation's robustness choice. Each replicate fits 2 * k_max solutions, so the default costs 20 hierarchy fits; PCA and EFA are fast enough that this is typically a few seconds.
seed	Integer seed for reproducible splits. NULL (default) uses the current RNG state.
...	Reserved for future arguments.

Details

For each of n_splits random half-splits, solutions at every level ⁠1..k_max⁠ are fit independently in each half. Each half-solution's factors are matched to the full-sample solution's factors (so coefficients are reported under the same ⁠m{k}f{j}⁠ labels you get from ackwards()), and the comparability coefficient for a factor is the correlation between its two matched half-solution scores, computed on the pooled correlation matrix via the same ⁠W'RW⁠ algebra used for between-level edges – applying both halves' scoring weights to the full sample, exactly Everett's procedure. Tucker's congruence coefficient (phi) between the matched half-solution loading columns is reported alongside: comparability asks whether the two halves' scores agree; phi asks whether their loading patterns agree.

Value

An object of class "comparability". Print it for a per-level summary; call autoplot() on it for a diagnostic plot. The list contains:

coefficients	Data frame with one row per split x level x factor: split, level, factor (full-sample ⁠m{k}f{j}⁠ label), r (score comparability), phi (Tucker's congruence of the matched loading columns). NA when the level did not converge in one of the halves.
summary	Data frame with one row per level x factor: level, factor, r_median, r_min, phi_median, phi_min (across splits), and n_splits_ok (splits in which both halves converged).
k_max	Deepest level evaluated. Can be lower than the k_max you asked for when the full-sample fit truncated (non-convergence at deep levels); the original request is kept in k_requested.
k_requested, n_splits, n_half, engine, cor, fm, n_obs, n_vars, seed	Metadata.

Interpreting the output

Coefficients near 1 mean the factor re-emerges in independent half-samples; a factor whose comparability is low is sample-idiosyncratic and should not anchor substantive interpretation. Conventional benchmarks: .90 is the split-half replication threshold of Goldberg's lexical research program (Saucier et al., 2005) and follows from Everett's (1983) rationale (split-half factors sharing at least 81% of their variance); .95 is the stricter bound at which two factors are conventionally treated as interchangeable (Lorenzo-Seva & ten Berge, 2006). These are conventions, not tests, so comparability() reports every coefficient and flags nothing. The deepest level at which all factors replicate is a natural hierarchy floor for ackwards()'s k_max; see vignette("ackwards-girard") for the full workflow.

A level that fails to converge in a half-sample yields NA coefficients for that split (convergence is data, not an error); the number of usable splits per factor is reported in summary$n_splits_ok and a message summarises any shortfall.

References

Everett, J. E. (1983). Factor comparability as a means of determining the number of factors and their rotation. Multivariate Behavioral Research, 18(2), 197–218. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1207/s15327906mbr1802_5")}

Saucier, G. (1997). Effects of variable selection on the factor structure of person descriptors. Journal of Personality and Social Psychology, 73(6), 1296–1312. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0022-3514.73.6.1296")}

Saucier, G., Georgiades, S., Tsaousis, I., & Goldberg, L. R. (2005). The factor structure of Greek personality adjectives. Journal of Personality and Social Psychology, 88(5), 856–875. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0022-3514.88.5.856")}

Lorenzo-Seva, U., & ten Berge, J. M. F. (2006). Tucker's congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2), 57–64. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1027/1614-2241.2.2.57")}

Forbes, M. K. (2023). Improving hierarchical models of individual differences: An extension of Goldberg's bass-ackward method. Psychological Methods. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/met0000546")}

See Also

suggest_k() for the plausible depth range (eigenstructure), factorability() for sampling adequacy before you fit, prune() for factors that perpetuate without differentiating (redundancy), and ackwards() for the extraction itself.

Examples

cmp <- comparability(sim16, k_max = 5, n_splits = 5, seed = 1)
cmp
cmp$summary

ackwards documentation built on July 25, 2026, 1:08 a.m.