psychmeta-package: 'psychmeta': Psychometric meta-analysis toolkit

In psychmeta: Psychometric Meta-Analysis Toolkit

psychmeta: Psychometric meta-analysis toolkit

Description

Overview of the psychmeta package.

Details

The psychmeta package provides tools for computing bare-bones and psychometric meta-analyses and for generating psychometric data for use in meta-analysis simulations. Currently, psychmeta supports bare-bones, individual-correction, and artifact-distribution methods for meta-analyzing correlations and d values. Please refer to the overview tutorial vignette for an introduction to psychmeta's functions and workflows.

Running a meta-analysis

The main functions for conducting meta-analyses in psychmeta are ma_r for correlations and ma_d for d values. These functions take meta-analytic dataframes including effect sizes and sample sizes (and, optionally, study labels, moderators, construct and measure labels, and psychometric artifact information) and return the full results of psychometric meta-analyses for all of the specified variable pairs. Examples of correctly formatted meta-analytic datasets for ma functions are data_r_roth_2015, data_r_gonzalezmule_2014, and data_r_mcdaniel_1994. Individual parts of the meta-analysis process can also be run separately; these functions are described in detail below.

Preparing a database for meta-analysis

The convert_es function can be used to convert a variety of effect sizes to either correlations or d values. Sporadic psychometric artifacts, such as artificial dichotomization or uneven splits for a truly dichotomous variable, can be individually corrected using correct_r and correct_d. These functions can also be used to compute confidence intervals for observed, converted, and corrected effect sizes. 'Wide' meta-analytic coding sheets can be reformatted to the 'long' data frames used by psychmeta with reshape_wide2long. A correlation matrix and accompanying vectors of information can be similarly reformatted using reshape_mat2dat.

Meta-analytic models

psychmeta can compute barebones meta-analyses (no corrections for psychometric artifacts), as well as models correcting for measurement error in one or both variables, univariate direct (Case II) range restriction, univariate indirect (Case IV) range restriction, bivariate direct range restriction, bivariate indirect (Case V) range restriction, and multivariate range restriction. Artifacts can be corrected individually or using artifact distributions. Artifact distribution corrections can be applied using either Schmidt and Hunter's (2015) interactive method or Taylor series approximation models. Meta-analyses can be computed using various weights, including sample size (default for correlations), inverse variance (computed using either sample or mean effect size; error based on mean effect size is the default for d values), and weight methods imported from metafor.

Preparing artifact distributions meta-analyses

For individual-corrections meta-analyses, reliability and range restriction (u) values should be supplied in the same data frame as the effect sizes and sample sizes. Missing artifact data can be imputed using either bootstrap or other imputation methods. For artifact distribution meta-analyses, artifact distributions can be created automatically by ma_r or ma_d or manually by the create_ad family of functions.

Moderator analyses

Subgroup moderator analyses are run by supplying a moderator matrix to the ma_r or ma_d families of functions. Both simple and fully hierarchical moderation can be computed. Subgroup moderator analysis results are shown by passing an ma_obj to print(). Meta-regression analyses can be run using metareg.

Reporting results and supplemental analyses

Meta-analysis results can be viewed by passing an ma object to summary. Bootstrap confidence intervals, leave one out analyses, and other sensitivity analyses are available in sensitivity. Supplemental heterogeneity statistics (e.g., Q, I^{2}) can be computed using heterogeneity. Meta-analytic results can be converted between the r and d metrics using convert_ma. Each ma_obj contains a metafor escalc object in ma$...$escalc that can be passed to metafor's functions for plotting, publication/availability bias, and other supplemental analyses. Second-order meta-analyses of correlations can be computed using ma_r_order2. Example second-order meta-analysis datasets from Schmidt and Oh (2013) are available. Tables of meta-analytic results can be written as markdown, Word, HTML, or PDF files using the metabulate function, which exports near publication-quality tables that will typically require only minor customization by the user.

Simulating psychometric meta-analyses

psychmeta can be used to run Monte Carlo simulations for different meta-analytic models. simulate_r_sample and simulate_d_sample simulate samples of correlations and d values, respectively, with measurement error and/or range restriction artifacts. simulate_r_database and simulate_d_database can be used to simulate full meta-analytic databases of sample correlations and d values, respectively, with artifacts. Example datasets fitting different meta-analytic models simulated using these functions are available (data_r_meas, data_r_uvdrr, data_r_uvirr, data_r_bvdrr, data_r_bvirr, data_r_meas_multi, and data_d_meas_multi). Additional simulation functions are also available.

An overview of our labels and abbreviations

Throughout the package documentation, we use several sets of labels and abbreviations to refer to methodological features of variables, statistics, analyses, and functions. We define sets of key labels and abbreviations below.

Abbreviations for meta-analytic methods:

• bb: Bare-bones meta-analysis.

• ic: Individual-correction meta-analysis.

• ad: Artifact-distribution meta-analysis.

Abbreviations for types of artifact distributions and artifact-distribution meta-analyses:

• int: Interactive approach.

• tsa: Taylor series approximation approach.

Notation used for variables involved in correlations:

• x or X: Scores on the observed variable designated as X by the analyst (i.e., scores containing measurement error). By convention, X typically represents a predictor variable.

• t or T: Scores on the construct associated with X (i.e., scores free from measurement error).

• y or Y: Scores on the observed variable designated as Y by the analyst (i.e., scores containing measurement error). By convention, Y typically represents a criterion variable.

• p or P: Scores on the construct associated with Y (i.e., scores free from measurement error).

Note: The use of lowercase or uppercase labels does not alter the meaning of the notation.

Notation used for variables involved in d values:

• g: Group membership status based on the observed group membership variable (i.e., statuses containing measurement/classification error).

• G: Group membership status based on the group membership construct (i.e., statuses free from measurement/classification error).

• y or Y: Scores on the observed variable being compared between groups (i.e., scores containing measurement error).

• p or P: Scores on the criterion construct being compared between groups (i.e., scores free from measurement error).

Note: There is always a distinction between the g and G labels because they differ in case. The use of lowercase or uppercase labels for y/Y or p/P does not alter the meaning of the notation.

Notation used for types of correlations:

• rxy: Observed correlation.

• rxp: Correlation corrected for measurement error in Y only.

• rty: Correlation corrected for measurement error in X only.

• rtp: True-score correlation corrected for measurement error in both X and Y.

Note: Correlations with labels that include "i" suffixes are range-restricted, and those with "a" suffixes are unrestricted or corrected for range restriction.

Notation used for types of d values:

• dgy: Observed d value.

• dgp: d value corrected for measurement error in Y only.

• dGy: d value corrected for measurement/classification error in the grouping variable only.

• dGp: True-score d value corrected for measurement/classification error in both X and the grouping variable.

Note: d values with labels that include "i" suffixes are range-restricted, and those with "a" suffixes are unrestricted or corrected for range restriction.

Types of correction methods (excluding sporadic corrections and outdated corrections implemented for posterity):

• meas: Correction for measurement error only.

• uvdrr: Correction for univariate direct range restriction (i.e., Case II). Can be applied to using range restriction information for either X or Y.

• uvirr: Correction for univariate indirect range restriction (i.e., Case IV). Can be applied to using range restriction information for either X or Y.

• bvdrr: Correction for bivariate direct range restriction. Use with caution: This correction is an approximation only and is known to have a positive bias.

• bvirr: Correction for bivariate indirect range restriction (i.e., Case V).

Note: Meta-analyses of d values that involve range-restriction corrections treat the grouping variable as "X."

Labels for types of output from psychometric meta-analyses:

• ts: True-score meta-analysis output. Represents fully corrected estimates.

• vgx: Validity generalization meta-analysis output with X treated as the predictor. Represents estimates corrected for all artifacts except measurement error in X. Artifact distributions will still account for variance in effects explained by measurement error in X.

• vgy: Validity generalization meta-analysis output with Y treated as the predictor. Represents estimates corrected for all artifacts except measurement error in Y. Artifact distributions will still account for variance in effects explained by measurement error in Y.

Author(s)

Maintainer: Jeffrey A. Dahlke [e-mail]

Authors:

• Brenton M. Wiernik [e-mail]

Other contributors:

• Wesley Gardiner (Unit tests) [contributor]

• Michael T. Brannick (Testing) [contributor]

• Jack Kostal (Code for reshape_mat2dat function) [contributor]

• Sean Potter (Testing; Code for cumulative and leave1out plots) [contributor]

• John Sakaluk (Code for funnel and forest plots) [contributor]

• Yuejia (Mandy) Teng (Testing) [contributor]

See Also

Useful links:

• Report bugs at https://github.com/psychmeta/psychmeta/issues

psychmeta documentation built on June 22, 2024, 6:52 p.m.