ma_r: Meta-analysis of correlations
In psychmeta: Psychometric Meta-Analysis Toolkit
ma_r R Documentation
Meta-analysis of correlations
Description
The ma_r_bb, ma_r_ic, and ma_r_ad functions implement bare-bones, individual-correction, and artifact-distribution correction methods for correlations, respectively.
The ma_r function is the master function for meta-analyses of correlations - it facilitates the computation of bare-bones, artifact-distribution, and individual-correction meta-analyses of correlations for any number of construct pairs.
When artifact-distribution meta-analyses are performed, ma_r will automatically extract the artifact information from a database and organize it into the requested type of artifact distribution object (i.e., either Taylor series or interactive artifact distributions).
ma_r is also equipped with the capability to clean databases containing inconsistently recorded artifact data, impute missing artifacts (when individual-correction meta-analyses are requested), and remove dependency among samples by forming composites or averaging effect sizes and artifacts.
The automatic compositing features in ma_r are employed when sample_ids and/or construct names are provided.
Usage
ma_r(
rxyi,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
ma_method = c("bb", "ic", "ad"),
ad_type = c("tsa", "int"),
correction_method = "auto",
construct_x = NULL,
construct_y = NULL,
facet_x = NULL,
facet_y = NULL,
measure_x = NULL,
measure_y = NULL,
construct_order = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
correct_rel = NULL,
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr = NULL,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr = NULL,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
rxx = NULL,
rxx_restricted = TRUE,
rxx_type = "alpha",
k_items_x = NULL,
ryy = NULL,
ryy_restricted = TRUE,
ryy_type = "alpha",
k_items_y = NULL,
ux = NULL,
ux_observed = TRUE,
uy = NULL,
uy_observed = TRUE,
sign_rz = NULL,
sign_rxz = 1,
sign_ryz = 1,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
supplemental_ads = NULL,
data = NULL,
control = control_psychmeta(),
...
)
ma_r_ad(
ma_obj,
ad_obj_x = NULL,
ad_obj_y = NULL,
correction_method = "auto",
use_ic_ads = c("tsa", "int"),
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
sign_rxz = 1,
sign_ryz = 1,
control = control_psychmeta(),
...
)
ma_r_bb(
r,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
data = NULL,
control = control_psychmeta(),
...
)
ma_r_ic(
rxyi,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
rxx = NULL,
rxx_restricted = TRUE,
rxx_type = "alpha",
k_items_x = NULL,
ryy = NULL,
ryy_restricted = TRUE,
ryy_type = "alpha",
k_items_y = NULL,
ux = NULL,
ux_observed = TRUE,
uy = NULL,
uy_observed = TRUE,
sign_rxz = 1,
sign_ryz = 1,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
supplemental_ads_x = NULL,
supplemental_ads_y = NULL,
data = NULL,
control = control_psychmeta(),
...
)
Arguments
rxyi, r
Vector or column name of observed correlations. The r argument is used with the ma_r_bb (i.e., the barebones function) function and the rxyi argument is used with ma_r and ma_r_ic (i.e., the function in which corrections are applied).
NOTE: Beginning in psychmeta version 2.5.2, rxyi values of exactly 0 in individual-correction meta-analyses are replaced with a functionally equivalent value via the zero_substitute argument for control_psychmeta to facilitate the estimation of corrected error variances.
n
Vector or column name of sample sizes.
n_adj
Optional: Vector or column name of sample sizes adjusted for sporadic artifact corrections.
sample_id
Optional vector of identification labels for samples/studies in the meta-analysis.
citekey
Optional vector of bibliographic citation keys for samples/studies in the meta-analysis (if multiple citekeys pertain to a given effect size, combine them into a single string entry with comma delimiters (e.g., "citkey1,citekey2").
ma_method
Method to be used to compute the meta-analysis: "bb" (barebones), "ic" (individual correction), or "ad" (artifact distribution).
ad_type
For when ma_method is "ad". Dpecifies the type of artifact distribution to use: "int" or "tsa".
correction_method
For when ma_method is "ad". Character scalar or a square matrix with the collective levels of construct_x and construct_y as row names and column names.
Select one of the following methods for correcting artifacts: "auto", "meas", "uvdrr", "uvirr", "bvdrr", "bvirr",
"rbOrig", "rb1Orig", "rb2Orig", "rbAdj", "rb1Adj", and "rb2Adj".
(note: "rb1Orig", "rb2Orig", "rb1Adj", and "rb2Adj" can only be used when Taylor series artifact distributions are provided and "rbOrig" and "rbAdj" can only
be used when interactive artifact distributions are provided). See "Details" of ma_r_ad for descriptions of the available methods.
construct_x, construct_y
Vector of construct names for constructs initially designated as "X" or as "Y".
facet_x, facet_y
Vector of facet names for constructs initially designated as "X" or as "Y".
Facet names "global", "overall", and "total" are reserved to indicate observations that represent effect sizes that have already been composited or that represent construct-level measurements rather than facet-level measurements.
To avoid double-compositing, any observation with one of these reserved names will only be eligible for auto-compositing with other such observations and will not be combined with narrow facets.
measure_x, measure_y
Vector of names for measures associated with constructs initially designated as "X" or as "Y".
construct_order
Vector indicating the order in which variables should be arranged, with variables listed earlier in the vector being preferred for designation as X.
wt_type
Type of weight to use in the meta-analysis: options are "sample_size", "inv_var_mean" (inverse variance computed using mean effect size), and
"inv_var_sample" (inverse variance computed using sample-specific effect sizes). Supported options borrowed from metafor are "DL", "HE", "HS", "SJ", "ML", "REML", "EB", and "PM"
(see metafor documentation for details about the metafor methods).
correct_bias
Logical scalar that determines whether to correct correlations for small-sample bias (TRUE) or not (FALSE).
correct_rel
Optional named vector that supersedes correct_rxx and correct_ryy. Names should correspond to construct names in construct_x and construct_y to determine which constructs should be corrected for unreliability.
correct_rxx, correct_ryy
Logical scalar or vector that determines whether to correct the X or Y variable for measurement error (TRUE) or not (FALSE).
correct_rr
Optional named vector that supersedes correct_rr_x and correct_rr_y. Names should correspond to construct names in construct_x and construct_y to determine which constructs should be corrected for range restriction.
correct_rr_x
Logical scalar, logical vector, or column name determining whether each correlation in rxyi should be corrected for range restriction in X (TRUE) or not (FALSE). If using artifact distribution methods, this must be a scalar value.
correct_rr_y
Logical scalar, logical vector, or column name determining whether each correlation in rxyi should be corrected for range restriction in Y (TRUE) or not (FALSE). If using artifact distribution methods, this must be a scalar value.
indirect_rr
Optional named vector that supersedes indirect_rr_x and indirect_rr_y. Names should correspond to construct names in construct_x and construct_y to determine which constructs should be corrected for indirect range restriction.
indirect_rr_x
Logical vector or column name determining whether each correlation in rxyi should be corrected for indirect range restriction in X (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_x (i.e., if correct_rr_x == FALSE, the value supplied for indirect_rr_x is disregarded).
indirect_rr_y
Logical vector or column name determining whether each correlation in rxyi should be corrected for indirect range restriction in Y (TRUE) or not (FALSE).
Superseded in evaluation by correct_rr_y (i.e., if correct_rr_y == FALSE, the value supplied for indirect_rr_y is disregarded).
rxx
Vector or column name of reliability estimates for X.
rxx_restricted
Logical vector or column name determining whether each element of rxx is an incumbent reliability (TRUE) or an applicant reliability (FALSE).
rxx_type, ryy_type
String vector identifying the types of reliability estimates supplied. Acceptable reliability types are:
"internal_consistency": A generic designation for internal-consistency reliability estimates derived from responses to a single test administration.
"multiple_administrations": A generic designation for reliability estimates derived from multiple administrations of a test.
"alpha": Coefficient alpha.
"lambda": Generic designation for a Guttman's lambda coefficient.
"lambda1": Guttman's lambda 1 coefficient.
"lambda2": Guttman's lambda 2 coefficient.
"lambda3": Guttman's lambda 3 coefficient.
"lambda4": Guttman's lambda 4 coefficient.
"lambda5": Guttman's lambda 5 coefficient.
"lambda6": Guttman's lambda 6 coefficient.
"omega": Omega coefficient indicating the proportion variance in a variable accounted for by modeled latent factors.
"icc": Intraclass correlation coefficient.
"interrater_r": Inter-rater correlation coefficient.
"interrater_r_sb": Inter-rater correlation coefficient, stepped up with the Spearman-Brown formula.
"splithalf": Split-half reliability coefficient.
"splithalf_sb": Split-half reliability coefficient, corrected toward the full test length with the Spearman-Brown formula.
"retest": Test-retest reliability coefficient.
"parallel": Parallel-forms reliability coefficient with tests taken during the same testing session.
"alternate": Alternate-forms reliability coefficient with tests taken during the same testing session.
"parallel_delayed": Parallel-forms reliability coefficient with tests taken during separate testing sessions with a time delay in between.
"alternate_delayed": Alternate-forms reliability coefficient with tests taken during separate testing sessions with a time delay in between.
k_items_x, k_items_y
Numeric vector identifying the number of items in each scale.
ryy
Vector or column name of reliability estimates for Y.
ryy_restricted
Logical vector or column name determining whether each element of ryy is an incumbent reliability (TRUE) or an applicant reliability (FALSE).
ux
Vector or column name of u ratios for X.
ux_observed
Logical vector or column name determining whether each element of ux is an observed-score u ratio (TRUE) or a true-score u ratio (FALSE).
uy
Vector or column name of u ratios for Y.
uy_observed
Logical vector or column name determining whether each element of uy is an observed-score u ratio (TRUE) or a true-score u ratio (FALSE).
sign_rz
Optional named vector that supersedes sign_rxz and sign_ryz. Names should correspond to construct names in construct_x and construct_y to determine the sign of each construct's relationship with the selection mechanism.
sign_rxz
Sign of the relationship between X and the selection mechanism (for use with bvirr corrections only).
sign_ryz
Sign of the relationship between Y and the selection mechanism (for use with bvirr corrections only).
moderators
Either (1) a vector of column names in data of moderator variables to be used in the meta-analysis (names can be quoted or unquoted), or (2) a vector, data frame, or matrix containing moderator variables.
cat_moderators
Either (1) A character vector listing the variable names in moderators that are categorical, or (2) a logical scalar or vector identifying whether each variable in moderators is categorical (TRUE) or continuous (FALSE).
moderator_type
Type of moderator analysis: "none" means that no moderators are to be used, "simple" means that moderators are to be examined one at a time, and
"hierarchical" means that all possible combinations and subsets of moderators are to be examined.
supplemental_ads
For ma_r only: Named list (named according to the constructs included in the meta-analysis) of supplemental artifact distribution information from studies not included in the meta-analysis. This is a list of lists, where the elements of a list associated with a construct are named like the arguments of the create_ad() function.
data
Data frame containing columns whose names may be provided as arguments to vector arguments and/or moderators.
control
Output from the control_psychmeta() function or a list of arguments controlled by the control_psychmeta() function. Ellipsis arguments will be screened for internal inclusion in control.
...
Further arguments to be passed to functions called within the meta-analysis.
ma_obj
For ma_r_ad only: Meta-analysis object of correlations or d values (regardless of input metric, output metric will be r).
ad_obj_x
For ma_r_ad only: Artifact-distribution object for the X variable (output of the create_ad function).
If ma_obj is of the class ma_master (i.e,. the output of ma_r or ma_d), the object supplied for
ad_obj_x must be a named list of artifact distributions with names corresponding to the "X" constructs in the meta-analyses contained within ma_obj.
ad_obj_y
For ma_r_ad only: Artifact-distribution object for the Y variable (output of the create_ad function).
If ma_obj is of the class ma_master, the object supplied for ad_obj_y must be a named list of artifact distributions with names
corresponding to the "Y" constructs in the meta-analyses contained within ma_obj.
use_ic_ads
For ma_r_ad only: Determines whether artifact distributions should be extracted from the individual correction results in ma_obj.
Only evaluated when ad_obj_x or ad_obj_y is NULL and ma_obj does not contain individual correction results.
Use one of the following commands: tsa to use the Taylor series method or int to use the interactive method.
supplemental_ads_x, supplemental_ads_y
For ma_r_ic only: List supplemental artifact distribution information from studies not included in the meta-analysis. The elements of this list are named like the arguments of the create_ad() function.
Details
The options for correction_method are:
"auto": Automatic selection of the most appropriate correction procedure, based on the available artifacts and the logical arguments provided to the function. (default)
"meas": Correction for measurement error only.
"uvdrr": Correction for univariate direct range restriction (i.e., Case II). The choice of which variable to correct for range restriction is made using the correct_rr_x and correct_rr_y arguments.
"uvirr": Correction for univariate indirect range restriction (i.e., Case IV). The choice of which variable to correct for range restriction is made using the correct_rr_x and correct_rr_y arguments.
"bvdrr": Correction for bivariate direct range restriction. Use with caution: This correction is an approximation only and is known to have a positive bias.
"bvirr": Correction for bivariate indirect range restriction (i.e., Case V).
"rbOrig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. We recommend using "uvdrr" instead.
"rbAdj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb1Orig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. We recommend using "uvdrr" instead.
"rb1Adj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb2Orig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. We recommend using "uvdrr" instead.
"rb2Adj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
Value
A nested tabular object of the class "ma_psychmeta".
Components of output tables for bare-bones meta-analyses:
pair_id: Unique identification number for each construct pairing.
construct_x: Name of the variable analyzed as construct X.
construct_y: Name of the variable analyzed as construct Y.
analysis_id: Unique identification number for each analysis.
analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k: Number of effect sizes meta-analyzed.
N: Total sample size of all effect sizes in the meta-analysis.
mean_r: Mean observed correlation.
var_r: Weighted variance of observed correlations.
var_e: Predicted sampling-error variance of observed correlations.
var_res: Variance of observed correlations after removing predicted sampling-error variance.
sd_r: Square root of var_r.
se_r: Standard error of mean_r.
sd_e: Square root of var_e.
sd_res: Square root of var_res.
CI_LL_XX: Lower limit of the confidence interval around mean_r, where "XX" represents the confidence level as a percentage.
CI_UL_XX: Upper limit of the confidence interval around mean_r, where "XX" represents the confidence level as a percentage.
CR_LL_XX: Lower limit of the credibility interval around mean_r, where "XX" represents the credibility level as a percentage.
CR_UL_XX: Upper limit of the credibility interval around mean_r, where "XX" represents the credibility level as a percentage.
Components of output tables for individual-correction meta-analyses:
pair_id: Unique identification number for each construct pairing.
construct_x: Name of the variable analyzed as construct X.
construct_y: Name of the variable analyzed as construct Y.
analysis_id: Unique identification number for each analysis.
analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k: Number of effect sizes meta-analyzed.
N: Total sample size of all effect sizes in the meta-analysis.
mean_r: Mean observed correlation.
var_r: Weighted variance of observed correlations.
var_e: Predicted sampling-error variance of observed correlations.
var_res: Variance of observed correlations after removing predicted sampling-error variance.
sd_r: Square root of var_r.
se_r: Standard error of mean_r.
sd_e: Square root of var_e.
sd_res: Square root of var_res.
mean_rho: Mean artifact-corrected correlation.
var_r_c: Variance of artifact-corrected correlations.
var_e_c: Predicted sampling-error variance of artifact-corrected correlations.
var_rho: Variance of artifact-corrected correlations after removing predicted sampling-error variance.
sd_r_c: Square root of var_r_c.
se_r_c: Standard error of mean_rho.
sd_e_c: Square root of var_e_c.
sd_rho: Square root of var_rho.
CI_LL_XX: Lower limit of the confidence interval around mean_rho, where "XX" represents the confidence level as a percentage.
CI_UL_XX: Upper limit of the confidence interval around mean_rho, where "XX" represents the confidence level as a percentage.
CR_LL_XX: Lower limit of the credibility interval around mean_rho, where "XX" represents the credibility level as a percentage.
CR_UL_XX: Upper limit of the credibility interval around mean_rho, where "XX" represents the credibility level as a percentage.
Components of output tables for artifact-distribution meta-analyses:
pair_id: Unique identification number for each construct pairing.
construct_x: Name of the variable analyzed as construct X.
construct_y: Name of the variable analyzed as construct Y.
analysis_id: Unique identification number for each analysis.
analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.
k: Number of effect sizes meta-analyzed.
N: Total sample size of all effect sizes in the meta-analysis.
mean_r: Mean observed correlation.
var_r: Weighted variance of observed correlations.
var_e: Predicted sampling-error variance of observed correlations.
var_art: Amount of variance in observed correlations that is attributable to measurement-error and range-restriction artifacts.
var_pre: Total predicted artifactual variance (i.e., the sum of var_e and var_art).
var_res: Variance of observed correlations after removing predicted sampling-error variance and predicted artifact variance.
sd_r: Square root of var_r.
se_r: Standard error of mean_r.
sd_e: Square root of var_e.
sd_art: Square root of var_art.
sd_pre: Square root of var_pre.
sd_res: Square root of var_res.
mean_rho: Mean artifact-corrected correlation.
var_r_c: Weighted variance of observed correlations corrected to the metric of rho.
var_e_c: Predicted sampling-error variance of observed correlations corrected to the metric of rho.
var_art_c: Amount of variance in observed correlations that is attributable to measurement-error and range-restriction artifacts corrected to the metric of rho.
var_pre_c: Total predicted artifactual variance (i.e., the sum of var_e and var_art) corrected to the metric of rho.
var_rho: Variance of artifact-corrected correlations after removing predicted sampling-error variance and predicted artifact variance.
sd_r_c: Square root of var_r corrected to the metric of rho.
se_r_c: Standard error of mean_r corrected to the metric of rho.
sd_e_c: Square root of var_e corrected to the metric of rho.
sd_art_c: Square root of var_art corrected to the metric of rho.
sd_pre_c: Square root of var_pre corrected to the metric of rho.
sd_rho: Square root of var_rho.
CI_LL_XX: Lower limit of the confidence interval around mean_rho, where "XX" represents the confidence level as a percentage.
CI_UL_XX: Upper limit of the confidence interval around mean_rho, where "XX" represents the confidence level as a percentage.
CR_LL_XX: Lower limit of the credibility interval around mean_rho, where "XX" represents the credibility level as a percentage.
CR_UL_XX: Upper limit of the credibility interval around mean_rho, where "XX" represents the credibility level as a percentage.
Note
The difference between "rb" methods with the "orig" and "adj" suffixes is that the original does not account for the impact of range restriction on criterion reliabilities, whereas
the adjusted procedure attempts to estimate the applicant reliability information for the criterion. The "rb" procedures are included for posterity: We strongly recommend using
the "uvdrr" procedure to appropriately correct for univariate range restriction.
References
Schmidt, F. L., & Hunter, J. E. (2015).
Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.).
Sage. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.4135/9781483398105")}. Chapter 4.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994).
Nonlinearity of range corrections in meta-analysis: Test of an improved procedure.
Journal of Applied Psychology, 79(3), 425–438. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0021-9010.79.3.425")}
Dahlke, J. A., & Wiernik, B. M. (2020). Not restricted to selection research:
Accounting for indirect range restriction in organizational research.
Organizational Research Methods, 23(4), 717–749. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/1094428119859398")}
Raju, N. S., & Burke, M. J. (1983).
Two new procedures for studying validity generalization.
Journal of Applied Psychology, 68(3), 382–395. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0021-9010.68.3.382")}
Examples
## Not run:
## The 'ma_r' function can compute multi-construct bare-bones meta-analyses:
ma_obj <- ma_r(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
## It can also perform multiple individual-correction meta-analyses:
ma_obj <- ma_r(ma_method = "ic", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
ma_obj$meta_tables[[1]]$individual_correction$true_score
## And 'ma_r' can also curate artifact distributions and compute multiple
## artifact-distribution meta-analyses:
ma_obj <- ma_r(ma_method = "ad", ad_type = "int", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Even if no studies in the database provide artifact information,
## pre-specified artifact distributions from previous meta-analyses
## can still be used! (These results should match the previous example.)
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi,
supplemental_ads =
list(X = list(mean_qxi = 0.8927818, var_qxi = 0.0008095520, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha"),
Y = list(mean_qxi = 0.8941266, var_qxi = 0.0009367234, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha"),
Z = list(mean_qxi = 0.8962108, var_qxi = 0.0007840593, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha")))
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Artifact information may also be supplied by passing "ad_obj" class objects with the
## "supplemental_ads" argument.
## Create a list of artifact-distribution objects:
ad_list <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
data = data_r_meas_multi)
ad_list <- setNames(ad_list$ad_x, ad_list$construct_x)
## Run the artifact-distribution meta-analysis:
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi,
supplemental_ads = ad_list)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Artifact information from studies not included in the meta-analysis can also be used to make
## corrections. Passing artifact information with the 'supplemental_ads' argument allows for
## additional artifact values and/or means and variances of artifacts to be used.
## The 'supplemental_ads' analysis below gives the same results as the prior meta-analysis.
x_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "X"
rxxi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[x_ids]
n_rxxi = c(data_r_meas_multi$n, data_r_meas_multi$n)[x_ids]
y_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "Y"
ryyi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[y_ids]
n_ryyi = c(data_r_meas_multi$n, data_r_meas_multi$n)[y_ids]
z_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "Z"
rzzi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[z_ids]
n_rzzi = c(data_r_meas_multi$n, data_r_meas_multi$n)[z_ids]
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name,
moderators = moderator, sample_id = sample_id, data = data_r_meas_multi,
supplemental_ads = list(X = list(rxxi = rxxi, n_rxxi = n_rxxi, wt_rxxi = n_rxxi),
Y = list(rxxi = ryyi, n_rxxi = n_ryyi, wt_rxxi = n_ryyi),
Z = list(rxxi = rzzi, n_rxxi = n_rzzi, wt_rxxi = n_rzzi)))
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## If 'use_all_arts' is set to TRUE, artifacts from studies without valid correlations
## will be used to inform artifact distributions. Below, correlations and artifacts
## are provided by non-overlapping sets of studies.
dat1 <- dat2 <- data_r_meas_multi
dat1$rxxi <- dat1$ryyi <- NA
dat2$rxyi <- NA
dat2$sample_id <- dat2$sample_id + 40
dat <- rbind(dat1, dat2)
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id, moderators = moderator,
use_all_arts = TRUE, data = dat)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
### Demonstration of ma_r_bb ###
## Example analysis using data from Gonzalez-Mule et al. (2014):
## Not correcting for bias and using normal distributions to compute uncertainty intervals
## allows for exact replication of the results reported in the text:
ma_r_bb(r = rxyi, n = n, correct_bias = FALSE, conf_method = "norm", cred_method = "norm",
data = data_r_gonzalezmule_2014)
## Using hs_override = TRUE allows one to easily implement the traditional Hunter-Schmidt method:
ma_r_bb(r = rxyi, n = n, hs_override = TRUE, data = data_r_gonzalezmule_2014)
## With hs_override = FALSE, the program defaults will compute unbiased variances and use
## t-distributions to estimate confidence and credibility intervals - these settings make
## a noticeable difference for small studies like the textbook example:
ma_r_bb(r = rxyi, n = n, hs_override = FALSE, data = data_r_gonzalezmule_2014)
### Demonstration of ma_r_ic ###
## Simulated example satisfying the assumptions of the Case IV
## range-restriction correction (parameter values: mean_rho = .3, sd_rho = .15):
ma_r_ic(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi, ux = ux, data = data_r_uvirr)
## Simulated example satisfying the assumptions of the Case V
## range-restriction correction
ma_r_ic(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
rxx_type = "parallel", ryy_type = "parallel",
ux = ux, uy = uy, data = data_r_bvirr)
## Published example from Gonzalez-Mule et al. (2014)
ma_r_ic(rxyi = rxyi, n = n, hs_override = TRUE, data = data_r_gonzalezmule_2014,
rxx = rxxi, ryy = ryyi, ux = ux, indirect_rr_x = TRUE,
moderators = c("Rating source", "Published", "Type", "Complexity"))
### Demonstration of ma_r_ad ###
## Compute barebones meta-analysis
ma_obj <- ma_r_bb(r = rxyi, n = n, correct_bias = FALSE,
conf_method = "norm", cred_method = "norm", data = data_r_mcdaniel_1994)
## Construct artifact distribution for X
ad_obj_x <- create_ad(ad_type = "tsa", mean_rxxi = data_r_mcdaniel_1994$Mrxxi[1],
var_rxxi = data_r_mcdaniel_1994$SDrxxi[1]^.5,
ux = data_r_mcdaniel_1994$ux,
wt_ux = data_r_mcdaniel_1994$`ux frequency`)
## Construct artifact distribution for Y
ad_obj_y <- create_ad(ad_type = "tsa", rxxi = data_r_mcdaniel_1994$ryyi,
wt_rxxi = data_r_mcdaniel_1994$`ryyi frequency`)
## Compute artifact-distribution meta-analysis, correcting for measurement error only
ma_r_ad(ma_obj = ma_obj, ad_obj_x = ad_obj_x, ad_obj_y = ad_obj_y, correction_method = "meas")
## Compute artifact-distribution meta-analysis, correcting for univariate direct range restriction
ma_r_ad(ma_obj = ma_obj, ad_obj_x = ad_obj_x, ad_obj_y = ad_obj_y, correction_method = "uvdrr",
correct_rr_y = FALSE, indirect_rr_x = FALSE)
# The results of ma_r() can also be corrected using artifact distributions
ma_obj <- ma_r(ma_method = "bb", rxyi = rxyi, n = n,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
# The create_ad_list function can be used to generate batches of artifact-distribution objects.
# Here is an example in which one distribution is created per construct.
ad_tibble <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
data = data_r_meas_multi)
# Passing that collection of distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_tibble <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble, ad_obj_y = ad_tibble)
summary(ma_obj_tibble)
ma_obj_tibble$meta_tables[[1]]$artifact_distribution$true_score
# The same outcomes as the previous example can be achieved by passing a named list of
# artifact information, with each element bearing the name of a construct:
ad_list <- setNames(ad_tibble$ad_x, ad_tibble$construct_x)
ma_obj_list <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_list, ad_obj_y = ad_list)
summary(ma_obj_list)
ma_obj_list$meta_tables[[1]]$artifact_distribution$true_score
# It is also possible to construct artifact distributions in a pairwise fashion.
# For example, if correlations between X and Y and between X and Z are being analyzed,
# X will get a different distribution for its relationships with Y than with Z.
# These pairwise distributions are based only on artifact data from specific construct pairs.
ad_tibble_pair <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(pairwise_ads = TRUE),
data = data_r_meas_multi)
# Passing these pairwise distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pair <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_pair, ad_obj_y = ad_tibble_pair)
summary(ma_obj_pair)
ma_obj_pair$meta_tables[[1]]$artifact_distribution$true_score
# Sometimes moderators have important influences on artifact distributions as well as
# distributions of effect sizes. When this occurs, moderated artifact distributions
# can be created to make more appropriate corrections.
ad_tibble_mod <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(moderated_ads = TRUE),
moderators = moderator,
data = data_r_meas_multi)
# Passing these moderated distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_mod <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_mod, ad_obj_y = ad_tibble_mod)
summary(ma_obj_mod)
ma_obj_mod$meta_tables[[1]]$artifact_distribution$true_score
# It is also possible to create pairwise moderated artifact distributions.
ad_tibble_pairmod <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(moderated_ads = TRUE,
pairwise_ads = TRUE),
moderators = moderator,
data = data_r_meas_multi)
# Passing these pairwise moderated distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pairmod <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_pairmod, ad_obj_y = ad_tibble_pairmod)
summary(ma_obj_pairmod)
ma_obj_pairmod$meta_tables[[1]]$artifact_distribution$true_score
# For even more control over which artifact distributions are used in corrections, you can supply
# un-named list of distributions in which the order of distributions corresponds to the order of
# meta-analyses in ma_obj. It is important for the elements to be un-named, as the absence of names
# and the length of the list are the two ways in which ma_r_ad() validates the lists.
ad_list_pairmod_x <- ad_tibble_pairmod$ad_x
ad_list_pairmod_y <- ad_tibble_pairmod$ad_y
# Passing these lists of distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pairmodlist <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_list_pairmod_x, ad_obj_y = ad_list_pairmod_y)
summary(ma_obj_pairmodlist)
ma_obj_pairmodlist$meta_tables[[1]]$artifact_distribution$true_score
## End(Not run)
psychmeta documentation built on June 22, 2024, 6:52 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| ma_r | R Documentation |
Meta-analysis of correlations
Description
The ma_r_bb, ma_r_ic, and ma_r_ad functions implement bare-bones, individual-correction, and artifact-distribution correction methods for correlations, respectively.
The ma_r function is the master function for meta-analyses of correlations - it facilitates the computation of bare-bones, artifact-distribution, and individual-correction meta-analyses of correlations for any number of construct pairs.
When artifact-distribution meta-analyses are performed, ma_r will automatically extract the artifact information from a database and organize it into the requested type of artifact distribution object (i.e., either Taylor series or interactive artifact distributions).
ma_r is also equipped with the capability to clean databases containing inconsistently recorded artifact data, impute missing artifacts (when individual-correction meta-analyses are requested), and remove dependency among samples by forming composites or averaging effect sizes and artifacts.
The automatic compositing features in ma_r are employed when sample_ids and/or construct names are provided.
Usage
ma_r(
rxyi,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
ma_method = c("bb", "ic", "ad"),
ad_type = c("tsa", "int"),
correction_method = "auto",
construct_x = NULL,
construct_y = NULL,
facet_x = NULL,
facet_y = NULL,
measure_x = NULL,
measure_y = NULL,
construct_order = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
correct_rel = NULL,
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr = NULL,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr = NULL,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
rxx = NULL,
rxx_restricted = TRUE,
rxx_type = "alpha",
k_items_x = NULL,
ryy = NULL,
ryy_restricted = TRUE,
ryy_type = "alpha",
k_items_y = NULL,
ux = NULL,
ux_observed = TRUE,
uy = NULL,
uy_observed = TRUE,
sign_rz = NULL,
sign_rxz = 1,
sign_ryz = 1,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
supplemental_ads = NULL,
data = NULL,
control = control_psychmeta(),
...
)
ma_r_ad(
ma_obj,
ad_obj_x = NULL,
ad_obj_y = NULL,
correction_method = "auto",
use_ic_ads = c("tsa", "int"),
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
sign_rxz = 1,
sign_ryz = 1,
control = control_psychmeta(),
...
)
ma_r_bb(
r,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
data = NULL,
control = control_psychmeta(),
...
)
ma_r_ic(
rxyi,
n,
n_adj = NULL,
sample_id = NULL,
citekey = NULL,
wt_type = c("sample_size", "inv_var_mean", "inv_var_sample", "DL", "HE", "HS", "SJ",
"ML", "REML", "EB", "PM"),
correct_bias = TRUE,
correct_rxx = TRUE,
correct_ryy = TRUE,
correct_rr_x = TRUE,
correct_rr_y = TRUE,
indirect_rr_x = TRUE,
indirect_rr_y = TRUE,
rxx = NULL,
rxx_restricted = TRUE,
rxx_type = "alpha",
k_items_x = NULL,
ryy = NULL,
ryy_restricted = TRUE,
ryy_type = "alpha",
k_items_y = NULL,
ux = NULL,
ux_observed = TRUE,
uy = NULL,
uy_observed = TRUE,
sign_rxz = 1,
sign_ryz = 1,
moderators = NULL,
cat_moderators = TRUE,
moderator_type = c("simple", "hierarchical", "none"),
supplemental_ads_x = NULL,
supplemental_ads_y = NULL,
data = NULL,
control = control_psychmeta(),
...
)
Arguments
rxyi, r |
Vector or column name of observed correlations. The |
n |
Vector or column name of sample sizes. |
n_adj |
Optional: Vector or column name of sample sizes adjusted for sporadic artifact corrections. |
sample_id |
Optional vector of identification labels for samples/studies in the meta-analysis. |
citekey |
Optional vector of bibliographic citation keys for samples/studies in the meta-analysis (if multiple citekeys pertain to a given effect size, combine them into a single string entry with comma delimiters (e.g., "citkey1,citekey2"). |
ma_method |
Method to be used to compute the meta-analysis: "bb" (barebones), "ic" (individual correction), or "ad" (artifact distribution). |
ad_type |
For when ma_method is "ad". Dpecifies the type of artifact distribution to use: "int" or "tsa". |
correction_method |
For when ma_method is "ad". Character scalar or a square matrix with the collective levels of |
construct_x, construct_y |
Vector of construct names for constructs initially designated as "X" or as "Y". |
facet_x, facet_y |
Vector of facet names for constructs initially designated as "X" or as "Y". Facet names "global", "overall", and "total" are reserved to indicate observations that represent effect sizes that have already been composited or that represent construct-level measurements rather than facet-level measurements. To avoid double-compositing, any observation with one of these reserved names will only be eligible for auto-compositing with other such observations and will not be combined with narrow facets. |
measure_x, measure_y |
Vector of names for measures associated with constructs initially designated as "X" or as "Y". |
construct_order |
Vector indicating the order in which variables should be arranged, with variables listed earlier in the vector being preferred for designation as X. |
wt_type |
Type of weight to use in the meta-analysis: options are "sample_size", "inv_var_mean" (inverse variance computed using mean effect size), and "inv_var_sample" (inverse variance computed using sample-specific effect sizes). Supported options borrowed from metafor are "DL", "HE", "HS", "SJ", "ML", "REML", "EB", and "PM" (see metafor documentation for details about the metafor methods). |
correct_bias |
Logical scalar that determines whether to correct correlations for small-sample bias ( |
correct_rel |
Optional named vector that supersedes |
correct_rxx, correct_ryy |
Logical scalar or vector that determines whether to correct the X or Y variable for measurement error ( |
correct_rr |
Optional named vector that supersedes |
correct_rr_x |
Logical scalar, logical vector, or column name determining whether each correlation in |
correct_rr_y |
Logical scalar, logical vector, or column name determining whether each correlation in |
indirect_rr |
Optional named vector that supersedes |
indirect_rr_x |
Logical vector or column name determining whether each correlation in |
indirect_rr_y |
Logical vector or column name determining whether each correlation in |
rxx |
Vector or column name of reliability estimates for X. |
rxx_restricted |
Logical vector or column name determining whether each element of |
rxx_type, ryy_type |
String vector identifying the types of reliability estimates supplied. Acceptable reliability types are:
|
k_items_x, k_items_y |
Numeric vector identifying the number of items in each scale. |
ryy |
Vector or column name of reliability estimates for Y. |
ryy_restricted |
Logical vector or column name determining whether each element of |
ux |
Vector or column name of u ratios for X. |
ux_observed |
Logical vector or column name determining whether each element of ux is an observed-score u ratio ( |
uy |
Vector or column name of u ratios for Y. |
uy_observed |
Logical vector or column name determining whether each element of uy is an observed-score u ratio ( |
sign_rz |
Optional named vector that supersedes |
sign_rxz |
Sign of the relationship between X and the selection mechanism (for use with bvirr corrections only). |
sign_ryz |
Sign of the relationship between Y and the selection mechanism (for use with bvirr corrections only). |
moderators |
Either (1) a vector of column names in |
cat_moderators |
Either (1) A character vector listing the variable names in |
moderator_type |
Type of moderator analysis: "none" means that no moderators are to be used, "simple" means that moderators are to be examined one at a time, and "hierarchical" means that all possible combinations and subsets of moderators are to be examined. |
supplemental_ads |
For |
data |
Data frame containing columns whose names may be provided as arguments to vector arguments and/or moderators. |
control |
Output from the |
... |
Further arguments to be passed to functions called within the meta-analysis. |
ma_obj |
For |
ad_obj_x |
For |
ad_obj_y |
For |
use_ic_ads |
For |
supplemental_ads_x, supplemental_ads_y |
For |
Details
The options for correction_method are:
"auto": Automatic selection of the most appropriate correction procedure, based on the available artifacts and the logical arguments provided to the function. (default)
"meas": Correction for measurement error only.
"uvdrr": Correction for univariate direct range restriction (i.e., Case II). The choice of which variable to correct for range restriction is made using the
correct_rr_xandcorrect_rr_yarguments."uvirr": Correction for univariate indirect range restriction (i.e., Case IV). The choice of which variable to correct for range restriction is made using the
correct_rr_xandcorrect_rr_yarguments."bvdrr": Correction for bivariate direct range restriction. Use with caution: This correction is an approximation only and is known to have a positive bias.
"bvirr": Correction for bivariate indirect range restriction (i.e., Case V).
"rbOrig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. We recommend using "uvdrr" instead.
"rbAdj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied interactively. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb1Orig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. We recommend using "uvdrr" instead.
"rb1Adj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA1 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
"rb2Orig": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. We recommend using "uvdrr" instead.
"rb2Adj": Not recommended: Raju and Burke's version of the correction for direct range restriction, applied using their TSA2 method. Adjusted to account for range restriction in the reliability of the Y variable. We recommend using "uvdrr" instead.
Value
A nested tabular object of the class "ma_psychmeta". Components of output tables for bare-bones meta-analyses:
pair_id: Unique identification number for each construct pairing.construct_x: Name of the variable analyzed as construct X.construct_y: Name of the variable analyzed as construct Y.analysis_id: Unique identification number for each analysis.analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.k: Number of effect sizes meta-analyzed.N: Total sample size of all effect sizes in the meta-analysis.mean_r: Mean observed correlation.var_r: Weighted variance of observed correlations.var_e: Predicted sampling-error variance of observed correlations.var_res: Variance of observed correlations after removing predicted sampling-error variance.sd_r: Square root ofvar_r.se_r: Standard error ofmean_r.sd_e: Square root ofvar_e.sd_res: Square root ofvar_res.CI_LL_XX: Lower limit of the confidence interval aroundmean_r, where "XX" represents the confidence level as a percentage.CI_UL_XX: Upper limit of the confidence interval aroundmean_r, where "XX" represents the confidence level as a percentage.CR_LL_XX: Lower limit of the credibility interval aroundmean_r, where "XX" represents the credibility level as a percentage.CR_UL_XX: Upper limit of the credibility interval aroundmean_r, where "XX" represents the credibility level as a percentage.
Components of output tables for individual-correction meta-analyses:
pair_id: Unique identification number for each construct pairing.construct_x: Name of the variable analyzed as construct X.construct_y: Name of the variable analyzed as construct Y.analysis_id: Unique identification number for each analysis.analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.k: Number of effect sizes meta-analyzed.N: Total sample size of all effect sizes in the meta-analysis.mean_r: Mean observed correlation.var_r: Weighted variance of observed correlations.var_e: Predicted sampling-error variance of observed correlations.var_res: Variance of observed correlations after removing predicted sampling-error variance.sd_r: Square root ofvar_r.se_r: Standard error ofmean_r.sd_e: Square root ofvar_e.sd_res: Square root ofvar_res.mean_rho: Mean artifact-corrected correlation.var_r_c: Variance of artifact-corrected correlations.var_e_c: Predicted sampling-error variance of artifact-corrected correlations.var_rho: Variance of artifact-corrected correlations after removing predicted sampling-error variance.sd_r_c: Square root ofvar_r_c.se_r_c: Standard error ofmean_rho.sd_e_c: Square root ofvar_e_c.sd_rho: Square root ofvar_rho.CI_LL_XX: Lower limit of the confidence interval aroundmean_rho, where "XX" represents the confidence level as a percentage.CI_UL_XX: Upper limit of the confidence interval aroundmean_rho, where "XX" represents the confidence level as a percentage.CR_LL_XX: Lower limit of the credibility interval aroundmean_rho, where "XX" represents the credibility level as a percentage.CR_UL_XX: Upper limit of the credibility interval aroundmean_rho, where "XX" represents the credibility level as a percentage.
Components of output tables for artifact-distribution meta-analyses:
pair_id: Unique identification number for each construct pairing.construct_x: Name of the variable analyzed as construct X.construct_y: Name of the variable analyzed as construct Y.analysis_id: Unique identification number for each analysis.analysis_type: Type of moderator analyses: Overall, Simple Moderator, or Hierarchical Moderator.k: Number of effect sizes meta-analyzed.N: Total sample size of all effect sizes in the meta-analysis.mean_r: Mean observed correlation.var_r: Weighted variance of observed correlations.var_e: Predicted sampling-error variance of observed correlations.var_art: Amount of variance in observed correlations that is attributable to measurement-error and range-restriction artifacts.var_pre: Total predicted artifactual variance (i.e., the sum ofvar_eandvar_art).var_res: Variance of observed correlations after removing predicted sampling-error variance and predicted artifact variance.sd_r: Square root ofvar_r.se_r: Standard error ofmean_r.sd_e: Square root ofvar_e.sd_art: Square root ofvar_art.sd_pre: Square root ofvar_pre.sd_res: Square root ofvar_res.mean_rho: Mean artifact-corrected correlation.var_r_c: Weighted variance of observed correlations corrected to the metric of rho.var_e_c: Predicted sampling-error variance of observed correlations corrected to the metric of rho.var_art_c: Amount of variance in observed correlations that is attributable to measurement-error and range-restriction artifacts corrected to the metric of rho.var_pre_c: Total predicted artifactual variance (i.e., the sum ofvar_eandvar_art) corrected to the metric of rho.var_rho: Variance of artifact-corrected correlations after removing predicted sampling-error variance and predicted artifact variance.sd_r_c: Square root ofvar_rcorrected to the metric of rho.se_r_c: Standard error ofmean_rcorrected to the metric of rho.sd_e_c: Square root ofvar_ecorrected to the metric of rho.sd_art_c: Square root ofvar_artcorrected to the metric of rho.sd_pre_c: Square root ofvar_precorrected to the metric of rho.sd_rho: Square root ofvar_rho.CI_LL_XX: Lower limit of the confidence interval aroundmean_rho, where "XX" represents the confidence level as a percentage.CI_UL_XX: Upper limit of the confidence interval aroundmean_rho, where "XX" represents the confidence level as a percentage.CR_LL_XX: Lower limit of the credibility interval aroundmean_rho, where "XX" represents the credibility level as a percentage.CR_UL_XX: Upper limit of the credibility interval aroundmean_rho, where "XX" represents the credibility level as a percentage.
Note
The difference between "rb" methods with the "orig" and "adj" suffixes is that the original does not account for the impact of range restriction on criterion reliabilities, whereas the adjusted procedure attempts to estimate the applicant reliability information for the criterion. The "rb" procedures are included for posterity: We strongly recommend using the "uvdrr" procedure to appropriately correct for univariate range restriction.
References
Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Sage. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.4135/9781483398105")}. Chapter 4.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994). Nonlinearity of range corrections in meta-analysis: Test of an improved procedure. Journal of Applied Psychology, 79(3), 425–438. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0021-9010.79.3.425")}
Dahlke, J. A., & Wiernik, B. M. (2020). Not restricted to selection research: Accounting for indirect range restriction in organizational research. Organizational Research Methods, 23(4), 717–749. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1177/1094428119859398")}
Raju, N. S., & Burke, M. J. (1983). Two new procedures for studying validity generalization. Journal of Applied Psychology, 68(3), 382–395. \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1037/0021-9010.68.3.382")}
Examples
## Not run:
## The 'ma_r' function can compute multi-construct bare-bones meta-analyses:
ma_obj <- ma_r(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
## It can also perform multiple individual-correction meta-analyses:
ma_obj <- ma_r(ma_method = "ic", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
ma_obj$meta_tables[[1]]$individual_correction$true_score
## And 'ma_r' can also curate artifact distributions and compute multiple
## artifact-distribution meta-analyses:
ma_obj <- ma_r(ma_method = "ad", ad_type = "int", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Even if no studies in the database provide artifact information,
## pre-specified artifact distributions from previous meta-analyses
## can still be used! (These results should match the previous example.)
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi,
supplemental_ads =
list(X = list(mean_qxi = 0.8927818, var_qxi = 0.0008095520, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha"),
Y = list(mean_qxi = 0.8941266, var_qxi = 0.0009367234, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha"),
Z = list(mean_qxi = 0.8962108, var_qxi = 0.0007840593, k_qxi = 40,
mean_n_qxi = 11927 / 40, qxi_dist_type = "alpha")))
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Artifact information may also be supplied by passing "ad_obj" class objects with the
## "supplemental_ads" argument.
## Create a list of artifact-distribution objects:
ad_list <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
data = data_r_meas_multi)
ad_list <- setNames(ad_list$ad_x, ad_list$construct_x)
## Run the artifact-distribution meta-analysis:
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
clean_artifacts = FALSE, impute_artifacts = FALSE,
moderators = moderator, data = data_r_meas_multi,
supplemental_ads = ad_list)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## Artifact information from studies not included in the meta-analysis can also be used to make
## corrections. Passing artifact information with the 'supplemental_ads' argument allows for
## additional artifact values and/or means and variances of artifacts to be used.
## The 'supplemental_ads' analysis below gives the same results as the prior meta-analysis.
x_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "X"
rxxi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[x_ids]
n_rxxi = c(data_r_meas_multi$n, data_r_meas_multi$n)[x_ids]
y_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "Y"
ryyi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[y_ids]
n_ryyi = c(data_r_meas_multi$n, data_r_meas_multi$n)[y_ids]
z_ids <- c(data_r_meas_multi$x_name, data_r_meas_multi$y_name) == "Z"
rzzi <- c(data_r_meas_multi$rxxi, data_r_meas_multi$ryyi)[z_ids]
n_rzzi = c(data_r_meas_multi$n, data_r_meas_multi$n)[z_ids]
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name,
moderators = moderator, sample_id = sample_id, data = data_r_meas_multi,
supplemental_ads = list(X = list(rxxi = rxxi, n_rxxi = n_rxxi, wt_rxxi = n_rxxi),
Y = list(rxxi = ryyi, n_rxxi = n_ryyi, wt_rxxi = n_ryyi),
Z = list(rxxi = rzzi, n_rxxi = n_rzzi, wt_rxxi = n_rzzi)))
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
## If 'use_all_arts' is set to TRUE, artifacts from studies without valid correlations
## will be used to inform artifact distributions. Below, correlations and artifacts
## are provided by non-overlapping sets of studies.
dat1 <- dat2 <- data_r_meas_multi
dat1$rxxi <- dat1$ryyi <- NA
dat2$rxyi <- NA
dat2$sample_id <- dat2$sample_id + 40
dat <- rbind(dat1, dat2)
ma_obj <- ma_r(ma_method = "ad", rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
correct_rr_x = FALSE, correct_rr_y = FALSE,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id, moderators = moderator,
use_all_arts = TRUE, data = dat)
summary(ma_obj)
ma_obj$meta_tables[[1]]$artifact_distribution$true_score
### Demonstration of ma_r_bb ###
## Example analysis using data from Gonzalez-Mule et al. (2014):
## Not correcting for bias and using normal distributions to compute uncertainty intervals
## allows for exact replication of the results reported in the text:
ma_r_bb(r = rxyi, n = n, correct_bias = FALSE, conf_method = "norm", cred_method = "norm",
data = data_r_gonzalezmule_2014)
## Using hs_override = TRUE allows one to easily implement the traditional Hunter-Schmidt method:
ma_r_bb(r = rxyi, n = n, hs_override = TRUE, data = data_r_gonzalezmule_2014)
## With hs_override = FALSE, the program defaults will compute unbiased variances and use
## t-distributions to estimate confidence and credibility intervals - these settings make
## a noticeable difference for small studies like the textbook example:
ma_r_bb(r = rxyi, n = n, hs_override = FALSE, data = data_r_gonzalezmule_2014)
### Demonstration of ma_r_ic ###
## Simulated example satisfying the assumptions of the Case IV
## range-restriction correction (parameter values: mean_rho = .3, sd_rho = .15):
ma_r_ic(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi, ux = ux, data = data_r_uvirr)
## Simulated example satisfying the assumptions of the Case V
## range-restriction correction
ma_r_ic(rxyi = rxyi, n = n, rxx = rxxi, ryy = ryyi,
rxx_type = "parallel", ryy_type = "parallel",
ux = ux, uy = uy, data = data_r_bvirr)
## Published example from Gonzalez-Mule et al. (2014)
ma_r_ic(rxyi = rxyi, n = n, hs_override = TRUE, data = data_r_gonzalezmule_2014,
rxx = rxxi, ryy = ryyi, ux = ux, indirect_rr_x = TRUE,
moderators = c("Rating source", "Published", "Type", "Complexity"))
### Demonstration of ma_r_ad ###
## Compute barebones meta-analysis
ma_obj <- ma_r_bb(r = rxyi, n = n, correct_bias = FALSE,
conf_method = "norm", cred_method = "norm", data = data_r_mcdaniel_1994)
## Construct artifact distribution for X
ad_obj_x <- create_ad(ad_type = "tsa", mean_rxxi = data_r_mcdaniel_1994$Mrxxi[1],
var_rxxi = data_r_mcdaniel_1994$SDrxxi[1]^.5,
ux = data_r_mcdaniel_1994$ux,
wt_ux = data_r_mcdaniel_1994$`ux frequency`)
## Construct artifact distribution for Y
ad_obj_y <- create_ad(ad_type = "tsa", rxxi = data_r_mcdaniel_1994$ryyi,
wt_rxxi = data_r_mcdaniel_1994$`ryyi frequency`)
## Compute artifact-distribution meta-analysis, correcting for measurement error only
ma_r_ad(ma_obj = ma_obj, ad_obj_x = ad_obj_x, ad_obj_y = ad_obj_y, correction_method = "meas")
## Compute artifact-distribution meta-analysis, correcting for univariate direct range restriction
ma_r_ad(ma_obj = ma_obj, ad_obj_x = ad_obj_x, ad_obj_y = ad_obj_y, correction_method = "uvdrr",
correct_rr_y = FALSE, indirect_rr_x = FALSE)
# The results of ma_r() can also be corrected using artifact distributions
ma_obj <- ma_r(ma_method = "bb", rxyi = rxyi, n = n,
construct_x = x_name, construct_y = y_name, sample_id = sample_id,
moderators = moderator, data = data_r_meas_multi)
# The create_ad_list function can be used to generate batches of artifact-distribution objects.
# Here is an example in which one distribution is created per construct.
ad_tibble <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
data = data_r_meas_multi)
# Passing that collection of distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_tibble <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble, ad_obj_y = ad_tibble)
summary(ma_obj_tibble)
ma_obj_tibble$meta_tables[[1]]$artifact_distribution$true_score
# The same outcomes as the previous example can be achieved by passing a named list of
# artifact information, with each element bearing the name of a construct:
ad_list <- setNames(ad_tibble$ad_x, ad_tibble$construct_x)
ma_obj_list <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_list, ad_obj_y = ad_list)
summary(ma_obj_list)
ma_obj_list$meta_tables[[1]]$artifact_distribution$true_score
# It is also possible to construct artifact distributions in a pairwise fashion.
# For example, if correlations between X and Y and between X and Z are being analyzed,
# X will get a different distribution for its relationships with Y than with Z.
# These pairwise distributions are based only on artifact data from specific construct pairs.
ad_tibble_pair <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(pairwise_ads = TRUE),
data = data_r_meas_multi)
# Passing these pairwise distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pair <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_pair, ad_obj_y = ad_tibble_pair)
summary(ma_obj_pair)
ma_obj_pair$meta_tables[[1]]$artifact_distribution$true_score
# Sometimes moderators have important influences on artifact distributions as well as
# distributions of effect sizes. When this occurs, moderated artifact distributions
# can be created to make more appropriate corrections.
ad_tibble_mod <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(moderated_ads = TRUE),
moderators = moderator,
data = data_r_meas_multi)
# Passing these moderated distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_mod <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_mod, ad_obj_y = ad_tibble_mod)
summary(ma_obj_mod)
ma_obj_mod$meta_tables[[1]]$artifact_distribution$true_score
# It is also possible to create pairwise moderated artifact distributions.
ad_tibble_pairmod <- create_ad_list(n = n, rxx = rxxi, ryy = ryyi,
construct_x = x_name, construct_y = y_name,
sample_id = sample_id,
control = control_psychmeta(moderated_ads = TRUE,
pairwise_ads = TRUE),
moderators = moderator,
data = data_r_meas_multi)
# Passing these pairwise moderated distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pairmod <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_tibble_pairmod, ad_obj_y = ad_tibble_pairmod)
summary(ma_obj_pairmod)
ma_obj_pairmod$meta_tables[[1]]$artifact_distribution$true_score
# For even more control over which artifact distributions are used in corrections, you can supply
# un-named list of distributions in which the order of distributions corresponds to the order of
# meta-analyses in ma_obj. It is important for the elements to be un-named, as the absence of names
# and the length of the list are the two ways in which ma_r_ad() validates the lists.
ad_list_pairmod_x <- ad_tibble_pairmod$ad_x
ad_list_pairmod_y <- ad_tibble_pairmod$ad_y
# Passing these lists of distributions to ma_r_ad() corrects 'ma_obj' for artifacts:
ma_obj_pairmodlist <- ma_r_ad(ma_obj = ma_obj,
ad_obj_x = ad_list_pairmod_x, ad_obj_y = ad_list_pairmod_y)
summary(ma_obj_pairmodlist)
ma_obj_pairmodlist$meta_tables[[1]]$artifact_distribution$true_score
## End(Not run)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.