reproCheck: reproCheck function

In METRICS-CARPS/CARPSreports: Build reports for reproducibility assignments

Description

This is a helper function to aid comparison of values reported in articles to values obtained in reproducibility checks. (1) calculates the percentage error (PE) between a reported value and an obtained value. (2) identifies the error type (decision error, major numerical, minor numerical, no error) (3) user must specificy the value type from the defaults list (or use 'other' if not listed) Errors types are defined as follows: >> 'minor numerical': i.e., >= 2 >> 'major numerical' (i.e., PE >= 10 If p values are being compared, also returns an additional error type: >> 'decision error' (i.e., reported p and obtained p fall on different sides of the .05 threshold) The default value types are as follows: "p" (p-value), "mean" (mean), "sd" (standard deviation), "se" (standard error), "df" (degrees of freedom), "F" (F test statistic), "t" (t test statistic), "bf" (bayes factor), "ci" (confidence interval), "median" (median), "d" (Cohen's d), "irr" (inter-rater reliability), "r" (Pearson correlation coefficient), "z" (Wilcoxon z), "coeff" (regression coefficients), "n" (count or proportion), "x2" (chi squared), "phi" (phi), "pes" (partial eta squared), "other") Note that sometimes exact values are not reported in the article (e.g., p < .05 ot t <1). In these cases you should visually inspect (eyeball) whether the obtained value falls on the correct side of the specified boundary (e.g., p <.05). If eyeballing is necessary, change the "eyeballCheck" parameter to TRUE if the values appear to match or FALSE if the values do not apepar to match. The eyeballCheck parameter defaults to NA because usually no eyeball check is required.

Usage

reproCheck(reportedValue, obtainedValue, valueType = c("p", "mean", "sd",
  "se", "df", "F", "t", "bf", "ci", "median", "d", "irr", "r", "z",
  "coeff", "n", "x2", "phi", "pes", "other"), eyeballCheck = NA,
  round = TRUE, updatedReportObject = reportObject)

Arguments

reportedValue	Enter the value reported in the article
obtainedValue	Enter the corresponding value obtained in your reproducibility check
valueType	The type of value being compared
eyeballCheck	whether a manual 'eyeball' comparison was required, and the outcome of the comparison TRUE/FALSE. Defaults to NA.
round	whether the obtained value should be rounded to same number of decimal places as the reported value TRUE/FALSE. Defaults to TRUE.

Value

Returns a short text report noting the error type and the PE. Output can also be assigned to a reportObject to keep a running tally of comparisons

Examples

reproCheck(reportedValue = '3.45', obtainedValue = condition_mean, valueType = 'mean', eyeballCheck = NA)
reproCheck(reportedValue = '.054', obtainedValue = a_p_value, valueType = 'p', eyeballCheck = NA)
reproCheck(reportedValue = '15.63', obtainedValue = this_sd, valueType = 'sd', eyeballCheck = NA)
reproCheck(reportedValue = '<.05', obtainedValue = a_significant_p_value, valueType = 'p', eyeballCheck = TRUE)
reproCheck(reportedValue = '<.05', obtainedValue = a_nonsignificant_p_value, valueType = 'p', eyeballCheck = FALSE)

METRICS-CARPS/CARPSreports documentation built on June 8, 2020, 4:59 p.m.