gap: Gap Analysis

In EdSurvey: Analysis of NCES Education Survey and Assessment Data

Gap Analysis

Description

Compares the average levels of a variable between two groups that potentially share members.

Usage

gap(
  variable,
  data,
  groupA = "default",
  groupB = "default",
  percentiles = NULL,
  achievementLevel = NULL,
  achievementDiscrete = FALSE,
  stDev = FALSE,
  targetLevel = NULL,
  weightVar = NULL,
  jrrIMax = 1,
  varMethod = c("jackknife"),
  dropOmittedLevels = TRUE,
  defaultConditions = TRUE,
  recode = NULL,
  referenceDataIndex = 1,
  returnVarEstInputs = FALSE,
  returnSimpleDoF = FALSE,
  returnSimpleN = FALSE,
  returnNumberOfPSU = FALSE,
  noCov = FALSE,
  pctMethod = c("unbiased", "symmetric", "simple"),
  includeLinkingError = FALSE,
  omittedLevels = deprecated()
)

Arguments

variable	a character indicating the variable to be compared, potentially with a subject scale or subscale
data	an edsurvey.data.frame, a light.edsurvey.data.frame, or an edsurvey.data.frame.list
groupA	an expression or character expression that defines a condition for the subset. This subset will be compared to groupB. If not specified, it will define a whole sample as in data.
groupB	an expression or character expression that defines a condition for the subset. This subset will be compared to groupA. If not specified, it will define a whole sample as in data. If set to NULL, estimates for the second group will be dropped.
percentiles	a numeric vector. The gap function calculates the mean when this argument is omitted or set to NULL. Otherwise, the gap at the percentile given is calculated.
achievementLevel	the achievement level(s) at which percentages should be calculated
achievementDiscrete	a logical indicating if the achievement level specified in the achievementLevel argument should be interpreted as discrete so that just the percentage in that particular achievement level will be included. Defaults to FALSE so that the percentage at or above that achievement level will be included in the percentage.
stDev	a logical, set to TRUE to calculate the gap in standard deviations.
targetLevel	a character string. When specified, calculates the gap in the percentage of students at targetLevel in the variable argument. This is useful for comparing the gap in the percentage of students at a survey response level.
weightVar	a character indicating the weight variable to use. See Details.
jrrIMax	a numeric value; when using the jackknife variance estimation method, the default estimation option, jrrIMax=1, uses the sampling variance from the first plausible value as the component for sampling variance estimation. The Vjrr term, or sampeling variance term, can be estimated with any number of plausible values, and values larger than the number of plausible values on the survey (including Inf) will result in all plausible values being used. Higher values of jrrIMax lead to longer computing times and more accurate variance estimates.
varMethod	deprecated parameter, gap always uses the jackknife variance estimation
dropOmittedLevels	a logical value. When set to the default value of TRUE, drops those levels of all factor variables. Use print on an edsurvey.data.frame to see the omitted levels.
defaultConditions	a logical value. When set to the default value of TRUE, uses the default conditions stored in edsurvey.data.frame to subset the data. Use print on an edsurvey.data.frame to see the default conditions.
recode	a list of lists to recode variables. Defaults to NULL. Can be set as recode = list(var1 = list(from = c("a", "b", "c"), to = "d")).
referenceDataIndex	a numeric used only when the data argument is an edsurvey.data.frame.list, indicating which dataset is the reference dataset that other datasets are compared with. Defaults to 1.
returnVarEstInputs	a logical value; set to TRUE to return the inputs to the jackknife and imputation variance estimates which allows for the computation of covariances between estimates.
returnSimpleDoF	a logical value set to TRUE to return the degrees of freedom for some statistics (see Value section) that do not have a t-test; useful primarily for further computation
returnSimpleN	a logical value set to TRUE to add the count (n-size) of observations included in groups A and B in the percentage object
returnNumberOfPSU	a logical value set to TRUE to return the number of PSUs used in the calculation
noCov	set the covariances to zero in result
pctMethod	a character that is one of unbiased or simple. See the help for percentile for more information.
includeLinkingError	a logical value set to TRUE to include the linking error in variance estimation. Standard errors (e.g., diffAAse, diffBBse, and diffABABse) and p-values (e.g., diffAApValue, diffBBpValue, and diffABABpValue) would be adjusted for comparisons between digitally based assessments (DBA) and paper-based assessments (PBA) data. This option is supported only for NAEP data.
omittedLevels	this argument is deprecated. Use dropOmittedLevels.

Details

This function calculates the gap between groupA and groupB (which may be omitted to indicate the full sample). The gap is calculated for one of four statistics:

the gap in means

The mean score gap (in the score variable) identified in the variable argument. This is the default. The means and their standard errors are calculated using the methods described in the lm.sdf function documentation.

the gap in percentiles

The gap between respondents at the percentiles specified in the percentiles argument. This is returned when the percentiles argument is defined. The mean and standard error are computed as described in the percentile function documentation.

the gap in achievement levels

The gap in the percentage of students at (when achievementDiscrete is TRUE) or at or above (when achievementDiscrete is FALSE) a particular achievement level. This is used when the achievementLevel argument is defined. The mean and standard error are calculated as described in the achievementLevels function documentation.

the gap in a survey response

The gap in the percentage of respondents responding at targetLevel to variable. This is used when targetLevel is defined. The mean and standard deviation are calculated as described in the edsurveyTable function documentation.

Value

The return type depends on if the class of the data argument is an edsurvey.data.frame or an edsurvey.data.frame.list. Both include the call (called call), a list called labels, an object named percentage that shows the percentage in groupA and groupB, and an object that shows the gap called results.

The labels include the following elements:

definition	the definitions of the groups
nFullData	the n-size for the full dataset (before applying the definition)
nUsed	the n-size for the data after the group is subsetted and other restrictions (such as omitted values) are applied
nPSU	the number of PSUs used in calculation–only returned when returnNumberOfPSU = TRUE

The percentages are computed according to the vignette titled Statistical Methods Used in EdSurvey in the section “Estimation of Weighted Percentages When Plausible Values Are Not Present.” The standard errors are calculated according to “Estimation of the Standard Error of Weighted Percentages When Plausible Values Are Not Present, Using the Jackknife Method.” Standard errors of differences are calculated as the square root of the typical variance formula

Var(A-B) = Var(A) + Var(B) - 2 Cov(A,B)

where the covariance term is calculated as described in the vignette titled Statistical Methods Used in EdSurvey in the section “Estimation of Covariances.” These degrees of freedom are available only with the jackknife variance estimation. The degrees of freedom used for hypothesis testing are always set to the number of jackknife replicates in the data.

the data argument is an edsurvey.data.frame When the data argument is an edsurvey.data.frame, gap returns an S3 object of class gap.

The percentage object is a numeric vector with the following elements:

pctA	the percentage of respondents in groupA compared with the whole sample in data
pctAse	the standard error on the percentage of respondents in groupA
dofA	degrees of freedom appropriate for a t-test involving pctA. This value is returned only if returnSimpleDoF=TRUE.
pctB	the percentage of respondents in groupB.
pctBse	the standard error on the percentage of respondents in groupB
dofB	degrees of freedom appropriate for a t-test involving pctA. This value is returned only if returnSimpleDoF=TRUE.
diffAB	the value of pctA minus pctB
covAB	the covariance of pctA and pctB; used in calculating diffABse.
diffABse	the standard error of pctA minus pctB
diffABpValue	the p-value associated with the t-test used for the hypothesis test that diffAB is zero
dofAB	degrees of freedom used in calculating diffABpValue

The results object is a numeric data frame with the following elements:

estimateA	the mean estimate of groupA (or the percentage estimate if achievementLevel or targetLevel is specified)
estimateAse	the standard error of estimateA
dofA	degrees of freedom appropriate for a t-test involving meanA. This value is returned only if returnSimpleDoF=TRUE.
estimateB	the mean estimate of groupB (or the percentage estimate if achievementLevel or targetLevel is specified)
estimateBse	the standard error of estimateB
dofB	degrees of freedom appropriate for a t-test involving meanB. This value is returned only if returnSimpleDoF=TRUE.
diffAB	the value of estimateA minus estimateB
covAB	the covariance of estimateA and estimateB. Used in calculating diffABse.
diffABse	the standard error of diffAB
diffABpValue	the p-value associated with the t-test used for the hypothesis test that diffAB is zero.
dofAB	degrees of freedom used for the t-test on diffAB

If the gap was in achievement levels or percentiles and more than one percentile or achievement level is requested, then an additional column labeled percentiles or achievementLevel is included in the results object.

When results has a single row and when returnVarEstInputs is TRUE, the additional elements varEstInputs and pctVarEstInputs also are returned. These can be used for calculating covariances with varEstToCov.

the data argument is an edsurvey.data.frame.list When the data argument is an edsurvey.data.frame.list, gap returns an S3 object of class gapList.

The results object in the edsurveyResultList is a data.frame. Each row regards a particular dataset from the edsurvey.data.frame, and a reference dataset is dictated by the referenceDataIndex argument.

The percentage object is a data.frame with the following elements:

covs	a data frame with a column for each column in the covs. See previous section for more details.
...	all elements in the percentage object in the previous section
diffAA	the difference in pctA between the reference data and this dataset. Set to NA for the reference dataset.
covAA	the covariance of pctA in the reference data and pctA on this row. Used in calculating diffAAse.
diffAAse	the standard error for diffAA
diffAApValue	the p-value associated with the t-test used for the hypothesis test that diffAA is zero
diffBB	the difference in pctB between the reference data and this dataset. Set to NA for the reference dataset.
covBB	the covariance of pctB in the reference data and pctB on this row. Used in calculating diffAAse.
diffBBse	the standard error for diffBB
diffBBpValue	the p-value associated with the t-test used for the hypothesis test that diffBB is zero
diffABAB	the value of diffAB in the reference dataset minus the value of diffAB in this dataset. Set to NA for the reference dataset.
covABAB	the covariance of diffAB in the reference data and diffAB on this row. Used in calculating diffABABse.
diffABABse	the standard error for diffABAB
diffABABpValue	the p-value associated with the t-test used for the hypothesis test that diffABAB is zero

The results object is a data.frame with the following elements:

...	all elements in the results object in the previous section
diffAA	the value of groupA in the reference dataset minus the value in this dataset. Set to NA for the reference dataset.
covAA	the covariance of meanA in the reference data and meanA on this row. Used in calculating diffAAse.
diffAAse	the standard error for diffAA
diffAApValue	the p-value associated with the t-test used for the hypothesis test that diffAA is zero
diffBB	the value of groupB in the reference dataset minus the value in this dataset. Set to NA for the reference dataset.
covBB	the covariance of meanB in the reference data and meanB on this row. Used in calculating diffBBse.
diffBBse	the standard error for diffBB
diffBBpValue	the p-value associated with the t-test used for the hypothesis test that diffBB is zero
diffABAB	the value of diffAB in the reference dataset minus the value of diffAB in this dataset. Set to NA for the reference dataset.
covABAB	the covariance of diffAB in the reference data and diffAB on this row. Used in calculating diffABABse.
diffABABse	the standard error for diffABAB
diffABABpValue	the p-value associated with the t-test used for the hypothesis test that diffABAB is zero
sameSurvey	a logical value indicating if this line uses the same survey as the reference line. Set to NA for the reference line.

Author(s)

Paul Bailey, Trang Nguyen, and Huade Huo

Examples

## Not run: 
# read in the example data (generated, not real student data)
sdf <- readNAEP(path=system.file("extdata/data", "M36NT2PM.dat", package = "NAEPprimer"))

# find the mean score gap in the primer data between males and females
gap(variable="composite", data=sdf, groupA=dsex=="Male", groupB=dsex=="Female")

# find the score gap of the quartiles in the primer data between males and females
gap(variable="composite", data=sdf,
    groupA=dsex=="Male", groupB=dsex=="Female", percentile=50)
gap(variable="composite", data=sdf,
    groupA=dsex=="Male", groupB=dsex=="Female", percentile=c(25, 50, 75))

# find the percent proficient (or higher) gap in the primer data between males and females
gap(variable="composite", data=sdf, groupA=dsex=="Male", groupB=dsex=="Female", 
    achievementLevel=c("Basic", "Proficient", "Advanced"))

# find the discrete achievement level gap--this is harder to interpret
gap(variable="composite", data=sdf, groupA=dsex=="Male", groupB=dsex=="Female",
    achievementLevel="Proficient", achievementDiscrete=TRUE)

# find the percent talk about studies at home (b017451) never or hardly
# ever gap in the primer data between males and females
gap(variable="b017451", data=sdf, groupA=dsex=="Male", groupB=dsex=="Female", 
    targetLevel="Never or hardly ever")

# example showing how to compare multiple levels
gap(variable="b017451",
    data=sdf,
    groupA=dsex=="Male",
    groupB=dsex=="Female",
    targetLevel="Infrequently",
    recode=list(b017451=list(from=c("Never or hardly ever",
                                    "Once every few weeks",
                                    "About once a week"),
                             to=c("Infrequently"))))

# make subsets of sdf by scrpsu, "Scrambled PSU and school code"
sdfA <- subset(sdf, scrpsu %in% c(5,45,56))
sdfB <- subset(sdf, scrpsu %in% c(75,76,78))
sdfC <- subset(sdf, scrpsu %in% 100:200)
sdfD <- subset(sdf, scrpsu %in% 201:300)

sdfl <- edsurvey.data.frame.list(datalist=list(sdfA, sdfB, sdfC, sdfD),
                                 labels=c("A locations", "B locations",
                                          "C locations", "D locations"))

gap(variable="composite", data=sdfl, groupA=dsex=="Male", groupB=dsex=="Female", percentile=c(50))

## End(Not run)

## Not run: 
# example showing using linking error with gap
# load Grade 4 math data
# requires NAEP RUD license with these files in the folder the user is currectly in
g4math2015 <- readNAEP("M46NT1AT.dat")
g4math2017 <- readNAEP("M48NT1AT.dat")
g4math2019 <- readNAEP("M50NT1AT.dat")

# make an edsurvey.data.frame.list from math grade 4 2015, 2017, and 2019 data
g4math <- edsurvey.data.frame.list(datalist=list(g4math2019, g4math2017, g4math2015),
                                   labels = c("2019", "2017", "2015"))

# gap analysis with linking error in variance estimation across surveys
gap(variable="composite", data=g4math,
    groupA=dsex=="Male", groupB=dsex=="Female", includeLinkingError=TRUE)
gap(variable="composite", data=g4math,
    groupA=dsex=="Male", groupB=dsex=="Female", percentiles = c(10, 25), 
    includeLinkingError=TRUE)
gap(variable="composite", data=g4math, groupA=dsex=="Male", groupB=dsex=="Female", 
    achievementDiscrete = TRUE, achievementLevel=c("Basic", "Proficient", "Advanced"), 
    includeLinkingError=TRUE)

## End(Not run)

EdSurvey documentation built on June 27, 2024, 5:10 p.m.