In CenterForAssessment/Literasee: Assessment Literacy Content Development via R

Growth and Prior Achievement at the Student Level

To investigate the possibility that individual level misfit might impact summary level results, student level SGP results were examined relative to prior achievement. With perfect fit to data, the correlation between students' most recent prior achievement scores and their student growth percentiles is zero (i.e., the goodness of fit tables would have a uniform distribution of percentiles across all previous scale score levels). To investigate in another way, correlations between a) prior and current scale scores (achievement) and b) prior score and student growth percentiles were calculated. Evidence of good model fit begins with a strong positive relationship between prior and current achievement, which suggests that growth is detectable and modeling it is reasonable to begin with. A lack of relationship (zero correlation) between prior achievement and growth confirms that the model has fit the data well and produced a uniform distribution of percentiles across all previous scale score levels.

Grade level subjects

Student-level correlations for grade level subjects are presented in Table r tblNum(1), and the results are generally as expected. Strong relationships exist between prior and current scale scores for the grade level analyses (column 3). With cohort-referenced (uncorrected) percentiles, the correlation between students' most recent prior achievement scores and their student growth percentiles is zero when the model is perfectly fit to the data. This also indicates that students can demonstrate high (or low) growth regardless of prior achievement. Correlations for XXX-SGP_STATE-XXX's SGPs are all essentially zero (column 4).

    student.cor.grd <- sgp_object@Data[VALID_CASE=='VALID_CASE' & YEAR==sgp_year & CONTENT_AREA %in% GL_subjects][, list(
        `$\\\rr_ { Test Scores}$` = round(cor(SCALE_SCORE, SCALE_SCORE_PRIOR_STANDARDIZED, use='pairwise.complete'), 2),
        `$\\\rr_ { SGP}$` = round(cor(SGP, SCALE_SCORE_PRIOR_STANDARDIZED, use='pairwise.complete'), 2),
        N_Size = sum(!is.na(SGP))), keyby = list(CONTENT_AREA, GRADE)]

    gl_tmp_tbl <- student.cor.grd[!is.na(student.cor.grd[["$\\\rr_ { SGP}$"]])]
    gl_tmp_tbl[, GRADE := as.numeric(GRADE)]
    setkey(gl_tmp_tbl, CONTENT_AREA, GRADE)
    gl_tmp_tbl <- gl_tmp_tbl[][order(match(gl_tmp_tbl$CONTENT_AREA, GL_subjects))]

    tmp.cap <- "Student Level Correlations between Prior Standardized Scale Score and 1) Current Scale Score and 2) SGP."
    gl_tmp_tbl$CONTENT_AREA <- sapply(gl_tmp_tbl$CONTENT_AREA, capwords, USE.NAMES=FALSE)
    gl_tmp_tbl$CONTENT_AREA[duplicated(gl_tmp_tbl$CONTENT_AREA)] <- ""
    gl_tmp_tbl$N_Size <- prettyNum(gl_tmp_tbl$N_Size, preserve.width = "individual", big.mark=',')
    setnames(gl_tmp_tbl, c(1:2,5), sapply(names(gl_tmp_tbl)[c(1:2,5)], capwords))

  cat(dualTable(as.matrix(gl_tmp_tbl), align=paste(rep('r', dim(gl_tmp_tbl)[2]), collapse=''), caption = tmp.cap, use.xtable = TRUE))

EOCT subjects

Each end of course test subject is analyzed using more than one sequence of prior subjects, grades and years, and these unique progressions are disaggregated in Table r tblNum(1) using the most recent prior available for each norm group (although more prior years' scores are used in SGP calculations when available). These correlations between current and prior scale score are typically lower than in the grade level norm groups, which may be expected due to the change in specific subject from one course to the next in EOCT course progressions. Specific types of norm groups can also produce markedly lower correlations, including skipped year groups and course repeaters.

The relationships between growth and prior achievement reported in Table r tblNum(1) are still non-existent for cohort referenced SGPs. These results are as expected for appropriate fit to the respective models as discussed in the grade level subject section above.

    student.cor.eoct <- sgp_object@Data[VALID_CASE=='VALID_CASE' & YEAR==sgp_year & CONTENT_AREA %in% EOCT_subjects][, list(
        `$\\\rr_ {  Test Scores}$` = round(cor(SCALE_SCORE, SCALE_SCORE_PRIOR_STANDARDIZED, use='pairwise.complete'), 2),
        `$\\\rr_ {  SGP}$` = round(cor(SGP, SCALE_SCORE_PRIOR_STANDARDIZED, use='pairwise.complete'), 2),
        N_Size = sum(!is.na(SGP))), keyby = list(CONTENT_AREA, Most_Recent_Prior)]

    eoct_tmp_tbl <- student.cor.eoct[!is.na(student.cor.eoct[["$\\\rr_ {  SGP}$"]])] # & CONTENT_AREA %in% c("GRADE_9_LIT", "AMERICAN_LIT") # Don't need to break up into two tables in 2017

    eoct_tmp_tbl <- eoct_tmp_tbl[][order(match(eoct_tmp_tbl$CONTENT_AREA, EOCT_subjects))]
    eoct_tmp_tbl$CONTENT_AREA <- sapply(eoct_tmp_tbl$CONTENT_AREA, capwords, USE.NAMES=FALSE)

    tmp.cap <- "EOCT Student Level Correlations between Prior Standardized Scale Score and 1) Current Scale Score and 2) SGP - Disaggregated by Norm Group."

    eoct_tmp_tbl[, Most_Recent_Prior := sapply(gsub("/", " ", Most_Recent_Prior), capwords)]
    eoct_tmp_tbl[, Most_Recent_Prior := gsub(" Eoct", "", Most_Recent_Prior)]
    eoct_tmp_tbl[, Most_Recent_Prior := gsub(" 8", " Grade 8", Most_Recent_Prior)]
    eoct_tmp_tbl[, Most_Recent_Prior := gsub(" 7", " Grade 7", Most_Recent_Prior)]
    eoct_tmp_tbl[, Most_Recent_Prior := gsub(" 6", " Grade 6", Most_Recent_Prior)]
    eoct_tmp_tbl[, Most_Recent_Prior := gsub("Mathematics", "Math", Most_Recent_Prior)]

    eoct_tmp_tbl$CONTENT_AREA <- sapply(eoct_tmp_tbl$CONTENT_AREA, capwords, USE.NAMES=FALSE)
    eoct_tmp_tbl$CONTENT_AREA[duplicated(eoct_tmp_tbl$CONTENT_AREA)] <- ""
    eoct_tmp_tbl$N_Size <- prettyNum(eoct_tmp_tbl$N_Size, preserve.width = "individual", big.mark=',')
    setnames(eoct_tmp_tbl, c(1:2,5), sapply(names(eoct_tmp_tbl)[c(1:2,5)], capwords))

    cat(dualTable(as.matrix(eoct_tmp_tbl), align=paste(rep('r', dim(eoct_tmp_tbl)[2]), collapse=''), caption = tmp.cap, use.xtable = TRUE))