Nothing
writeAlizer: Scoring Model Development
In writeAlizer: Generate Predicted Writing Quality Scores
This reference describes how the scoring models were developed. For installation, importing files, and generating scores, start with the getting-started guide.
How to use this reference
Use the contents list to jump to your model. Each model section describes its training data, the algorithms combined in its predictions, and the features those algorithms used. An ensemble combines predictions from several algorithms. Weight tables describe that combination; feature-importance tables describe relative contributions, not causal effects.
These are historical model-development summaries. writeAlizer loads saved models rather than retraining them. During scoring, Models 2 and 3 standardize predictors within the submitted group of texts, so changing that group can change scores. Training-time feature selection described below is distinct from this scoring step.
The detailed tables are wide. On a small screen, scroll a table horizontally to see all algorithms.
Recommended Models for Use
ReaderBench-Model-3 and Coh-Metrix-Model-3 are the recommended models for generating predicted writing quality scores, and aWE-CBM-Model-1 is the best available model for generating automated written expression curriculum-based measurement scores.
Scoring Model Development {#scoring-model-development}
The general process used to generate all scoring models is presented below.
Predictive Algorithms and R Packages Used
The caret and caretEnsemble packages were used as wrappers for the following predictive algorithms:
-
Random forest regression (package randomForest)
-
Cubist regression (package Cubist)
-
Support vector machines with a radial kernel (package kernlab)
-
Bagged multivariate adaptive regression splines (package earth)
-
Stochastic gradient boosted trees (package gbm)
-
Partial least squares regression (package pls)
-
Elasticnet regression (package elasticnet)
These algorithms are described in detail in the following references (among others):
-
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer. https://doi.org/10.1007/b94608
-
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer. https://doi.org/10.1007/978-1-4614-6849-3
Steps
The following flowchart provides an overview of the scoring model development workflow, with more details on some steps provided below.
1. Import Data
Depending on the specific scoring model, ReaderBench, Coh-Metrix, and/or GAMET output files were imported into R using functions similar to the import_XXXX.R functions in writeAlizer (see https://shmercer.github.io/writeAlizer/reference/index.html)
2. Pre-Process Data
Automated data pre-processing was done using the preProcess() function in caret:
-
Predictors from the output file with near zero variance (defined based on defaults in the nearZeroVar() function) were removed, and the remaining predictors were standardized.
-
Highly correlated | r > .90 | predictors were identified, with the predictor that had the highest mean correlation with all of the other predictors removed.
-
The reduced set of predictors was submitted to the next step of the analysis.
3. Determine Optimal Tuning Parameters
The following tuning hyperparameters were optimized based on resampling (repeated 10 fold) in caret. Each algorithm was tuned separately. Full descriptions of the tuning parameters are available in each package's documentation.
-
Random forest regression: mtry
-
Cubist regression: committees, neighbors
-
Support vector machines: sigma, C
-
Bagged multivariate adaptive regression splines: nprune, degree
-
Stochastic gradient boosted trees: n.trees, interaction.depth, shrinkage, n.minobsinnode
-
Partial least squares regression: ncomp
-
Elastic net regression: fraction, lambda
4. Final/Optimal Model for each Algorithm
A model for each algorithm was fit with the hyperparameters set to the optimal values found in Step 3, with bootstrapped (1000 samples) resampling-based cross-validation so that an ensemble model (weighting each algorithm) could be built based on the resamples. This step was done with the caretList() function of the caretEnsemble package. This process is illustrated in more detail in the caretEnsemble vignette: https://zachmayer.github.io/caretEnsemble/articles/caretEnsemble-intro.html
5. Estimate an Ensemble Model to Combine the Algorithms
The caretEnsemble() function was used to determine the optimal linear weighting of the algorithms that minimized RMSE (i.e., discrepancy between actual writing quality scores and predicted quality scores) in the resamples from Step 4. The tables below report the fitted algorithm weights, including small or negative coefficients where present.
The varImp() function of caretEnsemble was used to generate estimates of relative predictor importance for the overall ensemble model and for each individual algorithm.
6. Generate Predicted Quality Scores from each Ensemble
The predict() function of caretEnsemble was used to generate/store predicted quality scores for the ensemble models.
7. Average Scores to get Final Predicted Quality Scores
The predicted scores from each ensemble were averaged to produce the final predictions.
ReaderBench Model 1 {#readerbench-model-1}
General Description
Model 1 has been replaced by the greatly simplified Model 2 that better handles multi-paragraph compositions. Model 2 is recommended over Model 1.
Model 1 is an ensemble (formed by averaging predicted quality scores) of the six sub-models described below.
All of these sub-models used ReaderBench scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
This scoring model was evaluated in the following publications: [@Matta2022; @Mercer2022; @Keller-Margulis2021]
ReaderBench Model 1a
This model was trained on fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:--------|:-------|:--------|:--------|:-------|:-------|:-------|
| -3.9077 | -0.1323 | 0.4789 | -0.0963 | -0.0361 | 0.3985 | 0.1297 | 0.3442 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------------------------|:------|:------|:-----|:-----|:------|:-----|:------|:------|
| WdEnt | 14.61 | 40.66 | 2.49 | 1.93 | 13.17 | 3.97 | 62.12 | 19.56 |
| LxcDiv | 3.02 | 4.3 | 2.2 | 1.52 | 2.69 | 2.77 | 0 | 5.55 |
| AvgUnqPrepositionBl | 2.88 | 3.21 | 2.18 | 1.42 | 6.67 | 1.98 | 0 | 5.84 |
| AvgNmdEntBl | 2.71 | 0.47 | 1.13 | 0.4 | 0.79 | 0.89 | 18.1 | 2.92 |
| AvgUnqVerbBl | 2.4 | 2.36 | 2.21 | 1.48 | 6.53 | 2.33 | 0 | 3.5 |
| WdDiffLemmaStem | 2.07 | 1.07 | 0.82 | 0.91 | 0 | 1.35 | 13 | 1.46 |
| RdbltyFlesch | 1.91 | 0.7 | 0.41 | 0.41 | 4.1 | 1.04 | 6.51 | 3.94 |
| AvgChainSpan | 1.84 | 3.5 | 1.77 | 1.16 | 0 | 2.11 | 0 | 2.04 |
| AvgDepsBl_det | 1.64 | 2.01 | 1.63 | 0.79 | 3.96 | 1.6 | 0 | 2.19 |
| AvgBlScore | 1.55 | 0.87 | 2.01 | 1.32 | 0 | 1.76 | 0 | 1.75 |
| AvgPronBl_first_person | 1.52 | 0.7 | 1.56 | 0.75 | 2.38 | 1.39 | 0 | 2.63 |
| AvgDepsBl_nsubj | 1.5 | 1.34 | 2.18 | 1.45 | 0 | 1.9 | 0 | 0.88 |
| AvgUnqAdverbBl | 1.48 | 1.86 | 1.88 | 1.06 | 1.3 | 2.12 | 0 | 0.73 |
| AvgDepsBl_punct | 1.45 | 1.45 | 1.77 | 0.93 | 1.9 | 2.1 | 0 | 0.88 |
| WdDiffWdStem | 1.43 | 2.02 | 1.47 | 0.84 | 1.36 | 1.04 | 0 | 2.34 |
| AvgDepsSen_punct | 1.41 | 0.64 | 1.33 | 0.65 | 4.76 | 1.04 | 0 | 2.63 |
| AvgDepsSen_dep | 1.34 | 0.04 | 0.65 | 0.33 | 5.78 | 0.53 | 0 | 4.09 |
| AvgSenScore | 1.28 | 0.39 | 0.27 | 0.22 | 0 | 0.53 | 0 | 4.82 |
| AvgPronounBl | 1.23 | 0.2 | 1.8 | 1.08 | 0 | 1.38 | 0 | 1.31 |
| TCorefChainDoc | 1.2 | 0.41 | 1.59 | 0.77 | 0 | 0.88 | 0 | 2.04 |
| AvgDepsBl_nmod | 1.12 | 0.52 | 1.9 | 1.09 | 0 | 1.57 | 0 | 0.29 |
| AvgAOASen_Shock | 1.09 | 1.04 | 0.77 | 0.52 | 3.75 | 1.04 | 0 | 1.9 |
| AvgSenBl | 1.03 | 0.42 | 1.51 | 0.68 | 0 | 1.29 | 0 | 0.88 |
| WdLettStdDev | 1.01 | 0.85 | 1.27 | 0.8 | 0 | 0.84 | 0 | 1.46 |
| AvgWdLen | 0.97 | 1.33 | 1.4 | 0.75 | 0 | 1.26 | 0 | 0.44 |
| AvgDepsBl_nummod | 0.95 | 0.75 | 1.16 | 0.42 | 0.04 | 1.23 | 0 | 1.02 |
| AvgCorefChain | 0.94 | 0.56 | 1.01 | 0.34 | 2.87 | 0.32 | 0 | 2.04 |
| TActCorefChainWd | 0.93 | 0.78 | 0.83 | 0.55 | 0.99 | 1.15 | 0 | 1.31 |
| RdbltyDaleChall | 0.92 | 0.93 | 0.95 | 0.38 | 0 | 1.07 | 0.27 | 1.17 |
| AvgDepsBl_advmod | 0.89 | 0.2 | 1.65 | 0.84 | 1 | 1.26 | 0 | 0 |
| AvgAOABl_Bristol | 0.86 | 0.73 | 0.93 | 0.41 | 0 | 0.52 | 0 | 1.75 |
| AvgUnqNoundBl | 0.85 | 0.28 | 1.64 | 0.82 | 0.65 | 0.88 | 0 | 0.29 |
| AvgDepsBl_dobj | 0.85 | 0.19 | 1.49 | 0.68 | 1.5 | 0.93 | 0 | 0.44 |
| AvgAOABl_Shock | 0.83 | 2.04 | 1.3 | 0.63 | 0.12 | 0.96 | 0 | 0 |
| SenStdDevWd | 0.77 | 0.67 | 1.07 | 0.56 | 0.83 | 1.42 | 0 | 0 |
| AvgDepsBl_mark | 0.76 | 0.15 | 1.31 | 0.53 | 1.23 | 0.7 | 0 | 0.58 |
| LexChainMaxSp | 0.75 | 0.31 | 1.57 | 0.75 | 0 | 0.85 | 0 | 0 |
| WdAvgDpthHypernymTree | 0.75 | 0.42 | 0.63 | 0.36 | 0.57 | 0.6 | 0 | 1.61 |
| AvgPronBl_indefinite | 0.72 | 0.12 | 1.44 | 0.64 | 0 | 0.62 | 0 | 0.44 |
| AvgSenAdjCoh_Path | 0.69 | 0.35 | 0.83 | 0.49 | 0.57 | 0.77 | 0 | 0.88 |
| AvgDepsBl_cop | 0.68 | 0.15 | 1.2 | 0.44 | 2.55 | 0.92 | 0 | 0 |
| CharEnt | 0.68 | 0.26 | 1.44 | 0.69 | 0.42 | 0.73 | 0 | 0 |
| AvgConnBl_simp_subords | 0.66 | 0.11 | 1.2 | 0.45 | 0.31 | 1.05 | 0 | 0 |
| LexChainAvgSpan | 0.66 | 0.49 | 1.17 | 0.7 | 0 | 0.94 | 0 | 0 |
| AvgConnBl_reas_purp | 0.66 | 0.11 | 1.26 | 0.5 | 0 | 1.02 | 0 | 0 |
| AvgDepsBl_advcl | 0.64 | 0.05 | 1.35 | 0.56 | 1.13 | 0.7 | 0 | 0 |
| TCorefChainBigSpan | 0.63 | 0.01 | 1.16 | 0.42 | 1.31 | 0.54 | 0 | 0.44 |
| AvgUnqAdjectiveBl | 0.63 | 0.48 | 1.37 | 0.58 | 0 | 0.61 | 0 | 0 |
| AvgDepsBl_amod | 0.63 | 0.05 | 1.28 | 0.51 | 0.32 | 0.85 | 0 | 0 |
| AvgDepsBl_aux | 0.62 | 0.57 | 0.98 | 0.3 | 0 | 1.07 | 0 | 0 |
| WdPathCntHypernymTree | 0.62 | 0.99 | 0.79 | 0.44 | 3.29 | 0.7 | 0 | 0.15 |
| WdSylCnt | 0.62 | 0.37 | 0.8 | 0.56 | 0 | 1.3 | 0 | 0 |
| AvgUnqPronounBl | 0.61 | 0.21 | 1.3 | 0.53 | 0.62 | 0.65 | 0 | 0 |
| FrqRhythmId | 0.59 | 0.73 | 0.9 | 0.44 | 0 | 0.94 | 0 | 0 |
| AvgDepsBl_nsubjpass | 0.57 | 0.05 | 1.01 | 0.32 | 1.18 | 0.88 | 0 | 0 |
| AvgDepsBl_ccomp | 0.55 | 0.26 | 1.13 | 0.39 | 1.76 | 0.54 | 0 | 0 |
| AvgConnSen_addition | 0.54 | 0.25 | 0.59 | 0.25 | 0 | 0.95 | 0 | 0.44 |
| AvgConnBl_order | 0.52 | 0.07 | 0.72 | 0.16 | 2.74 | 0.92 | 0 | 0 |
| AvgBlVoiceCoOcc | 0.52 | 0 | 1.09 | 0.36 | 0 | 0.73 | 0 | 0 |
| AvgInferenceDistChain | 0.51 | 0.32 | 0.74 | 0.45 | 0.3 | 0.79 | 0 | 0.15 |
| AvgAOESen_InvLinRegSlo | 0.5 | 0.56 | 0.65 | 0.35 | 0.86 | 0.28 | 0 | 0.73 |
| AvgRhythmUnitStreesSyll | 0.49 | 0.29 | 0.03 | 0.3 | 0 | 1.02 | 0 | 0.88 |
| AvgConnBl_semi_coords | 0.48 | 0.01 | 0.98 | 0.3 | 0 | 0.68 | 0 | 0 |
| LxcSoph | 0.48 | 0.25 | 0.83 | 0.32 | 0 | 0.8 | 0 | 0 |
| AvgConnBl_logical_cons | 0.47 | 0.05 | 0.58 | 0.1 | 1.58 | 0.63 | 0 | 0.44 |
| AvgDepsBl_neg | 0.47 | 0.04 | 0.81 | 0.21 | 0 | 0.86 | 0 | 0 |
| AvgCommaBl | 0.47 | 0.09 | 0.9 | 0.25 | 0 | 0.75 | 0 | 0 |
| AvgNounNmdEntBl | 0.47 | 0.08 | 0.67 | 0.14 | 0 | 0.43 | 0 | 0.73 |
| AvgNounSen | 0.46 | 0.23 | 0.12 | 0.07 | 0 | 0.62 | 0 | 1.17 |
| AvgAdverbSen | 0.45 | 0.17 | 0.46 | 0.55 | 0 | 0.81 | 0 | 0.29 |
| AvgConnBl_oppositions | 0.44 | 0.07 | 0.9 | 0.26 | 0 | 0.62 | 0 | 0 |
| AvgNmdEntSen | 0.44 | 0.62 | 0.13 | 0.5 | 0 | 0.93 | 0 | 0.44 |
| AvgConnBl_contrasts | 0.44 | 0.25 | 1.02 | 0.32 | 0 | 0.41 | 0 | 0 |
| AvgConnSen_semi_coords | 0.43 | 0.05 | 0.43 | 0.06 | 0.08 | 0.93 | 0 | 0.29 |
| AvgAOASen_Bristol | 0.43 | 0.48 | 0.32 | 0.48 | 1.36 | 0.7 | 0 | 0.29 |
| AvgDepsBl_xcomp | 0.43 | 0.11 | 1.15 | 0.41 | 0 | 0.23 | 0 | 0 |
| AvgConnSen_simp_subords | 0.43 | 0.74 | 0.28 | 0.31 | 1.49 | 0.97 | 0 | 0 |
| WdPolysemyCnt | 0.41 | 0.4 | 0 | 0.71 | 0.46 | 0.17 | 0 | 1.31 |
| AvgPronBl_third_person | 0.41 | 0.39 | 0.88 | 0.24 | 0 | 0.44 | 0 | 0 |
| AvgAOASen_Bird | 0.4 | 0.25 | 0.58 | 0.51 | 0 | 0.7 | 0 | 0 |
| AvgDepsBl_mwe | 0.4 | 0.03 | 0.49 | 0.08 | 3.31 | 0.71 | 0 | 0 |
| AvgPronounSen | 0.38 | 0.27 | 0.06 | 0.18 | 0 | 0.59 | 0 | 0.88 |
| AvgConnBl_addition | 0.37 | 0.23 | 0.58 | 0.1 | 0 | 0.71 | 0 | 0 |
| AvgAOABl_Kuperman | 0.37 | 0.6 | 0.32 | 0.48 | 0 | 0.43 | 0 | 0.44 |
| AvgAOASen_Kuperman | 0.36 | 0.16 | 0.62 | 0.55 | 0 | 0.52 | 0 | 0 |
| AvgAOABl_Cortese | 0.36 | 0.33 | 0.3 | 0.59 | 2.07 | 0.66 | 0 | 0 |
| AvgDepsSen_advcl | 0.35 | 0.22 | 0.5 | 0.51 | 0 | 0.6 | 0 | 0 |
| AvgDepsSen_det | 0.33 | 0.32 | 0.08 | 0.27 | 0 | 0.31 | 0 | 0.88 |
| AvgDepsBl_acl | 0.33 | 0.01 | 0.54 | 0.1 | 0 | 0.67 | 0 | 0 |
| AvgRhythmUnits | 0.33 | 0.54 | 0.6 | 0.43 | 0 | 0.22 | 0 | 0.15 |
| AggPronSen_indefinite | 0.32 | 0.14 | 0.24 | 0.49 | 0.23 | 0.55 | 0 | 0.29 |
| AvgConnBl_temp_cons | 0.31 | 0.16 | 0.87 | 0.24 | 0 | 0.1 | 0 | 0 |
| AvgDepsSen_ccomp | 0.3 | 0.19 | 0.04 | 0.34 | 0 | 0.79 | 0 | 0.29 |
| SenAsson | 0.29 | 0.02 | 0.43 | 0.06 | 1 | 0.57 | 0 | 0 |
| AggPronSen_third_person | 0.29 | 0.46 | 0.3 | 0.15 | 0 | 0.52 | 0 | 0.15 |
| AvgAOABl_Bird | 0.29 | 0.27 | 0.46 | 0.53 | 0 | 0.3 | 0 | 0.15 |
| AvgDepsSen_aux | 0.29 | 0.08 | 0.12 | 0.49 | 0 | 0.9 | 0 | 0 |
| AvgDepsSen_dobj | 0.28 | 0.1 | 0.11 | 0.06 | 0 | 0.95 | 0 | 0 |
| AvgConnSen_oppositions | 0.26 | 0.03 | 0.28 | 0.03 | 0 | 0.69 | 0 | 0 |
| AvgDepsSen_nmod | 0.26 | 0.18 | 0.37 | 0.37 | 0 | 0.47 | 0 | 0 |
| AvgDepsSen_amod | 0.25 | 0.26 | 0.17 | 0.1 | 0 | 0.47 | 0 | 0.29 |
| AvgAOASen_Cortese | 0.24 | 0.2 | 0.59 | 0.44 | 0 | 0.08 | 0 | 0 |
| AvgDepsSen_compound | 0.24 | 0.16 | 0.38 | 0.18 | 0 | 0.43 | 0 | 0 |
| AvgDepsSen_mark | 0.23 | 0.3 | 0.23 | 0.27 | 0 | 0.48 | 0 | 0 |
| AvgDepsSen_mwe | 0.22 | 0.1 | 0.2 | 0.02 | 0 | 0.6 | 0 | 0 |
| AvgAdjectiveSen | 0.22 | 0.16 | 0.04 | 0.08 | 0 | 0.78 | 0 | 0 |
| AvgAOEBl_IndPolyFAT.3 | 0.21 | 0.26 | 0.05 | 0.29 | 0.04 | 0.24 | 0 | 0.44 |
| AvgAOEBl_InvLinRegSlo | 0.21 | 0.44 | 0.17 | 0.25 | 0 | 0.43 | 0 | 0 |
| AvgDepsBl_dep | 0.21 | 0.09 | 0.17 | 0.01 | 0.69 | 0.57 | 0 | 0 |
| AvgDepsSen_xcomp | 0.2 | 0.26 | 0 | 0.39 | 0 | 0.4 | 0 | 0.29 |
| AvgDepsSen_cop | 0.19 | 0.26 | 0.13 | 0.36 | 0 | 0.31 | 0 | 0.15 |
| AvgDepsBl_compound | 0.18 | 0.07 | 0.17 | 0.01 | 0.45 | 0.23 | 0 | 0.29 |
| LangRhythmDiameter | 0.16 | 0.18 | 0.29 | 0.03 | 0.86 | 0.14 | 0 | 0 |
| AvgConnSen_temp_cons | 0.13 | 0.3 | 0.18 | 0.42 | 0 | 0.09 | 0 | 0 |
| AvgAOEBl_IndAbThr.0.3. | 0.12 | 0.31 | 0.01 | 0.33 | 0.04 | 0.27 | 0 | 0 |
| AvgDepsSen_acl | 0.12 | 0.04 | 0.13 | 0.01 | 0 | 0.3 | 0 | 0 |
| AvgConnSen_order | 0.12 | 0.21 | 0.17 | 0.01 | 0 | 0.23 | 0 | 0 |
| LangRhythmCoeff | 0.12 | 0.41 | 0 | 0.23 | 0 | 0.31 | 0 | 0 |
| AvgSenBlCoh_LeackChod | 0.12 | 0.06 | 0.08 | 0.42 | 0 | 0.28 | 0 | 0 |
| AvgUnqWdBl | 0.11 | 0 | 0 | 1.79 | 0 | 0 | 0 | 0 |
| AvgBlLen | 0.11 | 0 | 0 | 1.8 | 0 | 0 | 0 | 0 |
| AvgVerbBl | 0.1 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 |
| AvgDepsSen_neg | 0.1 | 0.05 | 0.02 | 0 | 0.08 | 0.37 | 0 | 0 |
| Words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 |
| Content.words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 |
| AvgWdBl | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 |
| AvgDepsBl_case | 0.08 | 0 | 0 | 1.27 | 0 | 0 | 0 | 0 |
| AvgNounBl | 0.07 | 0 | 0 | 1.15 | 0 | 0 | 0 | 0 |
| LangRhythmId | 0.07 | 0.02 | 0.23 | 0.02 | 0 | 0 | 0 | 0 |
| AvgPrepositionBl | 0.07 | 0 | 0 | 1.21 | 0 | 0 | 0 | 0 |
| AvgAdverbBl | 0.05 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LDA | 0.04 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Shock | 0.04 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_Path | 0.04 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_word2vec | 0.04 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 |
| SynSoph | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 |
| Sentences | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LSA | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_word2vec | 0.04 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_LSA | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 |
| AvgAOESen_InfPointPoly | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 |
| AvgUnqWdSen | 0.03 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 |
| AvgConnSen_reas_purp | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LeackChod | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgPrepositionSen | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgVerbSen | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| SenStDevUnqWd | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Kuperman | 0.03 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LDA | 0.03 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 |
| SenScoreStDev | 0.03 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 |
| AvgDepsSen_advmod | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 |
| AvgUnqNmdEntBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 |
| AvgAdjectiveBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 |
| RdbltyFog | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| AvgConnBl_sentence_link | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LSA | 0.03 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_word2vec | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_Path | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Cortese | 0.03 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvLinRegSlo | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 |
| AvgConnSen_sentence_link | 0.02 | 0 | 0 | 0.26 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgDepsSen_conj | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgConnBl_coord_conjs | 0.02 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgSenSyll | 0.02 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgConnBl_coord_conns | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgSemDep | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgSenStressedSyll | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgSenLen | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgAOEBl_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgAOESen_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_LDA | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 |
| AvgDepsSen_nsubj | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 |
| AvgAOESen_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 |
| AvgWdSen | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 |
| AvgVoice | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 |
| WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LeackChod | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_WuPalmer | 0.02 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 |
| AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bristol | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 |
| RdbltyKincaid | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 |
| AvgDepsSen_case | 0.02 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 |
| AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 |
| AvgConnSen_coord_conns | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 |
| AvgConnSen_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 |
| AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 |
| AvgDepsSen_cc | 0.01 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 |
| AvgDepsBl_cc | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 |
| AvgRhythmUnitSyll | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 |
| AvgConnSen_logical_cons | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 |
| AvgConnSen_contrasts | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 |
| AvgConnSen_coord_conjs | 0 | 0 | 0 | 0.06 | 0 | 0 | 0 | 0 |
ReaderBench Model 1b
This model was trained on winter data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:-------|:-------|:------|
| -2.0039 | 0.3112 | 0.1353 | 0.2667 | -0.0102 | 0.1234 | 0.0268 | 0.222 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:--------------------------|:------|:------|:-----|:-----|:-----|:-----|:------|:------|
| WdEnt | 11.34 | 23.56 | 2.43 | 1.65 | 0.74 | 3.11 | 31.83 | 12.9 |
| AvgDepsBl_det | 6.98 | 12.06 | 2.13 | 1.17 | 2.37 | 2.1 | 9.48 | 12.41 |
| AvgPronounBl | 3.71 | 2.92 | 1.9 | 0.99 | 0 | 1.27 | 7.58 | 10.22 |
| AvgUnqVerbBl | 3.43 | 6.53 | 2.29 | 1.36 | 1.89 | 2.17 | 0 | 3.65 |
| AvgDepsBl_nsubj | 2.97 | 5.88 | 2.3 | 1.43 | 1.14 | 2.11 | 0 | 2.19 |
| AvgUnqPrepositionBl | 2.78 | 4.75 | 2.01 | 1.04 | 1.59 | 1.86 | 0 | 3.65 |
| LxcDiv | 2.6 | 4 | 2.33 | 1.44 | 0.22 | 1.92 | 0 | 3.16 |
| WdDiffWdStem | 1.99 | 0.84 | 0.97 | 0.5 | 0.82 | 0.63 | 0 | 7.3 |
| AvgBlScore | 1.56 | 2.98 | 1.74 | 1.22 | 1.18 | 1.81 | 0 | 0 |
| AvgDepsBl_punct | 1.52 | 2.67 | 1.81 | 0.85 | 0.54 | 1.23 | 0 | 0.97 |
| AvgUnqNoundBl | 1.48 | 0.02 | 1.81 | 0.84 | 0.31 | 1.09 | 11.62 | 2.68 |
| AggPronSen_third_person | 1.37 | 0.36 | 0.55 | 0.21 | 0.28 | 0.67 | 18.65 | 2.19 |
| TCorefChainDoc | 1.3 | 0.78 | 1.88 | 0.91 | 0.67 | 1.7 | 0 | 2.19 |
| RdbltyFlesch | 1.3 | 2.18 | 0.2 | 0.44 | 1.65 | 0.98 | 0 | 2.19 |
| AvgPronBl_first_person | 1.19 | 0 | 1.69 | 0.74 | 1.34 | 0.66 | 0 | 3.65 |
| AvgSenBl | 1.16 | 2.23 | 1.8 | 0.84 | 0.27 | 0.97 | 0 | 0 |
| AvgDepsSen_advcl | 1.14 | 0.06 | 0.01 | 0.49 | 0.72 | 0.76 | 0 | 4.62 |
| AvgSenBlCoh_LeackChod | 1.12 | 1.42 | 1.49 | 0.61 | 0.42 | 1.35 | 0 | 1.22 |
| AvgSenBlCoh_LDA | 1.07 | 0.19 | 0.94 | 0.65 | 1.67 | 0.94 | 15.45 | 0.49 |
| LexChainMaxSp | 0.99 | 1.34 | 1.69 | 0.74 | 2.27 | 1.78 | 0 | 0 |
| AvgDepsBl_mark | 0.94 | 0.2 | 1.21 | 0.38 | 1.95 | 0.86 | 0 | 2.68 |
| CharEnt | 0.86 | 0.48 | 1.61 | 0.8 | 0.84 | 0.82 | 0 | 1.22 |
| AvgWdLen | 0.85 | 0.9 | 1.12 | 0.76 | 0.69 | 0.73 | 0 | 0.97 |
| AvgChainSpan | 0.79 | 0.68 | 1.73 | 1.07 | 1.32 | 1.08 | 0 | 0 |
| AvgAOESen_InfPointPoly | 0.73 | 0.22 | 0.35 | 0.23 | 0.12 | 0.39 | 0 | 2.68 |
| AvgDepsSen_det | 0.72 | 0.92 | 0.1 | 0.35 | 0.83 | 0.57 | 0 | 1.46 |
| AvgDepsSen_dobj | 0.69 | 0.6 | 0.53 | 0.4 | 0.94 | 0.52 | 0 | 1.46 |
| AvgUnqPronounBl | 0.66 | 0.14 | 1.72 | 0.76 | 0.12 | 1.13 | 0 | 0.49 |
| AvgConnBl_temp_conns | 0.65 | 1.22 | 1.12 | 0.33 | 1.18 | 0.69 | 0 | 0 |
| AvgDepsSen_compound | 0.64 | 0.99 | 0.55 | 0.18 | 1.46 | 1.18 | 0 | 0.49 |
| AvgSenAdjCoh_Path | 0.64 | 0.78 | 1.29 | 0.71 | 0.77 | 0.79 | 0 | 0 |
| WdDiffLemmaStem | 0.63 | 1.33 | 0.19 | 0.58 | 0.14 | 0.74 | 0 | 0 |
| RdbltyDaleChall | 0.63 | 1.19 | 1.14 | 0.4 | 0.66 | 0.48 | 0 | 0 |
| WdMaxDpthHypernymTree | 0.59 | 1.07 | 0.69 | 0.6 | 0.39 | 0.51 | 0 | 0 |
| LexChainAvgSpan | 0.57 | 0.28 | 0.88 | 0.7 | 1.27 | 0.59 | 3.91 | 0 |
| SenStdDevWd | 0.54 | 0.55 | 0.97 | 0.56 | 0.03 | 0.68 | 0 | 0.24 |
| AvgDepsSen_amod | 0.54 | 0.18 | 0.25 | 0.34 | 0.58 | 0.73 | 0 | 1.46 |
| AvgDepsBl_dobj | 0.54 | 0.02 | 1.67 | 0.72 | 2.01 | 0.84 | 0 | 0.24 |
| AvgDepsSen_dep | 0.52 | 0.87 | 0.47 | 0.4 | 0.18 | 1.08 | 0 | 0 |
| AvgDepsBl_nmod | 0.52 | 0.05 | 1.74 | 0.78 | 0.19 | 0.95 | 0 | 0 |
| FrqRhythmId | 0.51 | 0.56 | 1.17 | 0.42 | 0.91 | 0.9 | 0 | 0 |
| AvgDepsBl_advcl | 0.5 | 0.1 | 1.08 | 0.3 | 1.09 | 0.56 | 0 | 0.97 |
| AvgCorefChain | 0.5 | 0.3 | 1.13 | 0.49 | 1.28 | 0.45 | 0 | 0.49 |
| AvgDepsSen_nmod | 0.49 | 0.06 | 0.34 | 0.5 | 0.22 | 0.58 | 0 | 1.22 |
| AvgConnSen_addition | 0.49 | 0.1 | 0.62 | 0.49 | 0.44 | 0.68 | 0 | 0.97 |
| AvgAdjectiveBl | 0.48 | 0.23 | 1.51 | 0.59 | 0.56 | 0.7 | 0 | 0 |
| WdLettStdDev | 0.46 | 0.17 | 1.22 | 0.74 | 0.3 | 0.72 | 0 | 0 |
| AvgPronBl_indefinite | 0.44 | 0.23 | 1.25 | 0.4 | 0.87 | 1.02 | 0 | 0 |
| AvgConnBl_addition | 0.44 | 0.63 | 0.99 | 0.25 | 0.74 | 0.66 | 0 | 0 |
| AvgDepsSen_mark | 0.44 | 0.16 | 0.1 | 0.39 | 1.46 | 0.73 | 0 | 0.97 |
| LangRhythmCoeff | 0.44 | 0.71 | 0.54 | 0.34 | 0.04 | 0.82 | 0 | 0 |
| AvgVoice | 0.43 | 0.26 | 1.31 | 0.44 | 0.83 | 0.69 | 0 | 0 |
| AvgUnqAdverbBl | 0.43 | 0.03 | 1.52 | 0.6 | 0.67 | 0.76 | 0 | 0 |
| AvgAOABl_Shock | 0.43 | 0.25 | 1.21 | 0.44 | 0.69 | 0.88 | 0 | 0 |
| AvgBlLen | 0.41 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 |
| AvgPronBl_third_person | 0.41 | 0.2 | 1.12 | 0.32 | 0.64 | 0.33 | 0 | 0.49 |
| AvgUnqWdBl | 0.41 | 0 | 0 | 1.7 | 0 | 0 | 0 | 0 |
| Content.words | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 |
| AvgWdBl | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 |
| Words | 0.39 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 |
| AvgAOEBl_InfPointPoly | 0.39 | 0.14 | 0.34 | 0.24 | 0.81 | 0.48 | 0 | 0.97 |
| TCorefChainBigSpan | 0.38 | 0 | 1.06 | 0.29 | 1.22 | 1.08 | 1.47 | 0 |
| AvgDepsBl_cop | 0.37 | 0 | 1.4 | 0.51 | 1.32 | 0.58 | 0 | 0 |
| AvgConnSen_semi_coords | 0.37 | 0.06 | 0.12 | 0.29 | 0.84 | 0.63 | 0 | 0.97 |
| AvgVerbBl | 0.37 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 |
| AvgDepsSen_xcomp | 0.35 | 0.38 | 0.05 | 0.6 | 1.2 | 0.67 | 0 | 0 |
| AvgRhythmUnitStreesSyll | 0.34 | 0.4 | 0.2 | 0.33 | 1.14 | 0.93 | 0 | 0 |
| AvgAOEBl_InvLinRegSlo | 0.34 | 0.7 | 0.09 | 0.23 | 0.67 | 0.59 | 0 | 0 |
| AvgNmdEntSen | 0.34 | 0.3 | 0.21 | 0.4 | 0.48 | 1.1 | 0 | 0 |
| AvgConnBl_coord_connects | 0.34 | 0 | 1.25 | 0.4 | 1.85 | 0.63 | 0 | 0 |
| AvgAOABl_Bird | 0.34 | 0.25 | 0.44 | 0.41 | 1.53 | 0.92 | 0 | 0 |
| AvgDepsBl_xcomp | 0.33 | 0.18 | 1.15 | 0.34 | 0 | 0.45 | 0 | 0 |
| AvgAdverbSen | 0.33 | 0.27 | 0.2 | 0.4 | 2.52 | 0.97 | 0 | 0 |
| AvgConnBl_reas_purp | 0.33 | 0 | 0.96 | 0.24 | 1.1 | 0.45 | 0 | 0.49 |
| AvgAdverbBl | 0.33 | 0.04 | 1.29 | 0.43 | 0.75 | 0.5 | 0 | 0 |
| TActCorefChainWd | 0.33 | 0.62 | 0.39 | 0.29 | 0.22 | 0.34 | 0 | 0 |
| AvgAOABl_Bristol | 0.32 | 0.11 | 0.75 | 0.33 | 0.23 | 1 | 0 | 0 |
| AvgAOEBl_IndexPolyFAT.3 | 0.32 | 0.58 | 0 | 0.44 | 0.44 | 0.43 | 0 | 0 |
| AvgDepsBl_aux | 0.32 | 0 | 0.52 | 0.07 | 0.8 | 0.39 | 0 | 0.97 |
| AvgConnSen_logical_conns | 0.31 | 0.13 | 0.7 | 0.47 | 0.7 | 0.55 | 0 | 0 |
| AvgDepsSen_punct | 0.31 | 0.05 | 0.9 | 0.55 | 0.75 | 0.45 | 0 | 0 |
| AvgCommaBl | 0.3 | 0 | 1.16 | 0.35 | 0.23 | 0.69 | 0 | 0 |
| AvgDepsSen_aux | 0.29 | 0.02 | 0.45 | 0.36 | 0.43 | 1.18 | 0 | 0 |
| AvgNounBl | 0.29 | 0 | 0 | 1.2 | 0 | 0 | 0 | 0 |
| WdPolysemyCnt | 0.29 | 0.56 | 0.15 | 0.14 | 0.03 | 0.68 | 0 | 0 |
| AvgConnSen_reas_purp | 0.29 | 0.02 | 0.16 | 0.26 | 1.13 | 0.85 | 0 | 0.49 |
| AvgDepsBl_compound | 0.29 | 0.02 | 0.12 | 0 | 1.6 | 0.57 | 0 | 0.97 |
| AvgDepsBl_amod | 0.28 | 0 | 1.1 | 0.32 | 0.78 | 0.53 | 0 | 0 |
| AvgAOASen_Bird | 0.28 | 0.51 | 0.15 | 0.23 | 1.13 | 0.46 | 0 | 0 |
| WdPathCntHypernymTree | 0.28 | 0.06 | 0.7 | 0.51 | 0.06 | 0.52 | 0 | 0 |
| AvgDepsBl_acl | 0.27 | 0.03 | 1 | 0.26 | 0.7 | 0.59 | 0 | 0 |
| AvgDepsBl_ccomp | 0.27 | 0 | 0.9 | 0.21 | 0.92 | 0.86 | 0 | 0 |
| AvgAOASen_Shock | 0.27 | 0.02 | 0.53 | 0.5 | 0.19 | 0.66 | 0 | 0 |
| AggPronSen_indefinite | 0.27 | 0.07 | 0.04 | 0.61 | 0.75 | 0.79 | 0 | 0 |
| AvgDepsSen_cop | 0.27 | 0.16 | 0.03 | 0.55 | 0.5 | 0.75 | 0 | 0 |
| AvgConnBl_simp_subords | 0.27 | 0.06 | 1.07 | 0.3 | 0.95 | 0.41 | 0 | 0 |
| AvgConnBl_semi_coords | 0.26 | 0 | 0.68 | 0.12 | 0.22 | 0.42 | 0 | 0.49 |
| AvgAOESen_IndexAbThr.0.3. | 0.26 | 0.1 | 0.23 | 0.53 | 0.02 | 0.64 | 0 | 0 |
| AvgAOEBl_IndexAbThr.0.3. | 0.26 | 0.21 | 0 | 0.61 | 0.28 | 0.43 | 0 | 0 |
| WdSylCnt | 0.25 | 0.15 | 0.39 | 0.48 | 0.97 | 0.3 | 0 | 0 |
| AvgDepsBl_neg | 0.25 | 0.03 | 0.86 | 0.19 | 0.09 | 0.84 | 0 | 0 |
| AvgConnBl_contrasts | 0.23 | 0 | 0.73 | 0.14 | 0.65 | 0.85 | 0 | 0 |
| AvgNounSen | 0.23 | 0.05 | 0.42 | 0.24 | 0.23 | 0.89 | 0 | 0 |
| AvgDepsSen_ccomp | 0.23 | 0.18 | 0.07 | 0.29 | 0.47 | 0.83 | 0 | 0 |
| AvgConnBl_logical_conns | 0.23 | 0.16 | 0.87 | 0.19 | 0.05 | 0.27 | 0 | 0 |
| AvgConnSen_simp_subords | 0.23 | 0.1 | 0.05 | 0.54 | 0 | 0.54 | 0 | 0 |
| AvgAdjectiveSen | 0.23 | 0.1 | 0.21 | 0.34 | 0.85 | 0.79 | 0 | 0 |
| AvgConnBl_oppositions | 0.23 | 0 | 0.79 | 0.16 | 0.75 | 0.8 | 0 | 0 |
| AvgInferenceDistChain | 0.23 | 0.1 | 0.42 | 0.26 | 0.53 | 0.76 | 0 | 0 |
| AvgConnBl_order | 0.22 | 0 | 0.9 | 0.21 | 2.32 | 0.29 | 0 | 0 |
| AvgDepsBl_conj | 0.22 | 0 | 0.79 | 0.16 | 1.65 | 0.57 | 0 | 0 |
| AvgPrepositionBl | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 |
| AvgSenScore | 0.22 | 0.05 | 0.14 | 0.33 | 0.33 | 0.9 | 0 | 0 |
| AvgDepsBl_case | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 |
| AvgAOESen_IndexPolyFAT.3 | 0.22 | 0.07 | 0.26 | 0.42 | 0.35 | 0.55 | 0 | 0 |
| AvgAOABl_Cortese | 0.22 | 0.05 | 0.35 | 0.32 | 1.05 | 0.67 | 0 | 0 |
| AvgConnSen_oppositions | 0.21 | 0.03 | 0.03 | 0.35 | 1.28 | 0.84 | 0 | 0 |
| AvgIntraBlCoh_word2vec | 0.21 | 0 | 0 | 0.88 | 0 | 0 | 0 | 0 |
| AvgRhythmUnits | 0.21 | 0 | 0.34 | 0.42 | 1.08 | 0.53 | 0 | 0 |
| AvgNounNmdEntBl | 0.2 | 0.14 | 0.52 | 0.07 | 2.7 | 0.46 | 0 | 0 |
| Sentences | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| AvgAOABl_Kuperman | 0.2 | 0.13 | 0.06 | 0.32 | 0.93 | 0.64 | 0 | 0 |
| AvgIntraBlCoh_LDA | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| AvgNmdEntBl | 0.2 | 0.03 | 0.82 | 0.18 | 1.51 | 0.35 | 0 | 0 |
| AvgConnSen_order | 0.19 | 0.11 | 0.13 | 0.37 | 0.55 | 0.41 | 0 | 0 |
| AvgConnSen_temp_conns | 0.19 | 0.27 | 0.12 | 0 | 1.47 | 0.77 | 0 | 0 |
| LxcSoph | 0.19 | 0.05 | 0.18 | 0.34 | 0.45 | 0.61 | 0 | 0 |
| LangRhythmDiameter | 0.19 | 0 | 0.3 | 0.02 | 1.45 | 0.37 | 0 | 0.49 |
| AvgIntraBlCoh_LSA | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_Path | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 |
| AvgAOASen_Kuperman | 0.18 | 0.28 | 0.06 | 0.17 | 0.06 | 0.45 | 0 | 0 |
| AvgDepsBl_nsubjpass | 0.18 | 0.04 | 0.66 | 0.11 | 0.38 | 0.53 | 0 | 0 |
| AvgAOASen_Bristol | 0.18 | 0.13 | 0.18 | 0.25 | 0.36 | 0.55 | 0 | 0 |
| AvgSenAdjCoh_LDA | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_Path | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LeackChod | 0.17 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LeackChod | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_word2vec | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_WuPalmer | 0.17 | 0 | 0 | 0.72 | 0 | 0 | 0 | 0 |
| SenAsson | 0.16 | 0 | 0.47 | 0.06 | 0.6 | 0.7 | 0 | 0 |
| AvgDepsBl_nummod | 0.16 | 0 | 0.47 | 0.06 | 0.07 | 0.75 | 0 | 0 |
| AvgSenBlCoh_word2vec | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_WuPalmer | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| SenStDevUnqWd | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndexAbThr.0.3. | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 |
| AvgUnqWdSen | 0.15 | 0 | 0 | 0.62 | 0 | 0 | 0 | 0 |
| SynSoph | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 |
| AvgUnqAdjectiveBl | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LSA | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 |
| AvgWdSen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 |
| AvgSenLen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 |
| AvgConnBl_sentence_link | 0.14 | 0 | 0 | 0.57 | 0 | 0 | 0 | 0 |
| WdAvgDpthHypernymTree | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_LSA | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_WuPalmer | 0.14 | 0 | 0 | 0.6 | 0 | 0 | 0 | 0 |
| AvgDepsSen_mwe | 0.13 | 0 | 0.31 | 0.03 | 0.32 | 0.7 | 0 | 0 |
| AvgDepsBl_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 |
| AvgDepsSen_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 |
| AvgConnSen_contrasts | 0.13 | 0 | 0.14 | 0.22 | 0.06 | 0.55 | 0 | 0 |
| AvgDepsSen_nsubj | 0.12 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 |
| SenScoreStDev | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 |
| AvgVerbSen | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 |
| AvgSenStressedSyll | 0.12 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 |
| AvgBlVoiceCoOcc | 0.12 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Shock | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndexPolyFAT.3 | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 |
| AvgSemDep | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 |
| AvgDepsSen_case | 0.11 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| AvgRhythmUnitSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 |
| AvgSenSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bird | 0.1 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 |
| AvgDepsSen_acl | 0.1 | 0.06 | 0.2 | 0.01 | 0.6 | 0.43 | 0 | 0 |
| RdbltyKincaid | 0.1 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 |
| AvgAOASen_Cortese | 0.1 | 0.12 | 0.06 | 0.25 | 0.22 | 0 | 0 | 0 |
| AvgConnBl_conjunctions | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 |
| AvgPronounSen | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 |
| RdbltyFog | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Cortese | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Kuperman | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| AvgPrepositionSen | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| AvgDepsBl_dep | 0.08 | 0.04 | 0.25 | 0.02 | 0.15 | 0.29 | 0 | 0 |
| AvgDepsBl_cc | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bristol | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgConnSen_sentence_link | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgDepsSen_neg | 0.08 | 0 | 0.13 | 0 | 0.14 | 0.58 | 0 | 0 |
| AvgDepsSen_conj | 0.08 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgDepsBl_mwe | 0.08 | 0 | 0.43 | 0.05 | 0.03 | 0.18 | 0 | 0 |
| AvgConnSen_coord_connects | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgConnSen_coord_conjs | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgAOEBl_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgAOESen_InvAverage | 0.07 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 |
| AvgDepsSen_cc | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgConnSen_conjunctions | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InfPointPoly | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 |
| AvgUnqNmdEntBl | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 |
| LangRhythmId | 0.06 | 0.03 | 0.04 | 0 | 0.33 | 0.37 | 0 | 0 |
| AvgAOESen_InvLinRegSlo | 0.05 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvLinRegSlo | 0.05 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 |
| AvgDepsBl_auxpass | 0.04 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 |
| AvgConnBl_coord_conjs | 0.03 | 0 | 0 | 0.12 | 0 | 0 | 0 | 0 |
ReaderBench Model 1c
This model was trained on spring data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:-------|:-------|:--------|
| -5.6692 | 0.1651 | 0.2625 | 0.1043 | -0.0146 | 0.4555 | 0.1632 | -0.0348 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:--------------------------|:-----|:------|:-----|:-----|:-----|:-----|:------|:------|
| WdEnt | 7.76 | 15.63 | 2.27 | 1.53 | 2 | 3.41 | 25.87 | 9.83 |
| AvgUnqVerbBl | 5.89 | 24.09 | 2.26 | 1.36 | 2.3 | 3.86 | 0 | 12.86 |
| AvgBlScore | 2.6 | 7.12 | 1.53 | 1.13 | 0.58 | 2.67 | 0 | 4.69 |
| AvgNounSen | 2.57 | 0.61 | 0.6 | 0.39 | 1.06 | 1.04 | 14.54 | 2.12 |
| LxcDiv | 2.18 | 3.58 | 2.18 | 1.33 | 0.72 | 2.54 | 0 | 3.78 |
| AvgDepsSen_compound | 2.17 | 1.45 | 1.12 | 0.56 | 0.27 | 1.54 | 7.94 | 2.12 |
| WdDiffLemmaStem | 2.07 | 0.57 | 0.93 | 0.49 | 0.22 | 1 | 10.51 | 0 |
| AvgDepsBl_dobj | 2.06 | 0.09 | 1.63 | 0.74 | 0.92 | 1.13 | 9.09 | 0 |
| AvgUnqPrepositionBl | 1.96 | 1.98 | 2.25 | 1.27 | 1.28 | 2.47 | 0 | 5.6 |
| AvgDepsBl_punct | 1.85 | 2.48 | 1.93 | 0.93 | 1.96 | 2.21 | 0 | 6.2 |
| AvgDepsBl_nsubj | 1.84 | 3.1 | 1.97 | 1.26 | 0.88 | 2.06 | 0 | 1.36 |
| RdbltyFlesch | 1.75 | 0.27 | 0.22 | 0.16 | 1.12 | 0.23 | 12.02 | 0.15 |
| AvgWdLen | 1.74 | 3.11 | 1.54 | 0.76 | 0.21 | 2.1 | 0 | 3.03 |
| AvgDepsBl_nmod | 1.65 | 2.75 | 1.94 | 0.92 | 0.42 | 1.77 | 0 | 2.42 |
| AvgUnqNoundBl | 1.28 | 0.06 | 0.99 | 0.68 | 1.79 | 0.1 | 6.94 | 2.72 |
| AvgPronounBl | 1.28 | 0.4 | 1.85 | 1.14 | 0.98 | 1.75 | 0 | 1.21 |
| AvgUnqPronounBl | 1.23 | 0.36 | 1.69 | 0.74 | 0.27 | 0.86 | 3 | 0.76 |
| WdSylCnt | 1.22 | 0.89 | 1.39 | 0.58 | 1.52 | 1.62 | 0 | 5.45 |
| AvgUnqAdjectiveBl | 1.19 | 0.24 | 1.37 | 0.54 | 0.34 | 0.6 | 4.3 | 0.76 |
| AvgDepsBl_ccomp | 1.18 | 0.35 | 0.61 | 0.05 | 0.35 | 0.61 | 5.78 | 0.76 |
| AvgChainSpan | 1.17 | 0.51 | 1.66 | 0.99 | 0.49 | 1.6 | 0 | 0.91 |
| LexChainMaxSp | 1.14 | 0.38 | 1.67 | 0.75 | 1.99 | 1.62 | 0 | 0 |
| WdDiffWdStem | 1.11 | 0.76 | 1.38 | 0.78 | 0.06 | 1.6 | 0 | 0 |
| AvgDepsBl_det | 1.1 | 0.29 | 1.71 | 0.72 | 0.82 | 1.52 | 0 | 0.76 |
| WdLettStdDev | 1.02 | 1.03 | 1.66 | 0.85 | 1.56 | 1.04 | 0 | 0.3 |
| FrqRhythmId | 1 | 0.65 | 1.44 | 0.59 | 0.95 | 1.32 | 0 | 0.76 |
| AvgAOABl_Shock | 0.97 | 0.91 | 1.13 | 0.67 | 0.02 | 1.34 | 0 | 0 |
| AvgSenBlCoh_LDA | 0.97 | 1.01 | 1.13 | 0.82 | 1.53 | 1.24 | 0 | 0 |
| AvgDepsBl_mark | 0.96 | 0.04 | 1.52 | 0.65 | 1.46 | 1.4 | 0 | 0 |
| AvgSenAdjCoh_word2vec | 0.9 | 0.39 | 1.37 | 0.67 | 0.29 | 1.18 | 0 | 1.06 |
| TCorefChainDoc | 0.87 | 0.32 | 1.67 | 0.71 | 2.43 | 0.96 | 0 | 0 |
| AvgDepsSen_punct | 0.81 | 0.4 | 1 | 0.68 | 0.26 | 1.2 | 0 | 0 |
| LangRhythmCoeff | 0.8 | 0.47 | 0.95 | 0.46 | 0.2 | 1.23 | 0 | 0 |
| AvgPronBl_first_person | 0.78 | 0.58 | 1.31 | 0.44 | 0.15 | 0.86 | 0 | 1.66 |
| RdbltyDaleChall | 0.78 | 1.44 | 0.96 | 0.38 | 1.08 | 0.76 | 0 | 1.06 |
| AvgConnBl_sentence_link | 0.74 | 0.09 | 1.31 | 0.45 | 3.5 | 0.92 | 0 | 0.3 |
| AvgDepsBl_compound | 0.74 | 1.08 | 0.77 | 0.12 | 1.07 | 0.86 | 0 | 3.48 |
| AvgConnSen_logical_conns | 0.71 | 0.31 | 0.54 | 0.51 | 0.72 | 1.24 | 0 | 0.76 |
| LexChainAvgSpan | 0.7 | 0.67 | 1.02 | 0.62 | 0.5 | 0.81 | 0 | 0 |
| AvgUnqAdverbBl | 0.69 | 0.01 | 1.46 | 0.57 | 0.78 | 0.75 | 0 | 0.76 |
| CharEnt | 0.66 | 0.48 | 1.35 | 0.78 | 1.51 | 0.49 | 0 | 0.76 |
| AvgDepsBl_amod | 0.66 | 0.23 | 1.31 | 0.48 | 1.2 | 0.73 | 0 | 0 |
| AvgRhythmUnitStreesSyll | 0.65 | 0.39 | 0.43 | 0.21 | 0.86 | 1.22 | 0 | 0 |
| AvgAdjectiveSen | 0.64 | 0.7 | 0.68 | 0.45 | 0.03 | 0.78 | 0 | 2.72 |
| AvgDepsBl_advcl | 0.62 | 0.07 | 1.21 | 0.42 | 1.83 | 0.74 | 0 | 0 |
| AvgConnBl_simp_subords | 0.61 | 0.04 | 1.25 | 0.45 | 0.28 | 0.69 | 0 | 0.76 |
| AvgConnBl_reas_purp | 0.61 | 0.57 | 0.98 | 0.27 | 0.17 | 0.74 | 0 | 0 |
| AvgCorefChain | 0.6 | 0.54 | 1.17 | 0.48 | 1.94 | 0.52 | 0 | 0 |
| AvgPronBl_indefinite | 0.59 | 0.2 | 1.33 | 0.5 | 0.05 | 0.57 | 0 | 0 |
| AvgDepsBl_xcomp | 0.58 | 0.36 | 1.14 | 0.37 | 0.09 | 0.63 | 0 | 0 |
| AvgBlVoiceCoOcc | 0.57 | 0 | 1.44 | 0.56 | 0.17 | 0.51 | 0 | 0 |
| AvgDepsBl_aux | 0.56 | 0.17 | 1.12 | 0.32 | 1.5 | 0.64 | 0 | 0 |
| AvgDepsBl_mwe | 0.55 | 0 | 0.93 | 0.23 | 1.62 | 0.79 | 0 | 0 |
| AvgPronBl_third_person | 0.55 | 0.13 | 1.23 | 0.39 | 0.24 | 0.53 | 0 | 1.06 |
| AggPronSen_third_person | 0.55 | 0.37 | 0.47 | 0.21 | 2.96 | 0.87 | 0 | 0.76 |
| AvgAOABl_Cortese | 0.55 | 0.56 | 0.64 | 0.7 | 0.31 | 0.68 | 0 | 0 |
| AvgDepsBl_neg | 0.54 | 0.08 | 0.71 | 0.15 | 0.29 | 0.91 | 0 | 0 |
| AvgConnSen_order | 0.54 | 0.06 | 0.26 | 0.52 | 0.98 | 1.07 | 0 | 0 |
| AvgDepsSen_amod | 0.53 | 0.43 | 0.6 | 0.49 | 0.07 | 0.72 | 0 | 0.61 |
| AvgInferenceDistChain | 0.53 | 0.52 | 0.46 | 0.31 | 1.67 | 0.81 | 0 | 0 |
| TCorefChainBigSpan | 0.52 | 0.04 | 1.24 | 0.35 | 0.13 | 0.53 | 0 | 0 |
| SenAsson | 0.51 | 0.33 | 0.85 | 0.21 | 1.05 | 0.63 | 0 | 0.15 |
| AvgConnBl_contrasts | 0.5 | 0.06 | 0.85 | 0.21 | 0 | 0.72 | 0 | 0 |
| AggPronSen_indefinite | 0.49 | 0.31 | 0.15 | 0.39 | 0.83 | 0.93 | 0 | 0.76 |
| AvgDepsSen_xcomp | 0.48 | 0.76 | 0.22 | 0.37 | 0.33 | 0.72 | 0 | 0 |
| AvgConnBl_temp_conns | 0.47 | 0.2 | 1.04 | 0.26 | 0.13 | 0.46 | 0 | 0.61 |
| AvgDepsBl_cop | 0.46 | 0.25 | 0.92 | 0.23 | 0.13 | 0.5 | 0 | 0 |
| AvgAOASen_Kuperman | 0.46 | 0.19 | 0.45 | 0.37 | 1.7 | 0.71 | 0 | 0.61 |
| AvgDepsSen_nmod | 0.46 | 0.2 | 0.11 | 0.46 | 1.37 | 0.74 | 0 | 4.39 |
| AvgDepsSen_dobj | 0.45 | 0.17 | 0.44 | 0.42 | 0.55 | 0.74 | 0 | 0 |
| AvgDepsBl_nummod | 0.44 | 0.07 | 0.35 | 0.02 | 0.46 | 0.88 | 0 | 0 |
| AvgAOEBl_IndexPolyFAT.3 | 0.44 | 0.19 | 0.42 | 0.4 | 1.29 | 0.71 | 0 | 0 |
| AvgConnBl_order | 0.42 | 0.05 | 0.62 | 0.11 | 0.53 | 0.67 | 0 | 0 |
| LxcSoph | 0.42 | 0.28 | 0.06 | 0.07 | 0.93 | 0.88 | 0 | 0.76 |
| AvgDepsSen_neg | 0.42 | 0.06 | 0.25 | 0.02 | 0.84 | 0.91 | 0 | 0 |
| AvgConnSen_simp_subords | 0.41 | 0.36 | 0.07 | 0.56 | 0.46 | 0.74 | 0 | 0 |
| SenStdDevWd | 0.4 | 0.08 | 0.88 | 0.66 | 0.4 | 0.32 | 0 | 0 |
| AvgConnBl_oppositions | 0.4 | 0.02 | 0.95 | 0.24 | 1.8 | 0.37 | 0 | 0 |
| AvgDepsSen_ccomp | 0.4 | 0.1 | 0.57 | 0.44 | 0.58 | 0.47 | 0 | 2.12 |
| AvgAOABl_Kuperman | 0.39 | 0.38 | 0.37 | 0.53 | 0.22 | 0.51 | 0 | 0 |
| AvgRhythmUnits | 0.39 | 0.28 | 0.16 | 0.48 | 0.22 | 0.68 | 0 | 0 |
| AvgAOEBl_InvLinRegSlo | 0.37 | 0.13 | 0.67 | 0.33 | 0.23 | 0.41 | 0 | 0.76 |
| TActCorefChainWd | 0.37 | 0.29 | 0.51 | 0.26 | 0.8 | 0.49 | 0 | 0 |
| AvgPronounSen | 0.36 | 0.32 | 0.6 | 0.37 | 0.49 | 0.33 | 0 | 0.76 |
| AvgDepsSen_mwe | 0.36 | 0.01 | 0.33 | 0.03 | 0.68 | 0.73 | 0 | 0 |
| AvgAOASen_Bristol | 0.35 | 0.6 | 0.24 | 0.13 | 0.88 | 0.42 | 0 | 1.36 |
| LangRhythmDiameter | 0.34 | 0.13 | 0.23 | 0.01 | 0.31 | 0.68 | 0 | 0 |
| AvgCommaBl | 0.34 | 0.07 | 0.66 | 0.12 | 0.49 | 0.43 | 0 | 0 |
| AvgAOASen_Cortese | 0.34 | 0.33 | 0.69 | 0.45 | 0.27 | 0.24 | 0 | 0 |
| AvgDepsSen_mark | 0.34 | 0.11 | 0.22 | 0.54 | 0.34 | 0.58 | 0 | 0 |
| AvgDepsSen_acl | 0.33 | 0.04 | 0.71 | 0.13 | 0.93 | 0.38 | 0 | 0 |
| AvgConnBl_logical_conns | 0.33 | 0.05 | 0.53 | 0.06 | 0.26 | 0.51 | 0 | 0 |
| WdPathCntHypernymTree | 0.32 | 0.52 | 0.45 | 0.13 | 0.56 | 0.32 | 0 | 0 |
| AvgConnBl_semi_coords | 0.32 | 0.14 | 0.78 | 0.16 | 0.43 | 0.27 | 0 | 0 |
| AvgDepsSen_cop | 0.31 | 0.2 | 0.47 | 0.47 | 0.17 | 0.34 | 0 | 0 |
| AvgAOESen_InfPointPoly | 0.3 | 0.34 | 0.31 | 0.16 | 1.12 | 0.4 | 0 | 0 |
| AvgConnSen_semi_coords | 0.3 | 0.06 | 0.01 | 0.34 | 0.68 | 0.64 | 0 | 0 |
| AvgAOASen_Shock | 0.3 | 0.28 | 0.21 | 0.49 | 0.06 | 0.4 | 0 | 0.61 |
| AvgConnSen_oppositions | 0.3 | 0.08 | 0.02 | 0 | 0.39 | 0.73 | 0 | 0 |
| AvgDepsSen_advmod | 0.29 | 0.32 | 0.18 | 0.47 | 0.28 | 0.42 | 0 | 0 |
| AvgConnBl_addition | 0.28 | 0.18 | 0.64 | 0.1 | 2.79 | 0.2 | 0 | 0 |
| WdPolysemyCnt | 0.28 | 0.33 | 0.11 | 0.6 | 0.09 | 0.39 | 0 | 0 |
| AvgAOABl_Bristol | 0.28 | 0.19 | 0.22 | 0.38 | 0.49 | 0.43 | 0 | 0 |
| AvgNmdEntSen | 0.27 | 0.26 | 0.54 | 0.39 | 0.36 | 0.19 | 0 | 0 |
| AvgAOEBl_InfPointPoly | 0.27 | 0.2 | 0.35 | 0.23 | 2.11 | 0.3 | 0 | 0.76 |
| AvgConnSen_temp_conns | 0.27 | 0.35 | 0.12 | 0 | 0.93 | 0.48 | 0 | 0 |
| WdAvgDpthHypernymTree | 0.27 | 0.43 | 0.4 | 0.2 | 0.15 | 0.25 | 0 | 0 |
| AvgDepsSen_dep | 0.27 | 0.06 | 0.38 | 0.28 | 2.35 | 0.34 | 0 | 0.3 |
| AvgAOESen_InvLinRegSlo | 0.26 | 0.27 | 0.52 | 0.25 | 0.02 | 0.19 | 0 | 0.3 |
| AvgAOASen_Bird | 0.26 | 0.53 | 0.04 | 0.16 | 0.75 | 0.39 | 0 | 0.15 |
| AvgNmdEntBl | 0.26 | 0.17 | 0.48 | 0.04 | 1.53 | 0.3 | 0 | 0 |
| AvgConnSen_reas_purp | 0.25 | 0.12 | 0.08 | 0.48 | 1.14 | 0.4 | 0 | 0 |
| AvgDepsSen_det | 0.25 | 0.11 | 0.1 | 0.25 | 0.26 | 0.44 | 0 | 0.76 |
| AvgAOABl_Bird | 0.23 | 0.7 | 0.21 | 0.33 | 0.44 | 0.13 | 0 | 0 |
| AvgAOESen_IndexPolyFAT.3 | 0.23 | 0.11 | 0.28 | 0.31 | 0.64 | 0.31 | 0 | 0 |
| AvgDepsBl_dep | 0.22 | 0.1 | 0.44 | 0.05 | 1.31 | 0.22 | 0 | 0.61 |
| AvgDepsBl_nsubjpass | 0.21 | 0.02 | 0.84 | 0.2 | 0.07 | 0 | 0 | 0 |
| AvgAOESen_IndexAbThr.0.3. | 0.19 | 0.27 | 0.12 | 0.39 | 0.05 | 0.22 | 0 | 0 |
| AvgDepsSen_aux | 0.18 | 0.39 | 0 | 0.49 | 2.05 | 0.17 | 0 | 0 |
| AvgUnqWdBl | 0.15 | 0 | 0 | 1.66 | 0 | 0 | 0 | 0 |
| AvgBlLen | 0.15 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 |
| AvgDepsBl_acl | 0.14 | 0.01 | 0.42 | 0.04 | 0.2 | 0.1 | 0 | 0 |
| AvgVerbBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 |
| Content.words | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 |
| AvgWdBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 |
| AvgNounNmdEntBl | 0.14 | 0.29 | 0.05 | 0 | 1.08 | 0.21 | 0 | 0 |
| Words | 0.13 | 0 | 0 | 1.47 | 0 | 0 | 0 | 0 |
| AvgPrepositionBl | 0.11 | 0 | 0 | 1.22 | 0 | 0 | 0 | 0 |
| AvgDepsSen_advcl | 0.1 | 0.14 | 0.05 | 0.52 | 0.09 | 0.05 | 0 | 0 |
| AvgDepsBl_case | 0.09 | 0 | 0 | 0.96 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LDA | 0.09 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 |
| LangRhythmId | 0.09 | 0.01 | 0.19 | 0 | 0.98 | 0.1 | 0 | 0 |
| AvgIntraBlCoh_Path | 0.08 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LDA | 0.08 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LSA | 0.08 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_LSA | 0.07 | 0 | 0 | 0.74 | 0 | 0 | 0 | 0 |
| SenScoreStDev | 0.07 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_Path | 0.07 | 0 | 0 | 0.76 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_word2vec | 0.07 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_Path | 0.07 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 |
| AvgNounBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 |
| Sentences | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 |
| AvgSenBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LSA | 0.07 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_WuPalmer | 0.06 | 0 | 0 | 0.64 | 0 | 0 | 0 | 0 |
| SenStDevUnqWd | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| AvgSenAdjCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Shock | 0.06 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 |
| AvgSenBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_LeackChod | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 |
| AvgIntraBlCoh_word2vec | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Cortese | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 |
| AvgVoice | 0.05 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 |
| AvgConnSen_sentence_link | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 |
| AvgVerbSen | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 |
| AvgDepsSen_nsubj | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Kuperman | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| AvgConnSen_conjunctions | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 |
| AvgConnSen_coord_connects | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 |
| AvgAdjectiveBl | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 |
| AvgDepsSen_case | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndexPolyFAT.3 | 0.04 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 |
| AvgPrepositionSen | 0.04 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 |
| AvgAdverbSen | 0.04 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 |
| AvgSenSyll | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgAOEBl_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgConnSen_addition | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgSemDep | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgDepsSen_cc | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 |
| AvgDepsBl_advmod | 0.04 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| AvgWdSen | 0.03 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 |
| AvgSenStressedSyll | 0.03 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 |
| AvgConnSen_contrasts | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgSenScore | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvLinRegSlo | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 |
| AvgConnSen_coord_conjs | 0.03 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 |
| AvgDepsSen_conj | 0.03 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 |
| SynSoph | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 |
| AvgAOADoc_Bristol | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 |
| AvgAdverbBl | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 |
| AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 |
| AvgConnBl_coord_connects | 0.02 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 |
| AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 |
| WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 |
| AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 |
| AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 |
| RdbltyKincaid | 0.02 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 |
| AvgUnqWdSen | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 |
| RdbltyFog | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 |
| AvgRhythmUnitSyll | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 |
| AvgSenLen | 0.02 | 0 | 0 | 0.28 | 0 | 0 | 0 | 0 |
| AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 |
| AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 |
| AvgDepsBl_cc | 0.01 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 |
| AvgConnBl_coord_conjs | 0.01 | 0 | 0 | 0.16 | 0 | 0 | 0 | 0 |
| AvgUnqNmdEntBl | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 |
ReaderBench Model 1d
This model was trained on principal component scores for fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------|
| -8.4195 | 0.0406 | 0.8127 | 0.0694 | -0.0509 | 0.1058 | 0.0038 | 0.0448 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:-----|:------|:------|:----|:------|:-----|:------|:------|
| PC2 | 50.2 | 61.92 | 52.85 | 0 | 12.57 | 36.4 | 46.85 | 23.81 |
| PC1 | 8.03 | 1.95 | 9.25 | 0 | 1.38 | 5.78 | 0.27 | 8.39 |
| PC3 | 7.38 | 1.56 | 8.43 | 0 | 3.57 | 5.47 | 0 | 8.39 |
| PC5 | 5.42 | 3.46 | 5.74 | 0 | 5.67 | 3.71 | 0 | 13.83 |
| PC4 | 2 | 1.81 | 2.27 | 0 | 1.61 | 1.06 | 0 | 0 |
| PC24 | 1.9 | 0.9 | 1.48 | 0 | 6.31 | 1.73 | 16.76 | 1.59 |
| PC14 | 1.7 | 1.4 | 1.56 | 0 | 3.22 | 1.85 | 3.63 | 3.85 |
| PC8 | 1.47 | 0.85 | 1.45 | 0 | 1.88 | 2.26 | 0 | 1.59 |
| PC30 | 1.47 | 1.67 | 1.1 | 0 | 6.24 | 2.51 | 0 | 6.58 |
| PC6 | 1.09 | 0.73 | 0.95 | 0 | 0.78 | 2.84 | 0 | 0 |
| PC31 | 1.09 | 1.01 | 0.92 | 0 | 5.49 | 0.03 | 0 | 9.98 |
| PC7 | 1.03 | 0.17 | 1.17 | 0 | 1.27 | 0.95 | 0 | 0 |
| PC17 | 1.02 | 1.53 | 0.62 | 0 | 1.24 | 3.12 | 0 | 4.31 |
| PC43 | 0.94 | 1.87 | 0.58 | 0 | 5.88 | 1.5 | 0 | 4.76 |
| PC33 | 0.87 | 0.6 | 0.88 | 0 | 5.75 | 0.44 | 0 | 0 |
| PC32 | 0.82 | 0.21 | 0.6 | 0 | 3.29 | 2.36 | 0 | 1.59 |
| PC39 | 0.81 | 1.14 | 0.54 | 0 | 4.22 | 1.87 | 1.81 | 0 |
| PC27 | 0.77 | 0.48 | 0.66 | 0 | 2.74 | 0.62 | 0 | 4.99 |
| PC13 | 0.73 | 1.28 | 0.73 | 0 | 1.02 | 0.55 | 0 | 0 |
| PC34 | 0.7 | 0.46 | 0.2 | 0 | 0.43 | 2.05 | 13.64 | 0 |
| PC38 | 0.67 | 0.36 | 0.55 | 0 | 4.08 | 0.86 | 0 | 2.72 |
| PC23 | 0.64 | 0.5 | 0.72 | 0 | 2.45 | 0.1 | 0 | 0 |
| PC19 | 0.6 | 0.9 | 0.56 | 0 | 1.28 | 0.84 | 0 | 0 |
| PC16 | 0.6 | 0.67 | 0.22 | 0 | 0 | 2.25 | 6.99 | 0 |
| PC45 | 0.6 | 1.92 | 0.07 | 0 | 0 | 1.77 | 10.06 | 0.91 |
| PC20 | 0.56 | 0 | 0.53 | 0 | 1.3 | 1.26 | 0 | 0 |
| PC28 | 0.54 | 1.55 | 0.39 | 0 | 1.29 | 0.94 | 0 | 0.45 |
| PC40 | 0.54 | 0.77 | 0.52 | 0 | 4.01 | 0.04 | 0 | 0.45 |
| PC18 | 0.53 | 0.36 | 0.47 | 0 | 0.92 | 1.27 | 0 | 0 |
| PC11 | 0.51 | 0.87 | 0.55 | 0 | 0.48 | 0.18 | 0 | 0 |
| PC22 | 0.49 | 0.35 | 0.42 | 0 | 0.99 | 1.2 | 0 | 0 |
| PC36 | 0.46 | 0.95 | 0.09 | 0 | 0 | 3.14 | 0 | 0 |
| PC12 | 0.45 | 0.22 | 0.46 | 0 | 0.33 | 0.68 | 0 | 0 |
| PC29 | 0.44 | 0.95 | 0.29 | 0 | 0.79 | 1.36 | 0 | 0 |
| PC41 | 0.43 | 0.27 | 0.35 | 0 | 2.57 | 0.79 | 0 | 0 |
| PC42 | 0.4 | 0.32 | 0.31 | 0 | 2.25 | 1.03 | 0 | 0 |
| PC9 | 0.37 | 0.34 | 0.4 | 0 | 0.13 | 0.33 | 0 | 0 |
| PC44 | 0.29 | 0.31 | 0.22 | 0 | 1.53 | 0.68 | 0 | 0 |
| PC46 | 0.28 | 0.91 | 0.05 | 0 | 0 | 1.65 | 0 | 0.45 |
| PC15 | 0.26 | 0.87 | 0.14 | 0 | 0 | 0.59 | 0 | 1.36 |
| PC26 | 0.25 | 0.16 | 0.29 | 0 | 0.59 | 0 | 0 | 0 |
| PC35 | 0.23 | 0.02 | 0.19 | 0 | 0.43 | 0.7 | 0 | 0 |
| PC37 | 0.13 | 0.09 | 0.13 | 0 | 0.03 | 0.22 | 0 | 0 |
| PC10 | 0.12 | 0.4 | 0 | 0 | 0 | 0.99 | 0 | 0 |
| PC25 | 0.08 | 0.29 | 0.08 | 0 | 0 | 0.03 | 0 | 0 |
| PC21 | 0.06 | 0.64 | 0.03 | 0 | 0 | 0.01 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components
are displayed.
| Variable | RC1 | RC2 | RC3 | RC5 | RC4 |
|:----------------------|:------|:------|:------|:------|:-----|
| SS loadings | 44.56 | 31.07 | 19.29 | 10.07 | 9.38 |
| Proportion Var | 0.22 | 0.15 | 0.10 | 0.05 | 0.05 |
| Cumulative Var | 0.22 | 0.38 | 0.47 | 0.52 | 0.57 |
| Proportion Explained | 0.39 | 0.27 | 0.17 | 0.09 | 0.08 |
| Cumulative Proportion | 0.39 | 0.66 | 0.83 | 0.92 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC5 | RC4 |
|:---------------------------------------|:------:|:------:|:------:|:------:|:------:|
| Sentences | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 |
| Words | 0.086 | 0.947 | 0.097 | 0.203 | 0.039 |
| Content.words | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 |
| RdbltyFlesch | -0.891 | -0.159 | -0.034 | -0.06 | 0.033 |
| RdbltyFog | 0.95 | 0.117 | 0.07 | 0.127 | -0.022 |
| RdbltyKincaid | 0.948 | 0.116 | 0.044 | 0.128 | -0.02 |
| RdbltyDaleChall | 0.4 | -0.276 | -0.098 | 0.136 | -0.503 |
| AvgBlLen | -0.041 | 0.903 | 0.095 | 0.055 | 0.156 |
| AvgCommaBl | -0.1 | 0.309 | 0.05 | -0.227 | 0.006 |
| AvgSenLen | 0.91 | 0.198 | 0.058 | 0.044 | 0.17 |
| AvgSenBl | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 |
| AvgUnqWdBl | -0.015 | 0.901 | 0.103 | 0.081 | 0.118 |
| AvgUnqWdSen | 0.922 | 0.161 | 0.089 | 0.096 | 0.134 |
| AvgWdLen | -0.089 | 0.407 | 0.562 | -0.271 | 0.362 |
| AvgWdBl | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 |
| AvgWdSen | 0.931 | 0.147 | 0.057 | 0.086 | 0.135 |
| CharEnt | -0.006 | 0.489 | 0.287 | -0.198 | 0.059 |
| SenStDevUnqWd | -0.429 | 0.328 | 0.099 | 0.075 | 0.453 |
| SenStdDevWd | -0.354 | 0.326 | 0.076 | 0.112 | 0.472 |
| WdEnt | 0.01 | 0.886 | 0.21 | 0.066 | 0.085 |
| WdLettStdDev | -0.158 | 0.401 | 0.362 | -0.257 | 0.224 |
| LxcDiv | 0.065 | 0.791 | 0.172 | 0.047 | 0.204 |
| LxcSoph | 0.334 | 0.207 | 0.6 | -0.068 | 0.407 |
| SynSoph | 0.825 | 0.348 | 0.092 | 0.118 | 0.259 |
| AvgNounBl | -0.017 | 0.705 | 0.149 | 0.361 | 0.031 |
| AvgPronounBl | 0.159 | 0.77 | 0.042 | 0.026 | -0.004 |
| AvgVerbBl | 0.102 | 0.911 | 0.04 | 0.051 | 0.058 |
| AvgAdverbBl | 0.043 | 0.697 | 0.012 | -0.126 | -0.103 |
| AvgAdjectiveBl | -0.011 | 0.517 | 0.115 | 0.028 | 0.031 |
| AvgPrepositionBl | 0.078 | 0.786 | 0.06 | 0.154 | -0.004 |
| AvgNounSen | 0.834 | -0.018 | 0.109 | 0.357 | -0.019 |
| AvgPronounSen | 0.928 | 0.03 | 0.053 | 0.017 | -0.023 |
| AvgVerbSen | 0.957 | 0.1 | 0.031 | 0.013 | 0.042 |
| AvgAdverbSen | 0.701 | 0.22 | -0.042 | -0.147 | -0.05 |
| AvgAdjectiveSen | 0.699 | 0.007 | 0.137 | 0.004 | -0.055 |
| AvgPrepositionSen | 0.803 | 0.212 | 0.018 | 0.139 | -0.013 |
| AvgUnqNoundBl | -0.014 | 0.632 | 0.13 | 0.431 | -0.023 |
| AvgUnqPronounBl | -0.007 | 0.533 | 0.088 | 0.074 | 0.058 |
| AvgUnqVerbBl | 0.122 | 0.858 | 0.047 | 0.038 | 0.043 |
| AvgUnqAdverbBl | -0.007 | 0.719 | 0.011 | -0.17 | -0.142 |
| AvgUnqAdjectiveBl | -0.025 | 0.523 | 0.104 | -0.022 | 0.022 |
| AvgUnqPrepositionBl | 0.054 | 0.813 | 0.061 | 0.075 | -0.009 |
| AvgPronBl_first_person | 0.168 | 0.637 | 0.017 | 0.058 | -0.022 |
| AvgPronBl_indefinite | 0.011 | 0.577 | 0.018 | 0.033 | 0.138 |
| AggPronSen_indefinite | 0.677 | 0.159 | -0.005 | 0.004 | 0.112 |
| AvgPronBl_third_person | 0.131 | 0.424 | 0.057 | -0.018 | 0.019 |
| AggPronSen_third_person | 0.782 | -0.078 | 0.043 | -0.033 | 0.007 |
| AvgSemDep | 0.967 | 0.07 | 0.088 | 0.181 | 0.001 |
| WdDiffLemmaStem | -0.016 | 0.289 | 0.104 | 0.007 | -0.106 |
| WdDiffWdStem | 0.041 | 0.483 | 0.031 | -0.313 | 0.132 |
| WdMaxDpthHypernymTree | -0.08 | 0.083 | 0.356 | 0.059 | 0.349 |
| WdAvgDpthHypernymTree | -0.064 | 0.073 | 0.369 | 0.063 | 0.349 |
| WdPathCntHypernymTree | -0.066 | 0.123 | 0.217 | -0.019 | 0.464 |
| WdPolysemyCnt | 0 | -0.045 | -0.037 | 0.192 | 0.244 |
| WdSylCnt | -0.126 | 0.215 | 0.592 | -0.117 | 0.108 |
| AvgAOADoc_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 |
| AvgAOABl_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 |
| AvgAOASen_Shock | 0.289 | 0.196 | 0.419 | 0.105 | 0.111 |
| AvgAOADoc_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 |
| AvgAOABl_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 |
| AvgAOASen_Cortese | 0.261 | 0.168 | 0.575 | 0.045 | 0.211 |
| AvgAOADoc_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 |
| AvgAOABl_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 |
| AvgAOASen_Kuperman | 0.1 | 0.18 | 0.742 | 0.068 | -0.023 |
| AvgAOADoc_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 |
| AvgAOABl_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 |
| AvgAOASen_Bird | 0.262 | 0.14 | 0.554 | 0.029 | 0.173 |
| AvgAOADoc_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 |
| AvgAOABl_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 |
| AvgAOASen_Bristol | 0.413 | 0.102 | 0.453 | 0.113 | 0.185 |
| AvgAOEDoc_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 |
| AvgAOEBl_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 |
| AvgAOESen_IndexPolyFitAbThr.0.3. | 0.006 | 0.133 | 0.617 | 0.152 | -0.495 |
| AvgAOEDoc_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 |
| AvgAOEBl_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 |
| AvgAOESen_InverseLinearRegressionSlope | 0.077 | 0.163 | 0.795 | 0.054 | -0.145 |
| AvgAOEDoc_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 |
| AvgAOEBl_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 |
| AvgAOESen_InflectionPointPolynomial | 0.064 | 0.198 | 0.8 | -0.021 | -0.076 |
| AvgAOEDoc_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 |
| AvgAOEBl_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 |
| AvgAOESen_InverseAverage | 0.06 | 0.191 | 0.808 | -0.021 | -0.097 |
| AvgAOEDoc_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 |
| AvgAOEBl_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 |
| AvgAOESen_IndexAboveThreshold.0.3. | -0.071 | 0.107 | 0.491 | 0.174 | -0.556 |
| AvgNmdEntBl | 0.057 | 0.52 | -0.052 | 0.139 | 0.032 |
| AvgNounNmdEntBl | 0.088 | 0.363 | -0.01 | 0.198 | -0.015 |
| AvgUnqNmdEntBl | 0.119 | 0.537 | -0.058 | 0.14 | -0.003 |
| AvgNmdEntSen | 0.752 | 0.106 | -0.084 | 0.167 | 0.06 |
| TCorefChainDoc | -0.03 | 0.621 | 0.189 | 0.003 | -0.062 |
| AvgCorefChain | 0.143 | 0.47 | 0.111 | 0.016 | 0.079 |
| AvgChainSpan | 0.088 | 0.729 | 0.038 | -0.003 | 0.128 |
| AvgInferenceDistChain | 0.245 | 0.306 | 0.046 | 0.001 | -0.012 |
| TActCorefChainWd | -0.092 | -0.329 | 0.268 | -0.225 | -0.102 |
| TCorefChainBigSpan | 0.108 | 0.426 | 0.243 | -0.098 | 0.002 |
| AvgConnBl_addition | 0.067 | 0.309 | 0.032 | 0.777 | 0.055 |
| AvgConnSen_addition | 0.658 | -0.166 | 0.072 | 0.622 | -0.077 |
| AvgConnBl_conjunctions | 0.168 | 0.362 | 0.061 | 0.775 | -0.017 |
| AvgConnSen_conjunctions | 0.72 | -0.147 | 0.097 | 0.579 | -0.152 |
| AvgConnBl_contrasts | 0.076 | 0.444 | 0.114 | 0.035 | -0.118 |
| AvgConnSen_contrasts | 0.512 | 0.196 | 0.125 | -0.059 | -0.149 |
| AvgConnBl_coordinating_conjuncts | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 |
| AvgConnSen_coordinating_conjuncts | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 |
| AvgConnBl_coordinating_connectives | 0.255 | 0.506 | 0.035 | 0.7 | 0.003 |
| AvgConnSen_coordinating_connectives | 0.818 | -0.034 | 0.071 | 0.48 | -0.119 |
| AvgConnBl_logical_connectors | 0.186 | 0.317 | 0.046 | 0.76 | -0.008 |
| AvgConnSen_logical_connectors | 0.693 | -0.154 | 0.069 | 0.563 | -0.121 |
| AvgConnBl_oppositions | 0.096 | 0.391 | 0.08 | 0.048 | -0.132 |
| AvgConnSen_oppositions | 0.494 | 0.14 | 0.106 | -0.083 | -0.187 |
| AvgConnBl_order | -0.115 | 0.255 | -0.013 | 0.188 | 0.176 |
| AvgConnSen_order | 0.315 | 0.084 | -0.013 | 0.115 | 0.215 |
| AvgConnBl_reason_and_purpose | 0.204 | 0.536 | -0.017 | 0.194 | 0.163 |
| AvgConnSen_reason_and_purpose | 0.71 | 0.226 | 0.002 | 0.034 | 0.064 |
| AvgConnBl_semi_coordinators | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 |
| AvgConnSen_semi_coordinators | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 |
| AvgConnBl_sentence_linking | 0.241 | 0.603 | 0.014 | 0.606 | 0.063 |
| AvgConnSen_sentence_linking | 0.862 | 0.027 | 0.048 | 0.411 | -0.059 |
| AvgConnBl_simple_subordinators | 0.081 | 0.52 | 0.062 | -0.082 | -0.029 |
| AvgConnSen_simple_subordinators | 0.681 | 0.165 | -0.053 | -0.039 | 0.023 |
| AvgConnBl_temporal_connectors | 0.121 | 0.382 | -0.081 | -0.07 | 0.114 |
| AvgConnSen_temporal_connectors | 0.637 | 0.153 | -0.183 | -0.036 | 0.106 |
| LexChainAvgSpan | 0.128 | 0.407 | 0.308 | 0.073 | 0.415 |
| LexChainMaxSp | -0.007 | 0.663 | 0.02 | 0.171 | 0.207 |
| AvgBlScore | 0.148 | 0.801 | 0.043 | 0.165 | 0.285 |
| AvgSenScore | 0.909 | 0.12 | 0.044 | 0.055 | 0.165 |
| SenScoreStDev | -0.466 | 0.332 | 0.071 | 0.09 | 0.501 |
| AvgIntraBlCoh_LeackockChodorow | -0.741 | 0.302 | 0.152 | 0.008 | 0.471 |
| AvgSenAdjCoh_LeackockChodorow | -0.735 | 0.263 | 0.166 | -0.002 | 0.488 |
| AvgSenBlCoh_LeackockChodorow | 0.434 | -0.139 | 0.696 | 0.053 | 0.24 |
| AvgIntraBlCoh_WuPalmer | -0.748 | 0.296 | 0.152 | 0.001 | 0.465 |
| AvgSenAdjCoh_WuPalmer | -0.743 | 0.259 | 0.167 | -0.01 | 0.482 |
| AvgSenBlCoh_WuPalmer | 0.443 | -0.145 | 0.694 | 0.049 | 0.226 |
| AvgIntraBlCoh_Path | -0.749 | 0.277 | 0.154 | -0.007 | 0.46 |
| AvgSenAdjCoh_Path | -0.744 | 0.234 | 0.175 | -0.021 | 0.478 |
| AvgSenBlCoh_Path | 0.528 | -0.219 | 0.642 | 0.044 | 0.155 |
| AvgIntraBlCoh_LSA | -0.739 | 0.338 | 0.148 | 0.005 | 0.446 |
| AvgSenAdjCoh_LSA | -0.729 | 0.295 | 0.166 | -0.011 | 0.473 |
| AvgSenBlCoh_LSA | 0.598 | -0.209 | 0.547 | 0.09 | 0.148 |
| AvgIntraBlCoh_LDA | -0.738 | 0.345 | 0.151 | -0.004 | 0.449 |
| AvgSenAdjCoh_LDA | -0.718 | 0.324 | 0.158 | -0.003 | 0.473 |
| AvgSenBlCoh_LDA | 0.513 | -0.066 | 0.598 | 0.08 | 0.202 |
| AvgIntraBlCoh_word2vec | -0.743 | 0.31 | 0.158 | -0.026 | 0.457 |
| AvgSenAdjCoh_word2vec | -0.734 | 0.273 | 0.179 | -0.046 | 0.48 |
| AvgSenBlCoh_word2vec | 0.581 | -0.259 | 0.577 | 0.048 | 0.165 |
| AvgBlVoiceCoOcc | 0.083 | 0.49 | -0.117 | 0.351 | 0.258 |
| AvgVoice | 0.086 | 0.506 | -0.117 | 0.355 | 0.247 |
| AvgSenSyll | 0.973 | 0.081 | 0.076 | 0.138 | -0.008 |
| AvgSenStressedSyll | 0.939 | 0.141 | 0.049 | 0.106 | 0.117 |
| AvgRhythmUnits | 0.315 | 0.184 | -0.003 | -0.262 | 0.043 |
| AvgRhythmUnitSyll | 0.807 | -0.027 | 0.082 | 0.284 | -0.045 |
| AvgRhythmUnitStreesSyll | 0.809 | 0.024 | 0.053 | 0.235 | 0.073 |
| LangRhythmCoeff | 0.24 | 0.048 | -0.011 | 0.129 | -0.121 |
| LangRhythmId | -0.173 | -0.023 | -0.06 | 0.028 | -0.387 |
| FrqRhythmId | 0.657 | -0.336 | -0.016 | 0.096 | 0.056 |
| LangRhythmDiameter | 0.373 | 0.113 | 0.117 | 0.197 | -0.139 |
| SenAsson | -0.051 | 0.221 | 0.079 | 0.066 | 0.065 |
| AvgDepsBl_acl | 0.092 | 0.23 | 0.074 | 0.206 | 0.093 |
| AvgDepsSen_acl | 0.485 | 0.081 | 0.031 | 0.343 | 0.108 |
| AvgDepsBl_advcl | 0.394 | 0.519 | -0.037 | 0.042 | 0.175 |
| AvgDepsSen_advcl | 0.768 | 0.219 | -0.041 | 0.009 | 0.065 |
| AvgDepsBl_advmod | 0.141 | 0.71 | -0.023 | -0.136 | -0.045 |
| AvgDepsSen_advmod | 0.753 | 0.193 | -0.054 | -0.132 | -0.006 |
| AvgDepsBl_amod | -0.092 | 0.476 | 0.104 | 0.038 | 0.13 |
| AvgDepsSen_amod | 0.563 | 0.072 | 0.117 | -0.001 | 0.112 |
| AvgDepsBl_aux | -0.02 | 0.402 | 0.09 | -0.282 | 0.005 |
| AvgDepsSen_aux | 0.621 | 0.072 | 0.112 | -0.18 | -0.071 |
| AvgDepsBl_auxpass | -0.079 | 0.424 | 0.001 | -0.052 | -0.166 |
| AvgDepsBl_case | -0.121 | 0.772 | 0.094 | 0.169 | -0.072 |
| AvgDepsSen_case | 0.747 | 0.175 | 0.093 | 0.178 | -0.024 |
| AvgDepsBl_cc | 0.203 | 0.37 | 0.063 | 0.746 | -0.039 |
| AvgDepsSen_cc | 0.73 | -0.143 | 0.088 | 0.553 | -0.149 |
| AvgDepsBl_ccomp | 0.383 | 0.496 | 0.036 | -0.022 | 0.173 |
| AvgDepsSen_ccomp | 0.781 | 0.073 | 0.031 | -0.041 | 0.09 |
| AvgDepsBl_compound | 0.104 | 0.149 | 0.038 | 0.366 | -0.006 |
| AvgDepsSen_compound | 0.587 | -0.062 | 0.073 | 0.412 | -0.064 |
| AvgDepsBl_conj | 0.31 | 0.297 | 0.122 | 0.722 | -0.045 |
| AvgDepsSen_conj | 0.731 | -0.134 | 0.134 | 0.536 | -0.162 |
| AvgDepsBl_cop | -0.001 | 0.419 | 0.055 | -0.12 | -0.002 |
| AvgDepsSen_cop | 0.583 | 0.061 | 0.018 | -0.107 | 0.003 |
| AvgDepsBl_dep | 0.234 | 0.123 | 0.104 | 0.47 | -0.059 |
| AvgDepsSen_dep | 0.527 | -0.187 | 0.15 | 0.494 | -0.174 |
| AvgDepsBl_det | -0.182 | 0.549 | 0.13 | 0.222 | 0.071 |
| AvgDepsSen_det | 0.576 | -0.06 | 0.152 | 0.415 | -0.023 |
| AvgDepsBl_dobj | 0.119 | 0.64 | 0.078 | 0.196 | 0.13 |
| AvgDepsSen_dobj | 0.856 | -0.002 | 0.047 | 0.128 | 0.028 |
| AvgDepsBl_mark | 0.367 | 0.535 | 0.01 | 0.087 | 0.139 |
| AvgDepsSen_mark | 0.795 | 0.152 | -0.063 | 0.085 | 0.073 |
| AvgDepsBl_mwe | -0.01 | 0.262 | 0.021 | 0.142 | 0.008 |
| AvgDepsSen_mwe | 0.383 | 0.097 | 0.025 | 0.044 | 0.031 |
| AvgDepsBl_neg | 0.197 | 0.361 | 0.053 | -0.181 | -0.132 |
| AvgDepsSen_neg | 0.549 | 0.073 | -0.024 | -0.17 | -0.086 |
| AvgDepsBl_nmod | -0.134 | 0.727 | 0.094 | 0.237 | -0.052 |
| AvgDepsSen_nmod | 0.661 | 0.15 | 0.086 | 0.281 | -0.02 |
| AvgDepsBl_nsubj | 0.141 | 0.88 | 0.062 | 0.041 | 0.08 |
| AvgDepsSen_nsubj | 0.969 | 0.02 | 0.051 | 0.018 | 0.003 |
| AvgDepsBl_nsubjpass | -0.015 | 0.417 | -0.005 | -0.065 | -0.158 |
| AvgDepsBl_nummod | 0.132 | 0.492 | -0.071 | -0.009 | 0.13 |
| AvgDepsBl_punct | -0.483 | 0.644 | -0.017 | -0.108 | -0.086 |
| AvgDepsSen_punct | -0.192 | 0.389 | 0.063 | -0.232 | 0.191 |
| AvgDepsBl_xcomp | 0.053 | 0.501 | 0.03 | 0.186 | 0.068 |
| AvgDepsSen_xcomp | 0.666 | 0.075 | 0.061 | 0.196 | 0.024 |
ReaderBench Model 1e
This model was trained on principal component scores for winter data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------|
| -8.0185 | 0.0573 | 0.5839 | 0.5269 | -0.3984 | 0.1184 | 0.1066 | 0.0459 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:-----|:-----|:------|:----|:-----|:------|:------|:------|
| PC2 | 32.8 | 53 | 47.34 | 0 | 9.95 | 26.16 | 39.14 | 22.22 |
| PC1 | 7.63 | 1.98 | 13.73 | 0 | 1.84 | 4.83 | 0 | 11.11 |
| PC39 | 4.81 | 1.96 | 1.08 | 0 | 7.68 | 2.76 | 17.37 | 8.89 |
| PC5 | 4.35 | 2.14 | 4.78 | 0 | 4.24 | 3.9 | 0 | 13.33 |
| PC11 | 4.33 | 1.43 | 3.55 | 0 | 5.85 | 1.52 | 4.87 | 11.11 |
| PC37 | 4.21 | 1.94 | 1.15 | 0 | 7.47 | 1.75 | 12.15 | 6.67 |
| PC6 | 3.45 | 3.05 | 3.87 | 0 | 3.53 | 2.17 | 0 | 8.89 |
| PC26 | 3.28 | 2.41 | 1.25 | 0 | 4.41 | 1.69 | 14.49 | 0 |
| PC38 | 3.13 | 0.57 | 1.08 | 0 | 7.55 | 0.99 | 0 | 6.67 |
| PC14 | 2.89 | 4.55 | 1.94 | 0 | 3.33 | 6.23 | 0 | 6.67 |
| PC24 | 2.2 | 1.36 | 1.07 | 0 | 3.24 | 0.49 | 8.2 | 0 |
| PC40 | 2.12 | 0.85 | 0.72 | 0 | 5.11 | 1.28 | 0 | 2.22 |
| PC9 | 1.75 | 0.72 | 2.06 | 0 | 2.21 | 0.62 | 0 | 2.22 |
| PC33 | 1.63 | 0.91 | 0.67 | 0 | 2.86 | 2.41 | 2.5 | 0 |
| PC45 | 1.61 | 0.26 | 0.5 | 0 | 4.33 | 0.63 | 0 | 0 |
| PC44 | 1.51 | 0.65 | 0.49 | 0 | 3.6 | 1.94 | 0 | 0 |
| PC32 | 1.3 | 0.87 | 0.67 | 0 | 2.61 | 1.92 | 0 | 0 |
| PC20 | 1.29 | 1.94 | 1 | 0 | 2.28 | 0.76 | 0 | 0 |
| PC4 | 1.16 | 1.23 | 1.61 | 0 | 0.82 | 1.5 | 0 | 0 |
| PC19 | 1.07 | 1.79 | 0.78 | 0 | 1.41 | 2.19 | 0 | 0 |
| PC34 | 0.98 | 1.11 | 0.5 | 0 | 1.9 | 1.2 | 0.26 | 0 |
| PC43 | 0.97 | 0.27 | 0.42 | 0 | 2.53 | 0 | 0 | 0 |
| PC8 | 0.95 | 0.61 | 1.17 | 0 | 0.8 | 1.73 | 0 | 0 |
| PC15 | 0.94 | 1.05 | 0.88 | 0 | 1.21 | 1.47 | 0 | 0 |
| PC23 | 0.9 | 0.73 | 0.64 | 0 | 1.33 | 1.89 | 0 | 0 |
| PC17 | 0.9 | 0.77 | 0.85 | 0 | 1.41 | 0.6 | 0 | 0 |
| PC35 | 0.86 | 1.17 | 0.46 | 0 | 1.61 | 1.24 | 0 | 0 |
| PC3 | 0.81 | 0.64 | 1.28 | 0 | 0.16 | 1.75 | 0 | 0 |
| PC12 | 0.76 | 0.42 | 0.78 | 0 | 0.71 | 1.86 | 0 | 0 |
| PC28 | 0.73 | 0.92 | 0.48 | 0 | 1.01 | 1.9 | 0 | 0 |
| PC16 | 0.69 | 0.41 | 0.71 | 0 | 0.86 | 0.92 | 0 | 0 |
| PC25 | 0.59 | 0.21 | 0.47 | 0 | 0.73 | 1.59 | 0 | 0 |
| PC30 | 0.42 | 1.15 | 0.31 | 0 | 0.25 | 1.74 | 0 | 0 |
| PC42 | 0.41 | 0.61 | 0.22 | 0 | 0.34 | 1.92 | 0 | 0 |
| PC36 | 0.36 | 0.14 | 0.29 | 0 | 0.51 | 0.88 | 0 | 0 |
| PC27 | 0.35 | 0.38 | 0.35 | 0 | 0.34 | 0.77 | 0 | 0 |
| PC7 | 0.34 | 1.67 | 0.06 | 0 | 0 | 1.66 | 1.01 | 0 |
| PC13 | 0.28 | 0.89 | 0.01 | 0 | 0 | 2.56 | 0 | 0 |
| PC29 | 0.28 | 0.97 | 0.05 | 0 | 0 | 2.36 | 0 | 0 |
| PC10 | 0.27 | 0.54 | 0.27 | 0 | 0 | 1.27 | 0 | 0 |
| PC18 | 0.24 | 0.52 | 0.14 | 0 | 0 | 1.68 | 0 | 0 |
| PC31 | 0.18 | 0.35 | 0.08 | 0 | 0 | 1.51 | 0 | 0 |
| PC46 | 0.11 | 0.43 | 0.15 | 0 | 0 | 0.24 | 0 | 0 |
| PC21 | 0.07 | 0.18 | 0 | 0 | 0 | 0.64 | 0 | 0 |
| PC22 | 0.07 | 0.25 | 0.09 | 0 | 0 | 0.27 | 0 | 0 |
| PC41 | 0.06 | 0 | 0 | 0 | 0 | 0.6 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components
are displayed.
| Variable | RC1 | RC2 | RC3 | RC4 | RC5 |
|:----------------------|:------|:------|:------|:-----|:-----|
| SS loadings | 46.99 | 34.24 | 14.94 | 7.95 | 6.95 |
| Proportion Var | 0.23 | 0.17 | 0.07 | 0.04 | 0.03 |
| Cumulative Var | 0.23 | 0.40 | 0.48 | 0.52 | 0.55 |
| Proportion Explained | 0.42 | 0.31 | 0.13 | 0.07 | 0.06 |
| Cumulative Proportion | 0.42 | 0.73 | 0.87 | 0.94 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC4 | RC5 |
|:--------------------------|:------|:------|:------|:------|:------|
| Sentences | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 |
| Words | 0.07 | 0.97 | 0 | -0.08 | -0.05 |
| Content.words | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 |
| RdbltyFlesch | -0.82 | -0.19 | -0.02 | -0.12 | 0.09 |
| RdbltyFog | 0.91 | 0.17 | 0.01 | 0.04 | 0.01 |
| RdbltyKincaid | 0.91 | 0.18 | 0.01 | 0.05 | 0 |
| RdbltyDaleChall | 0.5 | -0.34 | 0.11 | -0.06 | -0.22 |
| AvgBlLen | -0.11 | 0.91 | 0.08 | 0.13 | -0.16 |
| AvgCommaBl | -0.23 | 0.38 | 0.06 | -0.1 | -0.1 |
| AvgSenLen | 0.9 | 0.26 | 0.08 | 0.15 | 0.03 |
| AvgSenBl | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 |
| AvgUnqWdBl | -0.08 | 0.91 | 0.05 | 0.12 | -0.15 |
| AvgUnqWdSen | 0.93 | 0.22 | 0.08 | 0.12 | 0.05 |
| AvgWdLen | -0.21 | 0.26 | -0.17 | 0.25 | -0.46 |
| AvgWdBl | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 |
| AvgWdSen | 0.93 | 0.23 | 0.07 | 0.09 | 0.06 |
| CharEnt | -0.14 | 0.54 | 0.11 | -0.02 | -0.03 |
| SenStDevUnqWd | -0.23 | 0.5 | 0.04 | 0.09 | 0.41 |
| SenStdDevWd | -0.16 | 0.48 | 0.04 | 0.07 | 0.45 |
| WdEnt | 0.03 | 0.86 | 0.04 | 0.1 | -0.13 |
| WdLettStdDev | -0.09 | 0.4 | 0.37 | 0.24 | 0.04 |
| LxcDiv | 0 | 0.81 | 0.09 | 0.2 | -0.07 |
| LxcSoph | 0.36 | 0.09 | -0.28 | 0.11 | -0.35 |
| SynSoph | 0.82 | 0.44 | 0.12 | 0.14 | 0.12 |
| AvgNounBl | 0.01 | 0.76 | 0.11 | 0.03 | -0.35 |
| AvgPronounBl | 0.1 | 0.79 | -0.1 | -0.13 | 0.18 |
| AvgVerbBl | 0.03 | 0.91 | -0.05 | -0.02 | 0.05 |
| AvgAdverbBl | 0.09 | 0.63 | -0.01 | -0.22 | 0.11 |
| AvgAdjectiveBl | -0.06 | 0.55 | 0.12 | -0.07 | -0.09 |
| AvgPrepositionBl | 0.11 | 0.75 | -0.04 | 0.27 | -0.1 |
| AvgNounSen | 0.9 | 0.06 | 0.06 | 0.06 | -0.14 |
| AvgPronounSen | 0.91 | 0.1 | -0.08 | -0.02 | 0.15 |
| AvgVerbSen | 0.94 | 0.16 | -0.01 | 0.04 | 0.09 |
| AvgAdverbSen | 0.77 | 0.15 | -0.02 | -0.13 | 0.18 |
| AvgAdjectiveSen | 0.71 | 0.11 | 0.13 | -0.13 | 0.1 |
| AvgPrepositionSen | 0.85 | 0.2 | -0.07 | 0.26 | -0.04 |
| AvgUnqNoundBl | 0.02 | 0.71 | 0.05 | 0.02 | -0.36 |
| AvgUnqPronounBl | 0.09 | 0.66 | -0.01 | -0.09 | 0.02 |
| AvgUnqVerbBl | 0.03 | 0.85 | -0.05 | 0.01 | 0.04 |
| AvgUnqAdverbBl | -0.01 | 0.61 | -0.02 | -0.13 | 0.05 |
| AvgUnqAdjectiveBl | -0.06 | 0.57 | 0.12 | -0.09 | -0.13 |
| AvgUnqPrepositionBl | 0.09 | 0.71 | 0 | 0.3 | -0.14 |
| AvgPronBl_first_person | 0.09 | 0.69 | -0.05 | -0.14 | 0.12 |
| AvgPronBl_indefinite | -0.05 | 0.45 | 0.01 | 0.24 | -0.04 |
| AggPronSen_indefinite | 0.62 | 0.17 | 0.05 | 0.23 | 0.02 |
| AvgPronBl_third_person | 0.13 | 0.48 | -0.08 | -0.12 | 0.14 |
| AggPronSen_third_person | 0.84 | -0.02 | -0.09 | -0.09 | 0.12 |
| AvgSemDep | 0.97 | 0.12 | -0.01 | -0.06 | 0.06 |
| WdDiffLemmaStem | -0.11 | 0.06 | -0.23 | -0.01 | -0.24 |
| WdDiffWdStem | -0.28 | 0.19 | 0.03 | 0.16 | -0.28 |
| WdMaxDpthHypernymTree | -0.02 | -0.26 | -0.09 | 0.14 | -0.23 |
| WdAvgDpthHypernymTree | 0 | -0.27 | -0.11 | 0.13 | -0.24 |
| WdPathCntHypernymTree | -0.1 | -0.26 | -0.17 | 0.16 | -0.09 |
| WdPolysemyCnt | 0.06 | 0.11 | -0.09 | -0.09 | 0.35 |
| WdSylCnt | -0.1 | 0.05 | -0.26 | 0.11 | -0.46 |
| AvgAOADoc_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 |
| AvgAOABl_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 |
| AvgAOASen_Shock | 0.46 | 0.21 | 0.05 | 0.31 | -0.27 |
| AvgAOADoc_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 |
| AvgAOABl_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 |
| AvgAOASen_Cortese | 0.14 | 0.01 | 0.55 | 0.3 | 0.17 |
| AvgAOADoc_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 |
| AvgAOABl_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 |
| AvgAOASen_Kuperman | -0.03 | -0.04 | 0.42 | 0.41 | -0.3 |
| AvgAOADoc_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 |
| AvgAOABl_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 |
| AvgAOASen_Bird | 0.23 | 0.12 | 0.43 | 0.36 | 0.21 |
| AvgAOADoc_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 |
| AvgAOABl_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 |
| AvgAOASen_Bristol | 0.37 | 0.08 | 0.34 | 0.26 | 0 |
| AvgAOEDoc_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 |
| AvgAOEBl_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 |
| AvgAOESen_IndexPolyFAT.3 | 0.02 | -0.02 | 0.74 | -0.12 | -0.17 |
| AvgAOEDoc_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 |
| AvgAOEBl_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 |
| AvgAOESen_InvLinRegSlo | 0.17 | -0.02 | 0.67 | -0.02 | -0.11 |
| AvgAOEDoc_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 |
| AvgAOEBl_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 |
| AvgAOESen_InfPointPoly | 0.07 | 0.03 | 0.73 | 0.02 | 0.05 |
| AvgAOEDoc_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 |
| AvgAOEBl_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 |
| AvgAOESen_InvAverage | 0.06 | 0.02 | 0.74 | 0.03 | 0.02 |
| AvgAOEDoc_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 |
| AvgAOEBl_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 |
| AvgAOESen_IndexAbThr.0.3. | 0.05 | 0 | 0.75 | -0.04 | -0.27 |
| AvgNmdEntBl | -0.07 | 0.32 | 0.15 | -0.11 | -0.44 |
| AvgNounNmdEntBl | -0.02 | 0.24 | 0.24 | -0.13 | -0.43 |
| AvgUnqNmdEntBl | -0.07 | 0.34 | 0.12 | -0.09 | -0.48 |
| AvgNmdEntSen | 0.59 | 0.01 | 0.19 | -0.09 | -0.26 |
| TCorefChainDoc | 0.04 | 0.69 | 0.06 | -0.2 | -0.01 |
| AvgCorefChain | 0.02 | 0.46 | -0.18 | 0 | 0.11 |
| AvgChainSpan | 0.06 | 0.73 | -0.03 | 0.07 | -0.1 |
| AvgInferenceDistChain | 0.45 | 0.31 | -0.02 | 0.31 | 0.07 |
| TActCorefChainWd | -0.02 | -0.25 | -0.04 | -0.19 | 0.04 |
| TCorefChainBigSpan | 0.19 | 0.52 | -0.02 | -0.09 | -0.01 |
| AvgConnBl_addition | 0.1 | 0.53 | 0.1 | -0.58 | -0.08 |
| AvgConnSen_addition | 0.74 | 0.04 | 0.05 | -0.4 | -0.02 |
| AvgConnBl_conjunctions | 0.15 | 0.59 | 0.09 | -0.57 | 0.06 |
| AvgConnSen_conjunctions | 0.83 | 0.04 | 0.06 | -0.36 | 0.06 |
| AvgConnBl_contrasts | 0.19 | 0.37 | 0 | -0.11 | 0.28 |
| AvgConnSen_contrasts | 0.63 | 0.11 | 0.04 | -0.11 | 0.22 |
| AvgConnBl_coord_conjs | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 |
| AvgConnSen_coord_conjs | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 |
| AvgConnBl_coord_connects | 0.21 | 0.7 | -0.02 | -0.4 | 0.18 |
| AvgConnSen_coord_connects | 0.88 | 0.1 | -0.02 | -0.23 | 0.17 |
| AvgConnBl_logical_conn | 0.09 | 0.5 | 0.07 | -0.64 | 0.05 |
| AvgConnSen_logical_conn | 0.67 | -0.02 | 0.03 | -0.5 | 0.1 |
| AvgConnBl_oppositions | 0.19 | 0.37 | 0.05 | 0.02 | 0.22 |
| AvgConnSen_oppositions | 0.6 | 0.16 | 0.07 | 0.12 | 0.11 |
| AvgConnBl_order | 0.12 | 0.38 | 0.05 | -0.29 | -0.16 |
| AvgConnSen_order | 0.57 | 0.14 | 0.02 | -0.07 | -0.1 |
| AvgConnBl_reas_purp | 0.18 | 0.58 | -0.07 | -0.03 | 0.09 |
| AvgConnSen_reas_purp | 0.73 | 0.25 | -0.06 | 0.08 | 0.09 |
| AvgConnBl_semi_coords | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 |
| AvgConnSen_semi_coords | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 |
| AvgConnBl_sentence_link | 0.2 | 0.77 | -0.01 | -0.33 | 0.13 |
| AvgConnSen_sentence_link | 0.92 | 0.14 | -0.03 | -0.13 | 0.12 |
| AvgConnBl_simp_subords | -0.02 | 0.41 | -0.05 | 0.27 | 0.11 |
| AvgConnSen_simp_subords | 0.52 | 0.18 | -0.09 | 0.34 | 0.16 |
| AvgConnBl_temp_conn | -0.14 | 0.36 | 0.05 | -0.23 | 0.08 |
| AvgConnSen_temp_conn | 0.36 | 0.06 | 0.04 | -0.25 | 0.27 |
| LexChainAvgSpan | 0.07 | 0.49 | 0.03 | -0.01 | 0.16 |
| LexChainMaxSp | 0 | 0.72 | 0 | -0.02 | 0.08 |
| AvgBlScore | 0.14 | 0.82 | 0.01 | 0 | 0.13 |
| AvgSenScore | 0.9 | 0.22 | 0.02 | 0.04 | 0.14 |
| SenScoreStDev | -0.27 | 0.5 | 0.04 | 0.05 | 0.45 |
| AvgIntraBlCoh_LeackChod | -0.73 | 0.44 | 0.16 | 0.2 | 0.17 |
| AvgSenAdjCoh_LeackChod | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 |
| AvgSenBlCoh_LeackChod | 0.77 | -0.38 | -0.13 | -0.08 | 0.09 |
| AvgIntraBlCoh_WuPalmer | -0.74 | 0.44 | 0.16 | 0.2 | 0.18 |
| AvgSenAdjCoh_WuPalmer | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 |
| AvgSenBlCoh_WuPalmer | 0.79 | -0.39 | -0.13 | -0.09 | 0.08 |
| AvgIntraBlCoh_Path | -0.73 | 0.43 | 0.16 | 0.2 | 0.19 |
| AvgSenAdjCoh_Path | -0.7 | 0.42 | 0.19 | 0.22 | 0.2 |
| AvgSenBlCoh_Path | 0.82 | -0.42 | -0.12 | -0.13 | 0.08 |
| AvgIntraBlCoh_LSA | -0.73 | 0.46 | 0.16 | 0.18 | 0.18 |
| AvgSenAdjCoh_LSA | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 |
| AvgSenBlCoh_LSA | 0.8 | -0.35 | -0.11 | -0.12 | 0.11 |
| AvgIntraBlCoh_LDA | -0.73 | 0.48 | 0.16 | 0.18 | 0.15 |
| AvgSenAdjCoh_LDA | -0.7 | 0.47 | 0.2 | 0.2 | 0.15 |
| AvgSenBlCoh_LDA | 0.75 | -0.21 | -0.13 | -0.11 | 0.08 |
| AvgIntraBlCoh_word2vec | -0.73 | 0.45 | 0.16 | 0.19 | 0.18 |
| AvgSenAdjCoh_word2vec | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 |
| AvgSenBlCoh_word2vec | 0.8 | -0.39 | -0.1 | -0.11 | 0.11 |
| AvgBlVoiceCoOcc | 0.03 | 0.63 | -0.05 | -0.04 | -0.01 |
| AvgVoice | -0.01 | 0.6 | -0.08 | -0.05 | 0 |
| AvgSenSyll | 0.98 | 0.12 | 0.01 | 0.02 | 0.02 |
| AvgSenStressedSyll | 0.94 | 0.22 | 0.07 | 0.08 | 0.05 |
| AvgRhythmUnits | 0.23 | 0.26 | 0.15 | 0.01 | 0.28 |
| AvgRhythmUnitSyll | 0.84 | 0.01 | -0.09 | 0.02 | -0.09 |
| AvgRhythmUnitStreesSyll | 0.81 | 0.11 | -0.01 | 0.08 | -0.05 |
| LangRhythmCoeff | 0.39 | -0.1 | -0.21 | -0.17 | -0.06 |
| LangRhythmId | -0.11 | -0.02 | -0.4 | -0.27 | -0.14 |
| FrqRhythmId | 0.72 | -0.3 | 0.01 | -0.12 | 0.04 |
| LangRhythmDiameter | 0.27 | -0.01 | -0.34 | -0.25 | -0.19 |
| SenAsson | 0.13 | 0.25 | 0.12 | -0.11 | -0.17 |
| AvgDepsBl_acl | -0.03 | 0.24 | 0.15 | 0.19 | -0.16 |
| AvgDepsSen_acl | 0.44 | 0.03 | 0.26 | 0.09 | -0.13 |
| AvgDepsBl_advcl | 0.23 | 0.52 | -0.05 | 0.1 | 0.16 |
| AvgDepsSen_advcl | 0.67 | 0.21 | -0.08 | 0.2 | 0.09 |
| AvgDepsBl_advmod | 0.07 | 0.67 | 0.03 | -0.25 | 0.13 |
| AvgDepsSen_advmod | 0.78 | 0.15 | 0.02 | -0.17 | 0.2 |
| AvgDepsBl_amod | -0.07 | 0.39 | 0.16 | 0.06 | -0.2 |
| AvgDepsSen_amod | 0.62 | 0.1 | 0.17 | 0 | -0.01 |
| AvgDepsBl_aux | 0.13 | 0.37 | -0.13 | 0.13 | 0.24 |
| AvgDepsSen_aux | 0.56 | 0.02 | -0.14 | -0.01 | 0.27 |
| AvgDepsBl_auxpass | 0.06 | 0.26 | 0.08 | 0.05 | 0.03 |
| AvgDepsBl_case | 0.06 | 0.69 | 0.01 | 0.15 | -0.26 |
| AvgDepsSen_case | 0.84 | 0.13 | -0.02 | 0.15 | -0.17 |
| AvgDepsBl_cc | 0.17 | 0.59 | 0.08 | -0.61 | 0.09 |
| AvgDepsSen_cc | 0.8 | 0.02 | 0.05 | -0.42 | 0.11 |
| AvgDepsBl_ccomp | 0.37 | 0.51 | -0.03 | -0.01 | 0.32 |
| AvgDepsSen_ccomp | 0.78 | 0.16 | -0.01 | -0.01 | 0.19 |
| AvgDepsBl_compound | 0.17 | 0.16 | 0.19 | -0.05 | -0.36 |
| AvgDepsSen_compound | 0.61 | -0.12 | 0.18 | -0.07 | -0.27 |
| AvgDepsBl_conj | 0.23 | 0.52 | 0.06 | -0.58 | 0.14 |
| AvgDepsSen_conj | 0.77 | 0.02 | 0.04 | -0.4 | 0.13 |
| AvgDepsBl_cop | 0.04 | 0.49 | 0.09 | 0.03 | -0.1 |
| AvgDepsSen_cop | 0.71 | 0.16 | 0.14 | 0.04 | -0.03 |
| AvgDepsBl_dep | 0.2 | 0.22 | 0.08 | -0.34 | 0.07 |
| AvgDepsSen_dep | 0.56 | -0.05 | 0.1 | -0.31 | 0.11 |
| AvgDepsBl_det | -0.09 | 0.58 | -0.03 | 0.05 | -0.28 |
| AvgDepsSen_det | 0.86 | 0.1 | -0.02 | 0.04 | -0.16 |
| AvgDepsBl_dobj | 0 | 0.73 | 0.03 | -0.2 | 0.01 |
| AvgDepsSen_dobj | 0.91 | 0.07 | -0.01 | -0.04 | 0.1 |
| AvgDepsBl_mark | 0.22 | 0.63 | -0.11 | 0.28 | 0.09 |
| AvgDepsSen_mark | 0.78 | 0.25 | -0.1 | 0.28 | 0.05 |
| AvgDepsBl_mwe | -0.07 | 0.19 | 0.06 | 0 | -0.02 |
| AvgDepsSen_mwe | 0.3 | 0 | 0.03 | -0.09 | 0.05 |
| AvgDepsBl_neg | 0.03 | 0.37 | -0.08 | -0.03 | 0.25 |
| AvgDepsSen_neg | 0.38 | 0.12 | -0.04 | 0.05 | 0.28 |
| AvgDepsBl_nmod | 0.05 | 0.64 | -0.01 | 0.17 | -0.27 |
| AvgDepsSen_nmod | 0.83 | 0.07 | -0.06 | 0.14 | -0.18 |
| AvgDepsBl_nsubj | 0.03 | 0.9 | -0.1 | -0.1 | 0.08 |
| AvgDepsSen_nsubj | 0.94 | 0.14 | -0.07 | 0.01 | 0.14 |
| AvgDepsBl_nsubjpass | 0.05 | 0.2 | 0.03 | 0.08 | 0.16 |
| AvgDepsBl_nummod | -0.09 | 0.17 | -0.05 | 0.04 | -0.2 |
| AvgDepsBl_punct | -0.53 | 0.52 | -0.01 | -0.06 | -0.17 |
| AvgDepsSen_punct | -0.13 | 0.35 | 0.14 | 0.12 | 0.26 |
| AvgDepsBl_xcomp | 0.08 | 0.49 | -0.03 | 0.01 | -0.11 |
| AvgDepsSen_xcomp | 0.63 | 0.15 | 0.02 | -0.06 | -0.1 |
ReaderBench Model 1f
This model was trained on principal component scores for spring data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:--------|:-------|:-------|
| -9.2262 | 0.1219 | 0.7713 | 0.1603 | -0.3706 | -0.0217 | 0.3129 | 0.0233 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:------|:------|:------|:----|:------|:------|:------|:------|
| PC2 | 31.31 | 56.99 | 35.09 | 0 | 11.67 | 23.86 | 35.8 | 21.83 |
| PC1 | 11.79 | 6.72 | 17.18 | 0 | 3.69 | 5.61 | 8.89 | 21.83 |
| PC4 | 10.38 | 7.69 | 9.11 | 0 | 8.29 | 5.24 | 16.85 | 12.66 |
| PC44 | 4.45 | 0.7 | 0.97 | 0 | 7.79 | 1.41 | 10.78 | 3.71 |
| PC9 | 4.36 | 2.62 | 4.85 | 0 | 7.31 | 1.81 | 0 | 10.92 |
| PC27 | 3.4 | 0.68 | 0.79 | 0 | 1.88 | 1.75 | 12.84 | 0 |
| PC14 | 3.37 | 1.1 | 3.45 | 0 | 6.67 | 1.95 | 0 | 7.21 |
| PC43 | 2.78 | 1.23 | 1.19 | 0 | 9.29 | 1.77 | 0 | 0 |
| PC6 | 2.51 | 0.56 | 3.24 | 0 | 3.33 | 1.2 | 0.08 | 9.17 |
| PC16 | 2.38 | 0.82 | 1.17 | 0 | 1.74 | 1.2 | 6.85 | 0 |
| PC15 | 2.31 | 1.41 | 2.25 | 0 | 4.31 | 1.89 | 0 | 10.92 |
| PC5 | 2.22 | 0.82 | 2.97 | 0 | 2.79 | 2.34 | 0.2 | 1.75 |
| PC28 | 1.47 | 0.78 | 1.17 | 0 | 3.69 | 1.94 | 0 | 0 |
| PC10 | 1.46 | 1.85 | 1.73 | 0 | 2.08 | 1.98 | 0 | 0 |
| PC11 | 1.33 | 1.77 | 1.55 | 0 | 1.91 | 2.26 | 0 | 0 |
| PC37 | 1.29 | 0.57 | 0.8 | 0 | 3.79 | 1.57 | 0 | 0 |
| PC12 | 1.19 | 0.4 | 1.5 | 0 | 1.87 | 2.16 | 0 | 0 |
| PC34 | 1.08 | 0.19 | 0.77 | 0 | 3.06 | 0.68 | 0 | 0 |
| PC31 | 1.05 | 0.52 | 0.79 | 0 | 2.74 | 2.09 | 0 | 0 |
| PC30 | 0.9 | 0.05 | 0.21 | 0 | 0 | 0.29 | 4.02 | 0 |
| PC19 | 0.84 | 1.4 | 0.92 | 0 | 1.29 | 0.05 | 0 | 0 |
| PC23 | 0.8 | 0.38 | 0.83 | 0 | 1.59 | 1.37 | 0 | 0 |
| PC3 | 0.77 | 0.92 | 0.36 | 0 | 0 | 2.84 | 2.68 | 0 |
| PC40 | 0.75 | 0.22 | 0.5 | 0 | 2.14 | 1.16 | 0 | 0 |
| PC21 | 0.74 | 0.02 | 0.7 | 0 | 0.87 | 1.03 | 1 | 0 |
| PC45 | 0.73 | 0.12 | 0.43 | 0 | 2.23 | 1.46 | 0 | 0 |
| PC25 | 0.65 | 0.57 | 0.67 | 0 | 1.22 | 1.65 | 0 | 0 |
| PC18 | 0.43 | 0.14 | 0.62 | 0 | 0.45 | 2.08 | 0 | 0 |
| PC20 | 0.38 | 0.49 | 0.55 | 0 | 0.3 | 1.36 | 0 | 0 |
| PC38 | 0.37 | 0.35 | 0.35 | 0 | 0.72 | 1.69 | 0 | 0 |
| PC35 | 0.37 | 0.16 | 0.4 | 0 | 0.7 | 0 | 0 | 0 |
| PC32 | 0.27 | 0.8 | 0.34 | 0 | 0.17 | 1.33 | 0 | 0 |
| PC17 | 0.24 | 0.08 | 0.47 | 0 | 0 | 1.41 | 0 | 0 |
| PC39 | 0.23 | 0.46 | 0.29 | 0 | 0.25 | 1.31 | 0 | 0 |
| PC13 | 0.23 | 1.06 | 0.32 | 0 | 0 | 2.26 | 0 | 0 |
| PC22 | 0.21 | 0.42 | 0.36 | 0 | 0 | 1.86 | 0 | 0 |
| PC41 | 0.2 | 0.42 | 0.27 | 0 | 0.16 | 1.77 | 0 | 0 |
| PC7 | 0.18 | 0.41 | 0.3 | 0 | 0 | 1.37 | 0 | 0 |
| PC26 | 0.17 | 1.46 | 0.12 | 0 | 0 | 2.05 | 0 | 0 |
| PC24 | 0.12 | 1.74 | 0 | 0 | 0 | 1.36 | 0 | 0 |
| PC42 | 0.1 | 0.32 | 0.15 | 0 | 0 | 1.77 | 0 | 0 |
| PC8 | 0.07 | 0.36 | 0.06 | 0 | 0 | 1.67 | 0 | 0 |
| PC33 | 0.06 | 0.12 | 0.08 | 0 | 0 | 1.34 | 0 | 0 |
| PC36 | 0.06 | 0 | 0.11 | 0 | 0 | 0.72 | 0 | 0 |
| PC29 | 0.02 | 0.13 | 0.02 | 0 | 0 | 2.08 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components
are displayed.
| RC1 | RC2 | RC3 | RC4 | RC5 | |
|:----------------------|:------|:------|:------|:------|:-----|
| SS loadings | 49.09 | 28.90 | 15.04 | 12.78 | 8.00 |
| Proportion Var | 0.24 | 0.14 | 0.07 | 0.06 | 0.04 |
| Cumulative Var | 0.24 | 0.39 | 0.46 | 0.53 | 0.57 |
| Proportion Explained | 0.43 | 0.25 | 0.13 | 0.11 | 0.07 |
| Cumulative Proportion | 0.43 | 0.69 | 0.82 | 0.93 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC4 | RC5 |
|:--------------------------|:------|:------|:------|:------|:------|
| Sentences | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 |
| Words | 0.13 | 0.87 | -0.07 | 0.37 | 0.19 |
| Content.words | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 |
| RdbltyFlesch | -0.83 | -0.11 | -0.08 | 0.04 | 0.04 |
| RdbltyFog | 0.9 | 0.08 | 0.01 | -0.01 | 0 |
| RdbltyKincaid | 0.89 | 0.07 | 0.02 | 0 | -0.01 |
| RdbltyDaleChall | 0.47 | -0.12 | 0.41 | -0.23 | 0.28 |
| AvgBlLen | -0.01 | 0.88 | 0.03 | 0.29 | -0.02 |
| AvgCommaBl | -0.18 | 0.28 | -0.06 | -0.18 | 0.05 |
| AvgSenLen | 0.94 | 0.15 | 0.04 | 0 | -0.04 |
| AvgSenBl | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 |
| AvgUnqWdBl | 0.02 | 0.89 | 0.06 | 0.22 | 0.02 |
| AvgUnqWdSen | 0.95 | 0.12 | 0.03 | 0.01 | -0.04 |
| AvgWdLen | -0.12 | 0.48 | 0.04 | -0.15 | -0.47 |
| AvgWdBl | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 |
| AvgWdSen | 0.95 | 0.1 | 0.02 | 0.01 | 0.03 |
| CharEnt | -0.18 | 0.59 | -0.07 | -0.04 | 0.21 |
| SenStDevUnqWd | -0.35 | 0.23 | 0.05 | 0.67 | 0.07 |
| SenStdDevWd | -0.29 | 0.21 | 0.03 | 0.67 | 0.08 |
| WdEnt | -0.01 | 0.88 | 0 | 0.12 | 0.16 |
| WdLettStdDev | -0.04 | 0.56 | 0.08 | 0.02 | -0.16 |
| LxcDiv | 0.08 | 0.81 | 0.05 | 0.16 | -0.06 |
| LxcSoph | 0.56 | 0.08 | 0.09 | -0.08 | -0.48 |
| SynSoph | 0.89 | 0.25 | 0.04 | 0.2 | 0.03 |
| AvgNounBl | 0.13 | 0.55 | 0.22 | 0.28 | 0.52 |
| AvgPronounBl | 0.04 | 0.71 | -0.19 | 0.37 | 0.11 |
| AvgVerbBl | 0.04 | 0.84 | -0.17 | 0.36 | 0.02 |
| AvgAdverbBl | 0.2 | 0.61 | -0.01 | 0.25 | -0.12 |
| AvgAdjectiveBl | -0.08 | 0.65 | 0.05 | 0.06 | 0.04 |
| AvgPrepositionBl | 0.11 | 0.79 | -0.1 | 0.18 | 0.08 |
| AvgNounSen | 0.89 | -0.04 | 0.07 | -0.01 | 0.26 |
| AvgPronounSen | 0.9 | -0.01 | -0.07 | 0.03 | 0.02 |
| AvgVerbSen | 0.97 | 0.08 | -0.05 | 0 | -0.04 |
| AvgAdverbSen | 0.8 | 0.17 | 0.04 | 0.09 | -0.15 |
| AvgAdjectiveSen | 0.81 | 0 | 0 | -0.16 | -0.05 |
| AvgPrepositionSen | 0.9 | 0.17 | 0 | 0 | 0.02 |
| AvgUnqNoundBl | 0.13 | 0.52 | 0.2 | 0.16 | 0.48 |
| AvgUnqPronounBl | -0.01 | 0.54 | -0.15 | 0.18 | 0.09 |
| AvgUnqVerbBl | 0.01 | 0.8 | -0.09 | 0.29 | 0.01 |
| AvgUnqAdverbBl | 0.12 | 0.69 | -0.01 | 0.1 | -0.11 |
| AvgUnqAdjectiveBl | -0.07 | 0.65 | 0.04 | -0.02 | 0.04 |
| AvgUnqPrepositionBl | 0.04 | 0.77 | -0.11 | 0.17 | 0.05 |
| AvgPronBl_first_person | -0.02 | 0.47 | -0.22 | 0.38 | 0.16 |
| AvgPronBl_indefinite | -0.03 | 0.57 | -0.13 | -0.04 | 0.14 |
| AggPronSen_indefinite | 0.74 | 0.09 | -0.09 | -0.12 | 0.15 |
| AvgPronBl_third_person | 0.08 | 0.54 | -0.03 | 0.14 | -0.03 |
| AggPronSen_third_person | 0.78 | 0.06 | -0.02 | -0.05 | -0.13 |
| AvgSemDep | 0.98 | 0.06 | -0.01 | 0.01 | 0.08 |
| WdDiffLemmaStem | -0.11 | 0.34 | 0.02 | -0.16 | -0.19 |
| WdDiffWdStem | -0.11 | 0.39 | -0.03 | -0.1 | -0.42 |
| WdMaxDpthHypernymTree | 0.02 | -0.21 | 0.13 | 0.1 | 0.23 |
| WdAvgDpthHypernymTree | 0.02 | -0.22 | 0.09 | 0.1 | 0.19 |
| WdPathCntHypernymTree | 0.01 | -0.23 | 0.09 | 0.04 | 0.27 |
| WdPolysemyCnt | -0.04 | -0.04 | -0.4 | 0.25 | 0.07 |
| WdSylCnt | -0.11 | 0.5 | 0.32 | -0.08 | -0.3 |
| AvgAOADoc_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 |
| AvgAOABl_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 |
| AvgAOASen_Shock | 0.49 | 0.17 | -0.02 | 0.08 | -0.39 |
| AvgAOADoc_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 |
| AvgAOABl_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 |
| AvgAOASen_Cortese | 0.25 | -0.19 | 0.35 | 0.03 | -0.39 |
| AvgAOADoc_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 |
| AvgAOABl_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 |
| AvgAOASen_Kuperman | 0.11 | -0.05 | 0.62 | 0 | -0.32 |
| AvgAOADoc_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 |
| AvgAOABl_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 |
| AvgAOASen_Bird | 0.35 | -0.02 | 0.26 | 0.18 | -0.53 |
| AvgAOADoc_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 |
| AvgAOABl_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 |
| AvgAOASen_Bristol | 0.41 | -0.05 | 0.1 | 0.01 | -0.51 |
| AvgAOEDoc_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 |
| AvgAOEBl_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 |
| AvgAOESen_IndexPolyFAT.3 | 0.06 | -0.03 | 0.83 | 0.04 | 0.08 |
| AvgAOEDoc_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 |
| AvgAOEBl_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 |
| AvgAOESen_InvLinRegSlo | 0.17 | -0.15 | 0.69 | 0.07 | -0.03 |
| AvgAOEDoc_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 |
| AvgAOEBl_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 |
| AvgAOESen_InfPointPoly | 0.21 | -0.05 | 0.7 | 0.01 | -0.14 |
| AvgAOEDoc_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 |
| AvgAOEBl_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 |
| AvgAOESen_InvAverage | 0.2 | -0.06 | 0.73 | -0.01 | -0.13 |
| AvgAOEDoc_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 |
| AvgAOEBl_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 |
| AvgAOESen_IndexAbThr.0.3. | -0.03 | 0.02 | 0.78 | 0.04 | 0.18 |
| AvgNmdEntBl | 0.06 | 0.31 | 0.06 | 0.04 | 0.62 |
| AvgNounNmdEntBl | 0.04 | 0.16 | 0.2 | 0 | 0.67 |
| AvgUnqNmdEntBl | 0.03 | 0.34 | 0.06 | 0.03 | 0.63 |
| AvgNmdEntSen | 0.57 | -0.06 | 0.08 | 0 | 0.43 |
| TCorefChainDoc | 0.02 | 0.51 | -0.04 | 0.33 | 0.25 |
| AvgCorefChain | 0 | 0.44 | -0.14 | 0.18 | 0.03 |
| AvgChainSpan | 0.17 | 0.66 | -0.09 | 0.15 | -0.05 |
| AvgInferenceDistChain | 0.26 | 0.29 | -0.06 | 0.06 | 0.15 |
| TActCorefChainWd | -0.09 | -0.34 | -0.03 | -0.01 | 0.03 |
| TCorefChainBigSpan | 0.21 | 0.34 | -0.02 | 0.16 | 0.02 |
| AvgConnBl_addition | 0.45 | 0.25 | -0.03 | 0.6 | 0.11 |
| AvgConnSen_addition | 0.85 | -0.02 | 0.02 | 0.17 | 0.09 |
| AvgConnBl_conjunctions | 0.48 | 0.32 | -0.03 | 0.43 | 0.19 |
| AvgConnSen_conjunctions | 0.91 | -0.02 | 0.02 | 0.05 | 0.11 |
| AvgConnBl_contrasts | 0.16 | 0.51 | 0.12 | -0.15 | 0 |
| AvgConnSen_contrasts | 0.67 | 0.22 | 0.05 | -0.24 | -0.1 |
| AvgConnBl_coord_conjs | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 |
| AvgConnSen_coord_conjs | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 |
| AvgConnBl_coord_connects | 0.47 | 0.41 | -0.03 | 0.52 | 0.09 |
| AvgConnSen_coord_connects | 0.94 | 0.01 | 0.02 | 0.08 | 0.05 |
| AvgConnBl_logical_conns | 0.44 | 0.24 | -0.04 | 0.5 | 0.17 |
| AvgConnSen_logical_conns | 0.86 | -0.04 | 0.03 | 0.12 | 0.1 |
| AvgConnBl_oppositions | 0.17 | 0.49 | 0.07 | -0.22 | 0.04 |
| AvgConnSen_oppositions | 0.66 | 0.19 | 0.03 | -0.29 | -0.05 |
| AvgConnBl_order | 0.04 | 0.16 | -0.02 | 0.45 | -0.06 |
| AvgConnSen_order | 0.4 | -0.09 | 0.01 | 0.28 | 0.01 |
| AvgConnBl_reas_purp | 0.07 | 0.32 | -0.03 | 0.49 | -0.16 |
| AvgConnSen_reas_purp | 0.58 | 0.04 | 0.03 | 0.25 | -0.17 |
| AvgConnBl_semi_coords | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 |
| AvgConnSen_semi_coords | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 |
| AvgConnBl_sentence_link | 0.39 | 0.51 | -0.1 | 0.58 | 0.07 |
| AvgConnSen_sentence_link | 0.95 | 0 | 0 | 0.11 | 0.06 |
| AvgConnBl_simp_subords | -0.13 | 0.52 | -0.14 | -0.04 | 0.01 |
| AvgConnSen_simp_subords | 0.52 | 0.16 | -0.03 | -0.1 | -0.02 |
| AvgConnBl_temp_conns | -0.1 | 0.34 | -0.21 | 0.22 | 0.04 |
| AvgConnSen_temp_conns | 0.33 | 0 | -0.12 | 0.07 | 0.06 |
| LexChainAvgSpan | -0.04 | 0.29 | -0.21 | 0.47 | 0.17 |
| LexChainMaxSp | 0.04 | 0.61 | -0.1 | 0.45 | 0.02 |
| AvgBlScore | 0.22 | 0.56 | -0.18 | 0.53 | 0.08 |
| AvgSenScore | 0.93 | 0.05 | -0.06 | 0.06 | 0.07 |
| SenScoreStDev | -0.39 | 0.27 | -0.02 | 0.67 | 0.07 |
| AvgIntraBlCoh_LeackChod | -0.71 | 0.35 | 0.01 | 0.54 | 0.08 |
| AvgSenAdjCoh_LeackChod | -0.69 | 0.33 | 0.02 | 0.56 | 0.07 |
| AvgSenBlCoh_LeackChod | 0.76 | -0.5 | -0.07 | -0.05 | -0.17 |
| AvgIntraBlCoh_WuPalmer | -0.71 | 0.35 | 0.01 | 0.53 | 0.08 |
| AvgSenAdjCoh_WuPalmer | -0.7 | 0.33 | 0.01 | 0.55 | 0.08 |
| AvgSenBlCoh_WuPalmer | 0.76 | -0.51 | -0.08 | -0.07 | -0.17 |
| AvgIntraBlCoh_Path | -0.72 | 0.34 | 0 | 0.53 | 0.09 |
| AvgSenAdjCoh_Path | -0.7 | 0.32 | 0 | 0.55 | 0.08 |
| AvgSenBlCoh_Path | 0.78 | -0.52 | -0.07 | -0.17 | -0.14 |
| AvgIntraBlCoh_LSA | -0.7 | 0.35 | -0.01 | 0.55 | 0.09 |
| AvgSenAdjCoh_LSA | -0.68 | 0.34 | -0.01 | 0.57 | 0.09 |
| AvgSenBlCoh_LSA | 0.76 | -0.49 | -0.13 | -0.04 | -0.15 |
| AvgIntraBlCoh_LDA | -0.7 | 0.37 | -0.02 | 0.53 | 0.08 |
| AvgSenAdjCoh_LDA | -0.68 | 0.35 | -0.02 | 0.55 | 0.07 |
| AvgSenBlCoh_LDA | 0.71 | -0.39 | -0.2 | 0.02 | -0.19 |
| AvgIntraBlCoh_word2vec | -0.69 | 0.35 | 0.01 | 0.55 | 0.08 |
| AvgSenAdjCoh_word2vec | -0.69 | 0.33 | 0 | 0.56 | 0.09 |
| AvgSenBlCoh_word2vec | 0.76 | -0.53 | -0.09 | -0.07 | -0.13 |
| AvgBlVoiceCoOcc | -0.05 | 0.47 | -0.14 | 0.51 | 0.08 |
| AvgVoice | -0.05 | 0.46 | -0.15 | 0.54 | 0.08 |
| AvgSenSyll | 0.98 | 0.06 | 0.01 | -0.01 | 0.06 |
| AvgSenStressedSyll | 0.96 | 0.08 | 0.03 | 0.01 | 0.06 |
| AvgRhythmUnits | 0.18 | 0 | -0.08 | -0.18 | 0.01 |
| AvgRhythmUnitSyll | 0.81 | -0.04 | 0.02 | 0.09 | 0.09 |
| AvgRhythmUnitStreesSyll | 0.8 | -0.03 | 0.04 | 0.11 | 0.1 |
| LangRhythmCoeff | 0.27 | -0.25 | -0.03 | 0.24 | 0.22 |
| LangRhythmId | -0.19 | 0.11 | 0.02 | -0.04 | 0.21 |
| FrqRhythmId | 0.68 | -0.44 | -0.04 | -0.07 | -0.11 |
| LangRhythmDiameter | 0.34 | 0.07 | 0 | 0.11 | 0.35 |
| SenAsson | -0.05 | 0.28 | -0.05 | 0.17 | -0.09 |
| AvgDepsBl_acl | -0.14 | 0.18 | -0.12 | 0.02 | -0.13 |
| AvgDepsSen_acl | 0.12 | -0.23 | -0.17 | -0.14 | -0.17 |
| AvgDepsBl_advcl | 0.14 | 0.49 | -0.08 | 0.25 | -0.2 |
| AvgDepsSen_advcl | 0.66 | 0.18 | 0.02 | 0.03 | -0.23 |
| AvgDepsBl_advmod | 0.14 | 0.61 | -0.03 | 0.33 | -0.12 |
| AvgDepsSen_advmod | 0.79 | 0.15 | 0.01 | 0.15 | -0.12 |
| AvgDepsBl_amod | -0.1 | 0.58 | 0.09 | 0.11 | 0.15 |
| AvgDepsSen_amod | 0.7 | 0 | 0.02 | -0.07 | 0.09 |
| AvgDepsBl_aux | 0.16 | 0.54 | -0.15 | 0.06 | -0.03 |
| AvgDepsSen_aux | 0.76 | 0.21 | -0.09 | -0.06 | -0.08 |
| AvgDepsBl_auxpass | 0.04 | 0.4 | -0.05 | 0.2 | -0.08 |
| AvgDepsBl_case | 0.08 | 0.67 | -0.08 | 0.16 | 0.28 |
| AvgDepsSen_case | 0.84 | 0.1 | -0.03 | -0.01 | 0.17 |
| AvgDepsBl_cc | 0.49 | 0.35 | 0 | 0.43 | 0.19 |
| AvgDepsSen_cc | 0.92 | 0 | 0.03 | 0.05 | 0.09 |
| AvgDepsBl_ccomp | 0.2 | 0.28 | -0.25 | 0.31 | 0.11 |
| AvgDepsSen_ccomp | 0.76 | -0.05 | -0.15 | -0.02 | 0.11 |
| AvgDepsBl_compound | -0.02 | -0.06 | 0.5 | 0.1 | 0.4 |
| AvgDepsSen_compound | 0.37 | -0.22 | 0.24 | 0.02 | 0.28 |
| AvgDepsBl_conj | 0.6 | 0.31 | 0.02 | 0.37 | 0.15 |
| AvgDepsSen_conj | 0.88 | 0.02 | 0.02 | 0.03 | 0.08 |
| AvgDepsBl_cop | -0.05 | 0.49 | -0.09 | -0.11 | -0.03 |
| AvgDepsSen_cop | 0.64 | 0.02 | -0.13 | -0.16 | -0.15 |
| AvgDepsBl_dep | 0.38 | 0.28 | 0.03 | 0.25 | 0.25 |
| AvgDepsSen_dep | 0.73 | 0.01 | 0.05 | 0.02 | 0.18 |
| AvgDepsBl_det | 0.14 | 0.58 | 0.03 | 0.31 | 0.22 |
| AvgDepsSen_det | 0.76 | 0.11 | -0.01 | 0.03 | 0.19 |
| AvgDepsBl_dobj | 0.14 | 0.56 | -0.02 | 0.47 | 0.08 |
| AvgDepsSen_dobj | 0.9 | -0.03 | 0 | 0.06 | 0.06 |
| AvgDepsBl_mark | 0.09 | 0.6 | -0.1 | 0.25 | -0.23 |
| AvgDepsSen_mark | 0.75 | 0.1 | -0.03 | 0 | -0.24 |
| AvgDepsBl_mwe | 0.14 | 0.34 | 0.04 | 0.06 | 0.15 |
| AvgDepsSen_mwe | 0.45 | 0.15 | 0.08 | 0.04 | 0.14 |
| AvgDepsBl_neg | 0 | 0.38 | 0.01 | -0.11 | 0.02 |
| AvgDepsSen_neg | 0.39 | 0.05 | 0.05 | -0.21 | -0.07 |
| AvgDepsBl_nmod | 0.09 | 0.62 | -0.1 | 0.16 | 0.3 |
| AvgDepsSen_nmod | 0.83 | 0.08 | -0.03 | -0.03 | 0.2 |
| AvgDepsBl_nsubj | 0.07 | 0.79 | -0.16 | 0.34 | 0.15 |
| AvgDepsSen_nsubj | 0.97 | 0.03 | -0.04 | 0 | 0.04 |
| AvgDepsBl_nsubjpass | 0.05 | 0.34 | -0.04 | 0.2 | -0.04 |
| AvgDepsBl_nummod | 0.02 | 0.2 | -0.15 | 0.06 | 0.24 |
| AvgDepsBl_punct | -0.53 | 0.62 | -0.01 | -0.15 | 0.13 |
| AvgDepsSen_punct | -0.11 | 0.31 | 0 | -0.06 | -0.15 |
| AvgDepsBl_xcomp | 0.03 | 0.41 | 0.03 | 0.34 | -0.22 |
| AvgDepsSen_xcomp | 0.71 | 0.07 | 0.07 | 0.06 | -0.24 |
ReaderBench Model 2 {#readerbench-model-2}
General Description
ReaderBench Model 2 is a simplified version of Model 1 that better handles multi-paragraph compositions, and Model 2 is recommended over Model 1.
Model 2 is an ensemble (formed by averaging predicted quality scores) of three sub-models that are described below.
All of these models used ReaderBench scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
ReaderBench Model 2a
This model was trained with fall data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
pls = partial least squares regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
svm = support vector machines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | svm | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|
| -4.338 | 0.2371 | 0.1755 | 0.1780 | 0.2234 | 0.2532 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | svm | cube |
|:------------------------------------------------|:--------|:-----|:------|:------|:-----|:------|
| WdEnt | 20.53 | 4.67 | 10.12 | 73.84 | 5.16 | 18.67 |
| AvgDepsSen_dep | 4.65 | 1.23 | 0.88 | 16.82 | 0.88 | 5.25 |
| Content.words | 4.59 | 4.32 | 4.77 | 0 | 4.68 | 7.87 |
| Words | 3.72 | 4.44 | 4.67 | 0 | 4.67 | 4.17 |
| LxcDiv | 3.1 | 4.08 | 3.29 | 0 | 4.06 | 3.4 |
| AvgAOASen_Shock | 2.77 | 1.45 | 1.16 | 9.34 | 1.39 | 1.7 |
| TCorefChainDoc | 2.62 | 2.98 | 0.81 | 0 | 2.06 | 5.86 |
| AvgChainSpan | 2.59 | 3.27 | 3.83 | 0 | 3.1 | 2.47 |
| WdDiffWdStem | 2.46 | 2.73 | 3.07 | 0 | 2.24 | 3.7 |
| SynSoph | 2.12 | 1.71 | 0.92 | 0 | 1.82 | 5.09 |
| AvgDepsSen_punct | 2.03 | 2.48 | 1.68 | 0 | 1.74 | 3.55 |
| TActCorefChainWd | 1.93 | 1.6 | 1.91 | 0 | 1.47 | 4.01 |
| WdDiffLemmaStem | 1.66 | 1.52 | 0.72 | 0 | 2.44 | 2.93 |
| RdbltyFlesch | 1.55 | 0.77 | 1.22 | 0 | 1.09 | 4.01 |
| WdLettStdDev | 1.44 | 2.35 | 1.51 | 0 | 2.13 | 0.93 |
| AvgAOESen_InverseAverage | 1.37 | 1.44 | 1.22 | 0 | 1.09 | 2.62 |
| Sentences | 1.3 | 2.84 | 1.77 | 0 | 1.82 | 0 |
| AvgWdLen | 1.27 | 2.65 | 1.57 | 0 | 2.02 | 0 |
| LexChainMaxSp | 1.26 | 2.89 | 1.19 | 0 | 2.02 | 0 |
| AvgAOADoc_Shock | 1.26 | 2.36 | 1.89 | 0 | 1.68 | 0.31 |
| AvgAOADoc_Kuperman | 1.25 | 0.72 | 1.01 | 0 | 1.3 | 2.78 |
| WdSylCnt | 1.15 | 1.57 | 1.83 | 0 | 1.51 | 0.77 |
| CharEnt | 1.14 | 2.65 | 0.96 | 0 | 1.85 | 0 |
| LexChainAvgSpan | 1.12 | 2.18 | 1.5 | 0 | 1.86 | 0 |
| AvgDepsSen_advcl | 1.07 | 0.93 | 0.85 | 0 | 1.35 | 1.85 |
| AvgAOASen_Kuperman | 1.04 | 1.23 | 1.48 | 0 | 1.46 | 0.93 |
| AvgCorefChain | 1 | 1.86 | 0.85 | 0 | 0.9 | 1.08 |
| WdAvgDpthHypernymTree | 1 | 1.14 | 0.87 | 0 | 0.97 | 1.7 |
| SenStdDevWd | 0.98 | 1.96 | 1.43 | 0 | 1.49 | 0 |
| TCorefChainBigSpan | 0.95 | 2.16 | 1.44 | 0 | 1.13 | 0 |
| AvgAOADoc_Bristol | 0.94 | 1.75 | 1.03 | 0 | 1.1 | 0.62 |
| LxcSoph | 0.92 | 1.64 | 1.2 | 0 | 0.85 | 0.77 |
| AvgAdverbSen | 0.88 | 0.89 | 1.38 | 0 | 1.46 | 0.62 |
| RdbltyDaleChall | 0.87 | 1.75 | 1.63 | 0 | 1 | 0 |
| AvgSenAdjCoh_LDA | 0.82 | 1.97 | 0.64 | 0 | 1.33 | 0 |
| AvgRhythmUnits | 0.82 | 1.12 | 1.13 | 0 | 1.15 | 0.62 |
| FrqRhythmId | 0.8 | 1.69 | 1.07 | 0 | 1.18 | 0 |
| AvgAOADoc_Bird | 0.78 | 0.95 | 0.3 | 0 | 1.43 | 0.93 |
| AvgVoice | 0.78 | 2.01 | 0.76 | 0 | 0.99 | 0 |
| AvgAOADoc_Cortese | 0.77 | 0.69 | 1.3 | 0 | 1.57 | 0.31 |
| WdPathCntHypernymTree | 0.71 | 1.45 | 0.84 | 0 | 1.17 | 0 |
| AvgConnSen_simple_subordinators | 0.7 | 0.51 | 2.49 | 0 | 0.82 | 0 |
| AvgAOASen_Bristol | 0.68 | 0.66 | 0.71 | 0 | 1.29 | 0.62 |
| AvgRhythmUnitStreesSyll | 0.63 | 0.08 | 0.91 | 0 | 0.81 | 1.23 |
| AvgInferenceDistChain | 0.62 | 1.39 | 0.34 | 0 | 1.2 | 0 |
| AggPronSen_indefinite | 0.62 | 0.45 | 0.63 | 0 | 1.31 | 0.62 |
| AvgAOASen_Bird | 0.6 | 1.13 | 0.37 | 0 | 1.37 | 0 |
| AvgDepsSen_compound | 0.6 | 0.72 | 0.5 | 0 | 0.48 | 1.08 |
| WdPolysemyCnt | 0.58 | 0 | 1.09 | 0 | 1.9 | 0 |
| AvgDepsSen_ccomp | 0.57 | 0.09 | 1.32 | 0 | 0.9 | 0.62 |
| AvgAOASen_Cortese | 0.55 | 1.15 | 0.3 | 0 | 1.17 | 0 |
| AvgDepsSen_cop | 0.54 | 0.24 | 0.58 | 0 | 0.97 | 0.77 |
| AvgPronounSen | 0.54 | 0.12 | 0.93 | 0 | 0.48 | 1.08 |
| AvgNmdEntSen | 0.52 | 0.24 | 1.12 | 0 | 1.33 | 0 |
| AvgNounSen | 0.52 | 0.24 | 0.15 | 0 | 0.18 | 1.7 |
| AvgDepsSen_nmod | 0.48 | 0.7 | 0.69 | 0 | 1 | 0 |
| AvgDepsSen_aux | 0.48 | 0.24 | 0.92 | 0 | 1.31 | 0 |
| AvgConnSen_addition | 0.48 | 1.1 | 0.6 | 0 | 0.66 | 0 |
| AvgDepsSen_dobj | 0.48 | 0.23 | 1.51 | 0 | 0.16 | 0.62 |
| AvgAOEDoc_InverseLinearRegressionSlope | 0.44 | 0.4 | 0.8 | 0 | 0.68 | 0.31 |
| AvgDepsSen_mark | 0.41 | 0.43 | 0.95 | 0 | 0.73 | 0 |
| AvgConnSen_temporal_connectors | 0.41 | 0.32 | 0.64 | 0 | 1.11 | 0 |
| AvgDepsSen_det | 0.4 | 0.18 | 0.4 | 0 | 0.72 | 0.62 |
| AvgConnSen_semi_coordinators | 0.38 | 0.8 | 0.15 | 0 | 0.16 | 0.62 |
| AvgConnSen_order | 0.36 | 0.31 | 1.74 | 0 | 0.03 | 0 |
| AggPronSen_third_person | 0.36 | 0.57 | 0.91 | 0 | 0.41 | 0 |
| LangRhythmDiameter | 0.35 | 0.57 | 0.79 | 0 | 0.08 | 0.31 |
| SenAsson | 0.35 | 0.8 | 0.83 | 0 | 0.16 | 0 |
| AvgAOEDoc_IndexAboveThreshold.0.3. | 0.33 | 0.03 | 0.43 | 0 | 0.87 | 0.31 |
| AvgDepsSen_amod | 0.29 | 0.33 | 0.98 | 0 | 0.27 | 0 |
| AvgAdjectiveSen | 0.28 | 0.1 | 1.28 | 0 | 0.21 | 0 |
| AvgConnSen_oppositions | 0.27 | 0.54 | 0.82 | 0 | 0.07 | 0 |
| AvgDepsSen_xcomp | 0.24 | 0.01 | 0.13 | 0 | 1.04 | 0 |
| AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.21 | 0.12 | 0.1 | 0 | 0.78 | 0 |
| LangRhythmId | 0.19 | 0.47 | 0.45 | 0 | 0.05 | 0 |
| AvgDepsSen_neg | 0.18 | 0.03 | 1.05 | 0 | 0 | 0 |
| AvgDepsSen_mwe | 0.17 | 0.38 | 0.47 | 0 | 0.04 | 0 |
| LangRhythmCoeff | 0.16 | 0 | 0.22 | 0 | 0.61 | 0 |
| AvgDepsSen_acl | 0.06 | 0.25 | 0 | 0 | 0.02 | 0 |
ReaderBench Model 2b
This model was trained with winter data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
pls = partial least squares regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
svm = support vector machines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | svm | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|
| -5.4658 | 0.2205 | 0.5768 | 0.2047 | 0.0528 | 0.0400 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | svm | cube |
|:------------------------------------------------|:--------|:-----|:-----|:------|:-----|:------|
| Content.words | 11.94 | 5.23 | 4.51 | 41.33 | 4.49 | 15.7 |
| WdEnt | 8.27 | 5.15 | 5.03 | 19.27 | 4.44 | 21.01 |
| SynSoph | 4.17 | 1.03 | 2.06 | 14.97 | 1.69 | 0 |
| LxcDiv | 3.24 | 4.93 | 3.5 | 0 | 3.86 | 5.8 |
| AvgDepsSen_det | 3.18 | 0.25 | 1.12 | 12.65 | 0.94 | 3.38 |
| TCorefChainDoc | 2.63 | 4 | 2.76 | 0 | 2.45 | 6.76 |
| AvgChainSpan | 2.28 | 3.64 | 2.57 | 0 | 2.88 | 1.45 |
| LexChainMaxSp | 2.25 | 3.56 | 2.72 | 0 | 1.98 | 0 |
| TActCorefChainWd | 2.22 | 0.83 | 0.71 | 7.46 | 0.78 | 6.76 |
| Sentences | 2.18 | 3.8 | 2.48 | 0 | 2.25 | 0 |
| AvgNounSen | 2.07 | 0.89 | 1.54 | 4.33 | 0.64 | 6.52 |
| CharEnt | 1.7 | 3.41 | 1.66 | 0 | 2.16 | 0.97 |
| RdbltyFlesch | 1.36 | 0.46 | 1.91 | 0 | 1.18 | 5.56 |
| WdLettStdDev | 1.31 | 2.58 | 1.31 | 0 | 2 | 0 |
| AvgSenAdjCoh_LeackockChodorow | 1.3 | 2.75 | 1.24 | 0 | 1.87 | 0 |
| FrqRhythmId | 1.28 | 2.48 | 1.39 | 0 | 1.13 | 0 |
| AvgDepsSen_aux | 1.25 | 0.94 | 1.69 | 0 | 0.97 | 3.38 |
| AvgWdLen | 1.24 | 2.36 | 1.26 | 0 | 2.04 | 0 |
| AvgAOADoc_Bristol | 1.21 | 1.59 | 1.61 | 0 | 0.88 | 0 |
| AvgDepsSen_compound | 1.2 | 1.17 | 1.78 | 0 | 0.48 | 0 |
| AvgVoice | 1.2 | 2.75 | 1.13 | 0 | 1.19 | 0 |
| AvgAOADoc_Shock | 1.16 | 2.54 | 1.13 | 0 | 1.17 | 0 |
| TCorefChainBigSpan | 1.13 | 2.24 | 1.22 | 0 | 0.78 | 0 |
| AvgConnSen_addition | 1.07 | 1.31 | 1.29 | 0 | 1.31 | 1.69 |
| WdDiffWdStem | 1.04 | 2.05 | 1.06 | 0 | 1.35 | 0 |
| AvgConnSen_logical_connectors | 1.03 | 1.49 | 1.11 | 0 | 1.27 | 2.17 |
| AvgCorefChain | 1.01 | 2.38 | 0.9 | 0 | 1.32 | 0 |
| AggPronSen_third_person | 0.98 | 1.18 | 1.23 | 0 | 0.56 | 1.93 |
| AvgDepsSen_punct | 0.98 | 1.89 | 1.01 | 0 | 1.48 | 0 |
| AvgDepsSen_dep | 0.95 | 1.01 | 1.31 | 0 | 1.07 | 0 |
| AvgRhythmUnitStreesSyll | 0.95 | 0.44 | 1.46 | 0 | 0.9 | 1.21 |
| AvgDepsSen_dobj | 0.95 | 1.12 | 1.04 | 0 | 1.07 | 3.38 |
| AvgAdjectiveSen | 0.91 | 0.44 | 1.47 | 0 | 0.9 | 0 |
| SenStdDevWd | 0.9 | 2.04 | 0.69 | 0 | 1.52 | 1.21 |
| LexChainAvgSpan | 0.87 | 1.85 | 0.77 | 0 | 1.88 | 0 |
| WdPathCntHypernymTree | 0.86 | 1.46 | 0.94 | 0 | 1.38 | 0 |
| AvgAOESen_InverseAverage | 0.85 | 0.71 | 1.27 | 0 | 0.81 | 0 |
| AvgDepsSen_mark | 0.83 | 0.22 | 1.4 | 0 | 1.06 | 0 |
| WdPolysemyCnt | 0.83 | 0.32 | 1.43 | 0 | 0.39 | 0 |
| AvgConnSen_reason_and_purpose | 0.82 | 0.35 | 1.3 | 0 | 0.7 | 0.97 |
| LangRhythmCoeff | 0.8 | 1.15 | 1 | 0 | 0.92 | 0 |
| AvgConnSen_simple_subordinators | 0.78 | 0.11 | 1.3 | 0 | 1.47 | 0 |
| AvgDepsSen_xcomp | 0.76 | 0.11 | 1.25 | 0 | 1.6 | 0 |
| AvgAOASen_Bird | 0.76 | 0.33 | 1.19 | 0 | 0.62 | 0.97 |
| AvgDepsSen_ccomp | 0.75 | 0.16 | 1.29 | 0 | 0.79 | 0 |
| RdbltyDaleChall | 0.75 | 2.41 | 0.41 | 0 | 1.07 | 0 |
| AvgAOEDoc_InflectionPointPolynomial | 0.73 | 0.7 | 1.06 | 0 | 0.65 | 0 |
| AvgAOESen_IndexAboveThreshold.0.3. | 0.7 | 0.47 | 1.01 | 0 | 1.43 | 0 |
| AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.7 | 0.55 | 1.01 | 0 | 1.13 | 0 |
| AggPronSen_indefinite | 0.7 | 0.09 | 1.07 | 0 | 1.64 | 1.21 |
| AvgDepsSen_cop | 0.7 | 0.09 | 1.16 | 0 | 1.49 | 0 |
| AvgNmdEntSen | 0.68 | 0.45 | 1.02 | 0 | 1.07 | 0 |
| AvgConnSen_contrasts | 0.68 | 0.32 | 1.03 | 0 | 0.59 | 1.21 |
| AvgConnSen_oppositions | 0.68 | 0.07 | 1.18 | 0 | 0.94 | 0 |
| AvgDepsSen_advcl | 0.67 | 0.03 | 1.13 | 0 | 1.31 | 0 |
| AvgAdverbSen | 0.67 | 0.43 | 1.01 | 0 | 1.08 | 0 |
| AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.66 | 0 | 1.14 | 0 | 1.18 | 0 |
| AvgDepsSen_nmod | 0.66 | 0.74 | 0.76 | 0 | 1.35 | 1.21 |
| AvgAOADoc_Bird | 0.65 | 0.95 | 0.77 | 0 | 1.11 | 0 |
| AvgDepsSen_amod | 0.65 | 0.53 | 0.69 | 0 | 0.9 | 3.86 |
| AvgConnSen_semi_coordinators | 0.64 | 0.24 | 1.04 | 0 | 0.78 | 0 |
| WdMaxDpthHypernymTree | 0.62 | 1.46 | 0.46 | 0 | 1.61 | 0 |
| AvgAOASen_Shock | 0.62 | 1.13 | 0.62 | 0 | 1.34 | 0 |
| AvgAOASen_Kuperman | 0.6 | 0.15 | 1.01 | 0 | 0.45 | 0.48 |
| AvgConnSen_temporal_connectors | 0.58 | 0.27 | 0.99 | 0 | 0.01 | 0 |
| AvgAOASen_Bristol | 0.57 | 0.38 | 0.87 | 0 | 0.67 | 0 |
| LangRhythmDiameter | 0.56 | 0.65 | 0.81 | 0 | 0.06 | 0 |
| AvgConnSen_order | 0.52 | 0.29 | 0.7 | 0 | 1 | 1.21 |
| AvgAOEDoc_IndexAboveThreshold.0.3. | 0.5 | 0.01 | 0.79 | 0 | 1.65 | 0 |
| AvgRhythmUnits | 0.5 | 0.73 | 0.57 | 0 | 1.13 | 0 |
| AvgAOADoc_Kuperman | 0.5 | 0.14 | 0.82 | 0 | 0.86 | 0 |
| AvgAOASen_Cortese | 0.49 | 0.13 | 0.83 | 0 | 0.66 | 0 |
| AvgInferenceDistChain | 0.48 | 0.87 | 0.51 | 0 | 0.71 | 0 |
| WdDiffLemmaStem | 0.48 | 0.4 | 0.62 | 0 | 1.55 | 0 |
| SenAsson | 0.42 | 0.99 | 0.4 | 0 | 0.15 | 0 |
| AvgDepsSen_mwe | 0.41 | 0.66 | 0.52 | 0 | 0.07 | 0 |
| AvgDepsSen_neg | 0.39 | 0.28 | 0.64 | 0 | 0 | 0 |
| AvgDepsSen_acl | 0.33 | 0.45 | 0.46 | 0 | 0.03 | 0 |
| LxcSoph | 0.31 | 0.39 | 0.35 | 0 | 0.92 | 0 |
| AvgAOEDoc_InverseLinearRegressionSlope | 0.27 | 0.19 | 0.39 | 0 | 0.61 | 0 |
| AvgAOADoc_Cortese | 0.24 | 0.76 | 0.09 | 0 | 0.85 | 0 |
| WdSylCnt | 0.23 | 0.83 | 0 | 0 | 1.29 | 0 |
| LangRhythmId | 0.03 | 0.09 | 0.03 | 0 | 0 | 0 |
ReaderBench Model 2c
This model was trained on spring data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
pls = partial least squares regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
gbm = stochastic gradient boosted trees
svm = support vector machines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|:-------|
| -7.3027 | 0.2354 | 0.1868 | 0.1595 | 0.1816 | 0.2191 | 0.0704 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | gbm | svm | cube |
|:------------------------------------------------|:--------|:-----|:-----|:------|:------|:-----|:------|
| Content.words | 11.99 | 4.55 | 5.81 | 30.16 | 21.71 | 4.24 | 11.11 |
| WdEnt | 7.28 | 4.3 | 5.74 | 0 | 21.09 | 4.12 | 12.09 |
| AvgDepsSen_compound | 3.97 | 2.07 | 1.98 | 13.22 | 2.22 | 1.52 | 6.82 |
| AvgWdLen | 3.87 | 2.64 | 2.65 | 7.11 | 4.85 | 2.04 | 7.02 |
| LxcDiv | 3.77 | 4.06 | 4.13 | 0 | 7.72 | 3.59 | 0.78 |
| AvgChainSpan | 3.36 | 3.09 | 2.64 | 5.1 | 4.15 | 2.66 | 2.34 |
| TCorefChainBigSpan | 2.64 | 2.23 | 1.48 | 10.59 | 0.43 | 0.93 | 0 |
| Sentences | 2.37 | 3.33 | 2.15 | 0 | 2.63 | 2.24 | 4.87 |
| AvgDepsSen_mark | 2.21 | 0.38 | 1.17 | 10.59 | 0.08 | 1.45 | 0 |
| AvgDepsSen_dobj | 2 | 0.81 | 0.96 | 8.72 | 0.14 | 1.13 | 0.97 |
| AvgSenAdjCoh_LSA | 1.95 | 2.68 | 1.87 | 0 | 3.17 | 2.26 | 0 |
| AvgCorefChain | 1.94 | 2.2 | 1 | 5.1 | 0.28 | 1.28 | 2.73 |
| WdDiffWdStem | 1.92 | 2.4 | 1.86 | 0 | 2.95 | 2.09 | 1.56 |
| LexChainMaxSp | 1.82 | 3.13 | 2.35 | 0 | 1.28 | 2.01 | 0.97 |
| WdLettStdDev | 1.79 | 3 | 1.66 | 0 | 1.64 | 2.28 | 0.97 |
| TCorefChainDoc | 1.62 | 3.23 | 1.85 | 0 | 0.17 | 1.92 | 2.14 |
| CharEnt | 1.59 | 2.56 | 0.9 | 0 | 0.29 | 2.1 | 5.46 |
| WdSylCnt | 1.53 | 2.45 | 1.7 | 0 | 1.52 | 1.55 | 1.36 |
| FrqRhythmId | 1.47 | 2.67 | 1.7 | 0 | 1.03 | 1.59 | 0.97 |
| AvgDepsSen_punct | 1.36 | 1.82 | 1.57 | 0 | 0.72 | 1.83 | 2.53 |
| AvgAOEDoc_InverseLinearRegressionSlope | 1.32 | 1.31 | 0.73 | 4.26 | 0.24 | 0.89 | 0.39 |
| RdbltyDaleChall | 1.25 | 1.81 | 1.27 | 0 | 1.04 | 1.02 | 3.51 |
| AvgAOADoc_Shock | 1.2 | 2.2 | 1.24 | 0 | 0.69 | 1.8 | 0 |
| LangRhythmCoeff | 1.06 | 1.61 | 1.41 | 0 | 1.03 | 1.24 | 0.19 |
| LexChainAvgSpan | 1.05 | 1.94 | 1.36 | 0 | 0.16 | 1.66 | 0 |
| SenAsson | 1.05 | 1.63 | 0.58 | 3.07 | 0.02 | 0.56 | 0 |
| AvgVoice | 1 | 2.62 | 0.58 | 0 | 0 | 1.36 | 0.39 |
| AvgNounSen | 0.97 | 1.09 | 1.59 | 0 | 0.47 | 1.06 | 2.14 |
| WdDiffLemmaStem | 0.94 | 1.65 | 1.01 | 0 | 0.36 | 1.32 | 0.78 |
| TActCorefChainWd | 0.94 | 0.93 | 0.74 | 0 | 1.05 | 0.71 | 4.09 |
| AvgAOADoc_Cortese | 0.93 | 1.16 | 0.9 | 0 | 0.56 | 1.89 | 0.39 |
| AvgAOASen_Bristol | 0.92 | 0.35 | 1.28 | 2.08 | 1.15 | 0.34 | 0.39 |
| SenStdDevWd | 0.92 | 1.6 | 0.97 | 0 | 0.09 | 1.78 | 0 |
| AvgDepsSen_xcomp | 0.83 | 0.41 | 1.61 | 0 | 1.42 | 0.99 | 0 |
| AvgAdjectiveSen | 0.83 | 1.24 | 1 | 0 | 0.18 | 1.21 | 1.36 |
| AvgDepsSen_nmod | 0.81 | 0.16 | 1.23 | 0 | 0.38 | 1.25 | 3.51 |
| AvgAOADoc_Kuperman | 0.8 | 0.7 | 1.18 | 0 | 0.65 | 1.44 | 0.39 |
| AvgDepsSen_amod | 0.79 | 1.11 | 0.91 | 0 | 0.1 | 1.33 | 1.36 |
| AvgDepsSen_ccomp | 0.78 | 1.06 | 1.41 | 0 | 0.27 | 1.18 | 0 |
| AvgAOASen_Kuperman | 0.78 | 0.78 | 0.83 | 0 | 0.41 | 0.99 | 2.73 |
| AvgNmdEntSen | 0.78 | 0.93 | 1 | 0 | 1.05 | 1.05 | 0 |
| AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.76 | 0.58 | 1.12 | 0 | 0.4 | 0.84 | 2.73 |
| AvgConnSen_simple_subordinators | 0.74 | 0.25 | 0.86 | 0 | 1.12 | 1.52 | 0.39 |
| AvgPronounSen | 0.72 | 1.09 | 1.13 | 0 | 0.02 | 0.99 | 0.97 |
| AvgAOASen_Shock | 0.69 | 0.48 | 1.51 | 0 | 0.21 | 1.32 | 0 |
| AvgConnSen_reason_and_purpose | 0.68 | 0.16 | 1.31 | 0 | 0.82 | 1.29 | 0 |
| AvgAOASen_Cortese | 0.66 | 1.25 | 0.45 | 0 | 0.24 | 1.22 | 0 |
| AvgAOESen_InverseLinearRegressionSlope | 0.66 | 0.99 | 1.02 | 0 | 0.31 | 0.68 | 0.97 |
| AvgAOEDoc_InflectionPointPolynomial | 0.65 | 0.64 | 0.36 | 0 | 0.37 | 0.61 | 3.7 |
| AvgConnSen_addition | 0.65 | 0.88 | 0.88 | 0 | 0.12 | 1.21 | 0.39 |
| AvgConnSen_order | 0.64 | 0.44 | 0.48 | 0 | 0.92 | 1.41 | 0 |
| AvgInferenceDistChain | 0.64 | 0.8 | 0.91 | 0 | 0.7 | 0.83 | 0 |
| WdPolysemyCnt | 0.62 | 0.27 | 0.93 | 0 | 0.37 | 1.61 | 0 |
| AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.61 | 0.83 | 0.71 | 0 | 0.07 | 1.07 | 0.97 |
| AvgRhythmUnits | 0.61 | 0.3 | 1.24 | 0 | 0.32 | 1.3 | 0 |
| AvgDepsSen_aux | 0.57 | 0.03 | 1.26 | 0 | 0.38 | 1.32 | 0 |
| SynSoph | 0.57 | 0.59 | 0.85 | 0 | 0.08 | 1.02 | 0.97 |
| AvgDepsSen_cop | 0.55 | 0.87 | 0.48 | 0 | 0.05 | 1.25 | 0 |
| AvgRhythmUnitStreesSyll | 0.52 | 0.76 | 1.17 | 0 | 0.14 | 0.56 | 0 |
| AvgDepsSen_advmod | 0.48 | 0.33 | 0.6 | 0 | 0.2 | 1.26 | 0 |
| AvgDepsSen_det | 0.48 | 0.22 | 1.04 | 0 | 0.45 | 0.68 | 0.39 |
| AggPronSen_third_person | 0.47 | 0.86 | 0.8 | 0 | 0.08 | 0.58 | 0 |
| AvgAOADoc_Bristol | 0.45 | 0.36 | 0.71 | 0 | 0.12 | 1.02 | 0.19 |
| AvgDepsSen_acl | 0.44 | 1.28 | 0.29 | 0 | 0.16 | 0.36 | 0 |
| AvgAOADoc_Bird | 0.44 | 0.38 | 0.84 | 0 | 0.13 | 0.89 | 0 |
| WdAvgDpthHypernymTree | 0.43 | 0.79 | 0.71 | 0 | 0.06 | 0.54 | 0 |
| RdbltyFlesch | 0.43 | 0.42 | 1.35 | 0 | 0.03 | 0.44 | 0 |
| AvgDepsSen_dep | 0.42 | 0.68 | 0.6 | 0 | 0.02 | 0.75 | 0 |
| AggPronSen_indefinite | 0.41 | 0.34 | 0.51 | 0 | 0.14 | 1.05 | 0 |
| AvgConnSen_semi_coordinators | 0.39 | 0 | 1.01 | 0 | 0.13 | 0.92 | 0 |
| AvgDepsSen_mwe | 0.38 | 0.6 | 1.17 | 0 | 0.1 | 0.07 | 0 |
| AvgDepsSen_advcl | 0.38 | 0.06 | 0.44 | 0 | 0.01 | 1.4 | 0 |
| AvgDepsSen_neg | 0.37 | 0.51 | 0.97 | 0 | 0.4 | 0.05 | 0 |
| WdPathCntHypernymTree | 0.36 | 0.89 | 0.33 | 0 | 0.19 | 0.35 | 0 |
| AvgAOESen_IndexAboveThreshold.0.3. | 0.35 | 0.27 | 0 | 0 | 0.41 | 1.05 | 0 |
| AvgAOASen_Bird | 0.33 | 0.04 | 0.75 | 0 | 0.59 | 0.42 | 0 |
| LxcSoph | 0.31 | 0.02 | 0.61 | 0 | 0.16 | 0.18 | 1.95 |
| AvgConnSen_oppositions | 0.26 | 0.11 | 0.98 | 0 | 0.36 | 0 | 0 |
| LangRhythmDiameter | 0.24 | 0.3 | 0.84 | 0 | 0.12 | 0.03 | 0 |
| AvgConnSen_temporal_connectors | 0.17 | 0.23 | 0.58 | 0 | 0.09 | 0.01 | 0 |
| LangRhythmId | 0.09 | 0.22 | 0.23 | 0 | 0.02 | 0.01 | 0 |
ReaderBench Model 3 {#readerbench-model-3}
General Description
ReaderBench Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below.
The models were trained on ReaderBench scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons).
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
More details on the sample will be provided once peer review is complete on the main study using this model.
ReaderBench Model 3narr
This model was trained on 15-minute narrative writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
gbm = stochastic gradient boosted trees
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | enet | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|:-------|:-------|
| 0.0000 | 0.1419 | 0.0945 | 0.3143 | 0.0729 | 0.0816 | 0.1792 | 0.1538 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |rf |mars |gbm |svm |enet|cube|
|-------------------------------------------------|-------|----|----|-----|-----|----|----|----|
|Content.words |13.76 |2 |2.22|32.41|16.24|2.38|4.95|8.73|
|RB.AvgWdLen |7.5 |0.81|1.06|21.05|1.14 |1.01|1.38|3.55|
|RB.AvgDepsBl_compound |4.66 |0.89|1.01|11.15|2.17 |0.45|2.45|3.09|
|RB.WdEnt |4.59 |1.7 |2.06|7.03 |4.52 |1.75|4.27|5.73|
|RB.LangRhythmId |3.17 |0.69|0.35|8.26 |0.06 |0.21|2.85|0.18|
|RB.RdbltyDaleChall |3.15 |1.27|1.24|4.63 |2.86 |0.95|4.05|3.27|
|RB.AvgUnqWdBl |2.75 |1.63|1.92|4.02 |0.46 |1.82|0 |6.45|
|RB.LxcDiv |2.67 |1.87|1.89|0 |11.82|2.04|3.58|4.27|
|Sentences |2.62 |1.71|2.13|0 |5.6 |1.52|4.67|5.91|
|RB.TCorefChainDoc |2.35 |1.99|1.91|0 |5.33 |1.97|5.29|3.09|
|RB.AvgAOADoc_Cortese |1.8 |0.21|0.37|3.53 |0.76 |0.8 |2.06|1.36|
|RB.CAF |1.78 |1.83|1.52|0 |1.84 |1.87|3.65|3.27|
|RB.AvgNounNmdEntBl |1.6 |0.53|0.45|4.63 |0.21 |0.19|0 |0.36|
|RB.AvgDepsBl_nsubjpass |1.42 |0.81|0.27|3.29 |0.2 |0.41|1.23|0.18|
|RB.AvgDepsBl_aux |1.21 |1.48|1.75|0 |2.12 |1.53|1.3 |2.36|
|RB.TActCorefChainWd |1.06 |0.33|0.95|0 |1.19 |1.17|2.65|2 |
|RB.AvgDepsBl_nsubj |1.05 |1.67|1.78|0 |2.94 |1.85|0 |2.09|
|RB.AvgPronounBl |0.99 |1.72|1.84|0 |3.59 |2.06|0 |1.18|
|RB.AvgUnqNoundBl |0.93 |0.71|0.59|0 |0.37 |0.65|2.75|1.55|
|RB.TCorefChainBigSpan |0.89 |1.61|1.19|0 |0.16 |1.33|2.58|0 |
|RB.AvgAOESen_InflectionPointPolynomial |0.87 |0.97|1.18|0 |0.57 |1.09|2.33|0.73|
|RB.AvgBlScore |0.83 |1.46|1.28|0 |0.92 |1.62|0 |2.18|
|RB.AvgConnBl_addition |0.81 |0.99|0.83|0 |0.77 |0.66|1.36|1.73|
|RB.AvgChainSpan |0.81 |1.52|1.28|0 |2.39 |1.7 |0 |1.27|
|RB.AvgPrepositionBl |0.79 |1.51|1.03|0 |0.76 |1.6 |0.95|1 |
|RB.AvgUnqPrepositionBl |0.76 |1.48|1.09|0 |0.48 |1.55|0.8 |1.09|
|RB.SenStdDevWd |0.74 |0.86|1.25|0 |1.41 |1.28|1.45|0.36|
|RB.AvgAOADoc_Shock |0.72 |0.86|0.94|0 |1.36 |1.15|0.8 |1.27|
|RB.AvgDepsBl_punct |0.68 |1.26|1.34|0 |1 |1 |0.35|1.18|
|RB.AvgCorefChain |0.68 |1.19|1.03|0 |0.34 |1.28|1.1 |0.73|
|RB.AvgNmdEntSen |0.67 |0.27|0.42|0 |0.17 |0.72|2.16|1 |
|RB.AvgPronBl_indefinite |0.65 |1.4 |1.66|0 |1.98 |1.31|0.13|0.27|
|RB.AvgDepsBl_det |0.65 |1.18|0.96|0 |0.43 |0.97|0.92|0.91|
|RB.AvgDepsBl_dobj |0.65 |1.4 |0.87|0 |0.37 |1.26|0 |1.73|
|RB.SynDiv |0.6 |0.64|0.7 |0 |0.42 |0.71|1.55|0.64|
|RB.AvgAOEBl_InflectionPointPolynomial |0.6 |0.88|0.95|0 |2.01 |1.2 |0 |1.09|
|RB.FrqRhythmId |0.59 |1.12|1.27|0 |0.11 |0.77|1.31|0.18|
|RB.LangRhythmDiameter |0.58 |0.18|0.35|0 |0.17 |0.01|2.28|0.82|
|RB.AvgDepsBl_expl |0.58 |0.64|0.28|0 |0.35 |0.2 |1.79|0.82|
|RB.CharEnt |0.57 |1.37|1.01|0 |0.53 |1.35|0.86|0 |
|RB.AvgNounSen |0.55 |0.74|0.91|0 |0.31 |0.46|1.46|0.36|
|RB.AvgDepsBl_amod |0.55 |0.92|0.33|0 |0.1 |0.55|1.08|1.09|
|RB.AvgUnqVerbBl |0.54 |1.56|1.01|0 |0.48 |1.48|0.16|0.36|
|RB.AvgUnqPronounBl |0.54 |1.6 |0.94|0 |1.5 |1.7 |0 |0 |
|RB.AvgPronBl_first_person |0.53 |1.35|0.7 |0 |0.34 |1.23|0.61|0.36|
|RB.AvgConnBl_sentence_linking |0.53 |1.45|1.07|0 |0.41 |1.41|0 |0.64|
|RB.LxcSoph |0.52 |0.32|0.77|0 |0.46 |0.6 |0.1 |2.09|
|RB.AvgAOEBl_IndexPolynomialFitAboveThreshold.0.3.|0.5 |0.94|1.01|0 |0.41 |1.09|0.73|0.27|
|RB.AvgRhythmUnitStreesSyll |0.49 |0.65|0.67|0 |0.45 |0.43|0.75|1 |
|RB.AvgDepsBl_mark |0.47 |1.36|0.95|0 |0.08 |1.19|0 |0.64|
|RB.AvgDepsBl_nmod |0.47 |1.27|0.79|0 |0.43 |1.1 |0 |0.73|
|RB.WdDiffLemmaStem |0.45 |0.64|0.88|0 |0.65 |1.02|0.74|0.18|
|RB.AvgDepsBl_conj |0.45 |0.95|0.52|0 |0.28 |0.64|0.41|0.91|
|RB.AvgAOABl_Bird |0.44 |0.56|0.39|0 |0.58 |0.57|0.96|0.55|
|RB.AvgPronBl_third_person |0.43 |1.36|1.14|0 |0.33 |1.26|0 |0.09|
|RB.AvgDepsBl_ccomp |0.43 |0.93|0.86|0 |0.04 |0.47|1.06|0 |
|RB.AggPronSen_third_person |0.43 |0.52|0.51|0 |0.11 |1.01|1.15|0.18|
|RB.AvgDepsSen_punct |0.43 |0.38|0.62|0 |0.19 |0.94|0.81|0.64|
|RB.AvgConnBl_simple_subordinators |0.42 |1.31|1.02|0 |0.73 |1.08|0 |0.09|
|RB.AvgConnSen_simple_subordinators |0.41 |0.15|0.52|0 |0.09 |0.52|1.54|0.18|
|RB.AvgSenBlCoh_LDA |0.4 |0.74|0.95|0 |0.2 |1.21|0 |0.73|
|RB.AvgDepsBl_xcomp |0.4 |1.15|0.82|0 |0.37 |0.88|0.45|0 |
|RB.AvgCommaBl |0.4 |0.72|0.45|0 |0.05 |0.39|0.96|0.45|
|RB.AvgAOASen_Shock |0.39 |0.4 |0.74|0 |0.45 |0.9 |0.79|0.18|
|RB.AvgSenBlCoh_word2vec |0.36 |1.11|0.78|0 |0.18 |1.03|0 |0.27|
|RB.WdLettStdDev |0.34 |0.72|0.63|0 |0.39 |0.7 |0.46|0.18|
|RB.AvgConnBl_temporal_connectors |0.34 |1.03|0.92|0 |0.02 |0.77|0.1 |0.27|
|RB.AvgDepsBl_acl |0.34 |0.58|0.44|0 |0.07 |0.2 |1.19|0 |
|RB.LangRhythmCoeff |0.33 |0.7 |0.59|0 |0.25 |0.66|0.61|0 |
|RB.WdSylCnt |0.33 |0.38|0.9 |0 |0.27 |0.79|0.26|0.45|
|RB.AvgDepsBl_auxpass |0.33 |0.86|0.55|0 |0.01 |0.5 |0.7 |0 |
|RB.AvgConnBl_oppositions |0.33 |0.85|0.49|0 |0 |0.42|0.63|0.18|
|RB.AvgAdverbBl |0.32 |1.1 |0.72|0 |0.11 |0.83|0 |0.18|
|RB.AvgConnBl_order |0.32 |0.73|0.27|0 |0.01 |0.29|1 |0 |
|RB.AvgAOABl_Bristol |0.31 |0.75|0.29|0 |0.7 |0.77|0.39|0 |
|RB.AvgDepsSen_nmod |0.31 |0.06|0.43|0 |0.11 |0.34|0.45|1 |
|RB.AvgPronounSen |0.31 |0.36|0.7 |0 |0.05 |0.55|0 |1 |
|RB.AvgIntraBlCoh_Path |0.3 |1.14|0.3 |0 |0.16 |0.97|0 |0.18|
|RB.AvgAOABl_Kuperman |0.3 |0.51|0.51|0 |0.71 |0.59|0.1 |0.45|
|RB.AvgDepsSen_nsubj |0.3 |0.05|0.83|0 |0.04 |0.49|0 |1.18|
|RB.AvgDepsSen_aux |0.3 |0.17|0.62|0 |0.21 |0.49|0.68|0.36|
|RB.AvgInferenceDistChain |0.29 |0.8 |0.71|0 |0.19 |0.74|0.25|0 |
|RB.AvgConnBl_conditions |0.29 |0.9 |0.45|0 |0.15 |0.49|0.44|0 |
|RB.AvgDepsBl_cop |0.28 |1.07|0.53|0 |0.08 |0.7 |0 |0.18|
|RB.RdbltyFlesch |0.28 |0.49|0.88|0 |0.38 |0.54|0 |0.45|
|RB.AvgConnSen_temporal_connectors |0.28 |0.28|0.82|0 |0.17 |0.05|0.75|0.18|
|RB.AvgUnqAdjectiveBl |0.27 |1.2 |0.24|0 |0.01 |0.97|0.03|0 |
|RB.AvgDepsBl_advcl |0.27 |1.18|0.37|0 |0.08 |0.9 |0.01|0 |
|RB.AvgDepsSen_advcl |0.27 |0.16|0.53|0 |0.12 |0.69|0.66|0.18|
|RB.AggPronSen_indefinite |0.25 |0.42|0.84|0 |0.26 |1.17|0.03|0 |
|RB.WdDiffWdStem |0.25 |0.65|0.56|0 |0.42 |0.81|0.13|0 |
|RB.AvgDepsBl_neg |0.24 |0.45|0.08|0 |0.02 |0.12|0.9 |0 |
|RB.AvgDepsBl_nummod |0.23 |0.45|0.11|0 |0 |0.12|0.89|0 |
|RB.AvgDepsBl_mwe |0.22 |0.29|0.46|0 |0 |0.06|0.76|0 |
|RB.AvgDepsSen_amod |0.22 |0.3 |0.64|0 |0.32 |0.72|0.22|0 |
|RB.AvgAOASen_Bird |0.21 |0.32|0.74|0 |0.27 |0.42|0.25|0 |
|RB.AvgPrepositionSen |0.21 |0.07|0.66|0 |0.05 |0.35|0 |0.73|
|RB.AvgConnBl_contrasts |0.21 |1.04|0.19|0 |0.01 |0.64|0 |0 |
|RB.AvgAOASen_Kuperman |0.21 |0.5 |0.2 |0 |0.88 |0.48|0 |0.18|
|RB.AvgDepsSen_xcomp |0.21 |0.19|0.71|0 |0.05 |1.01|0.23|0 |
|RB.AvgDepsBl_root |0.2 |0.04|0.29|0 |0.02 |0 |0.99|0 |
|RB.AvgDepsSen_cop |0.2 |0.06|0.49|0 |0.28 |0.36|0.6 |0 |
|RB.AvgConnSen_reason_and_purpose |0.19 |0.14|0.21|0 |0.11 |0.61|0.53|0 |
|RB.AvgDepsSen_conj |0.19 |0.14|0.56|0 |0.02 |0.42|0 |0.55|
|RB.AvgDepsSen_dobj |0.19 |0.06|0.6 |0 |0.23 |0.38|0 |0.55|
|RB.AvgDepsSen_dep |0.19 |0.49|0.57|0 |0.29 |0.65|0 |0 |
|RB.AvgAdverbSen |0.19 |0 |0.72|0 |0.05 |0.87|0 |0.36|
|RB.AvgSenLen |0.18 |0.06|0.76|0 |0.12 |0.29|0 |0.45|
|RB.AvgPronBl_second_person |0.18 |0.7 |0.62|0 |0.01 |0.3 |0 |0 |
|RB.AvgConnBl_disjunctions |0.18 |0.73|0.25|0 |0.01 |0.35|0.16|0 |
|RB.AvgConnBl_reason_and_purpose |0.18 |0.64|0.26|0 |0.03 |0.26|0.28|0 |
|RB.AggPronSen_second_person |0.18 |0.32|0.47|0 |0.01 |0.08|0.53|0 |
|RB.AvgConnBl_semi_coordinators |0.16 |0.35|0.31|0 |0.01 |0.09|0.43|0 |
|RB.AvgPronBl_interrogative |0.16 |0.7 |0.35|0 |0.01 |0.28|0.07|0 |
|RB.AvgAOASen_Bristol |0.15 |0.41|0.39|0 |0.14 |0.58|0 |0 |
|RB.AvgDepsSen_ccomp |0.15 |0.18|0.59|0 |0.11 |0.47|0 |0.18|
|RB.AvgDepsBl_iobj |0.15 |0.62|0.33|0 |0.01 |0.29|0 |0.09|
|RB.AvgDepsSen_det |0.15 |0.21|0.32|0 |0.24 |0.07|0.12|0.36|
|RB.AvgConnSen_addition |0.13 |0.11|0.27|0 |0.75 |0.46|0 |0 |
|RB.AvgDepsSen_acl |0.13 |0.38|0.58|0 |0.11 |0.08|0 |0.09|
|RB.AvgDepsSen_mark |0.12 |0.09|0.35|0 |0.04 |0.36|0 |0.27|
|RB.AvgConnSen_oppositions |0.12 |0.25|0.65|0 |0.08 |0.05|0.13|0 |
|RB.AvgDepsBl_dep |0.11 |0.58|0.04|0 |0.06 |0.19|0.04|0 |
|RB.AvgConnSen_semi_coordinators |0.11 |0.22|0.65|0 |0.23 |0.04|0 |0 |
|RB.AvgConnBl_complex_subordinators |0.11 |0.39|0.19|0 |0 |0.12|0.19|0 |
|RB.AvgAOASen_Cortese |0.11 |0.12|0.25|0 |0.35 |0.64|0 |0 |
|RB.AvgAdjectiveSen |0.1 |0.09|0.4 |0 |0.05 |0.56|0 |0 |
|RB.AvgDepsSen_iobj |0.07 |0.16|0.45|0 |0.02 |0.02|0 |0 |
|RB.AggPronSen_interrogative |0.07 |0.11|0.4 |0 |0.21 |0.01|0 |0 |
|RB.AvgConnSen_order |0.07 |0.02|0.48|0 |0.13 |0 |0 |0.09|
|RB.SenAsson |0.07 |0.24|0.41|0 |0 |0.02|0 |0 |
|RB.AvgConnSen_conditions |0.06 |0.07|0.49|0 |0.08 |0 |0 |0 |
|RB.AvgDepsBl_csubj |0.06 |0.02|0.31|0 |0.06 |0 |0.17|0 |
|RB.AvgDepsSen_neg |0.06 |0.17|0.4 |0 |0.03 |0.03|0 |0 |
|RB.AvgDepsBl_parataxis |0.05 |0.24|0.12|0 |0 |0.04|0 |0 |
|RB.AvgDepsBl_appos |0.04 |0.2 |0.08|0 |0 |0.04|0 |0 |
|RB.AvgDepsSen_nummod |0.04 |0.04|0.35|0 |0.02 |0 |0 |0 |
|RB.AggPronSen_first_person |0.04 |0.02|0.2 |0 |0.14 |0.1 |0 |0 |
|RB.AvgConnSen_disjunctions |0.03 |0.11|0.13|0 |0.04 |0.01|0 |0 |
|RB.SenAllit |0.03 |0.03|0.3 |0 |0 |0 |0 |0 |
|RB.AvgDepsSen_mwe |0.01 |0.07|0 |0 |0 |0.01|0 |0 |
ReaderBench Model 3exp
This model was trained on 15 min expository writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
gbm = stochastic gradient boosted trees
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | rf | mars | gbm | svm | enet | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|:-------|
| -0.0156 | 0.0826 | 0.3112 | 0.0319 | 0.1360 | 0.3306 | 0.1259 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|rf |mars |gbm |svm |enet |cube |
|--------------------------------------|-------|----|-----|-----|----|-----|-----|
|Content.words |20.83 |5.13|35.84|48.17|3.43|17.49|14.66|
|RB.AvgWdLen |4.5 |1.29|10.64|1.06 |0.42|1.6 |4.32 |
|RB.AvgDepsBl_compound |4.11 |0.71|8.06 |0.36 |0.06|3.36 |3.92 |
|RB.AvgConnBl_order |3.63 |0.6 |6.17 |0.15 |0.21|4.43 |1.81 |
|RB.SenStdDevWd |3.56 |1.02|5.2 |0.29 |1.18|4.38 |2.41 |
|RB.LangRhythmId |3.46 |0.96|10.64|0.29 |0.26|0 |0.7 |
|RB.TCorefChainDoc |3.34 |1.93|0 |4.43 |2.48|7.01 |3.51 |
|RB.WdEnt |3.21 |2.23|0 |1.79 |2.35|6.08 |5.52 |
|RB.AggPronSen_first_person |2.93 |0.99|8.76 |0.77 |0.12|0 |1.1 |
|Sentences |2.9 |1.37|0 |2.38 |1.47|6.99 |2.01 |
|RB.AvgSenAdjCoh_Path |2.68 |1.05|0 |1.28 |1.09|5.93 |3.92 |
|RB.CAF |2.51 |1.33|0 |1.24 |1.75|5.87 |1.81 |
|RB.AvgPronBl_third_person |2.49 |0.76|7.52 |0.04 |0.79|0 |0.2 |
|RB.AvgBlScore |2.27 |2.1 |4.33 |1.67 |2.38|0 |3.31 |
|RB.AvgPronBl_second_person |2.03 |1.17|0 |0.64 |0.95|4.73 |2.01 |
|RB.LangRhythmDiameter |1.92 |0.73|2.84 |0.28 |0.15|2.24 |1.91 |
|RB.TActCorefChainWd |1.4 |0.76|0 |0.47 |1.04|2.97 |1.81 |
|RB.TCorefChainBigSpan |1.31 |0.44|0 |1.17 |1.63|2.75 |1 |
|RB.AvgUnqAdjectiveBl |1.06 |1.01|0 |0.23 |1.7 |1.95 |0.9 |
|RB.WdDiffWdStem |0.99 |1.06|0 |1.8 |0.52|2.17 |0.6 |
|RB.AvgDepsSen_nmod |0.94 |0.95|0 |0.52 |1.14|1.67 |1.2 |
|RB.AvgDepsBl_expl |0.89 |0.79|0 |0.42 |0.42|1.86 |1.2 |
|RB.RdbltyDaleChall |0.86 |1.25|0 |0.91 |0.78|1.01 |2.41 |
|RB.AvgAOEBl_InflectionPointPolynomial |0.77 |0.72|0 |0.27 |0.7 |1.85 |0.1 |
|RB.AvgConnBl_temporal_connectors |0.76 |0.91|0 |0.04 |0.5 |1.37 |1.41 |
|RB.AvgPronBl_indefinite |0.75 |2.03|0 |5.56 |1.56|0 |1.61 |
|RB.SynDiv |0.71 |0.61|0 |0.28 |1.04|1.39 |0.5 |
|RB.LxcDiv |0.69 |1.51|0 |1.57 |2.14|0 |1.91 |
|RB.AvgAOASen_Bristol |0.66 |0.56|0 |0.14 |0.31|1.56 |0.5 |
|RB.AvgDepsBl_root |0.65 |0.09|0 |0.04 |0.04|1.95 |0 |
|RB.AvgDepsBl_nsubj |0.62 |1.9 |0 |0.88 |2.19|0 |1.2 |
|RB.AvgPronounBl |0.59 |1.54|0 |0.25 |1.94|0 |1.61 |
|RB.AvgPrepositionBl |0.59 |1.37|0 |1.41 |2.07|0 |1.31 |
|RB.AvgUnqNoundBl |0.49 |0.83|0 |0.41 |1.02|0 |2.21 |
|RB.AvgDepsBl_parataxis |0.47 |0.53|0 |0.01 |0.15|1.26 |0 |
|RB.LangRhythmCoeff |0.44 |0.58|0 |0.33 |0.4 |0.93 |0.2 |
|RB.AvgUnqPrepositionBl |0.43 |0.94|0 |0.2 |2.05|0 |0.6 |
|RB.AvgAOASen_Bird |0.43 |0.63|0 |0.4 |0.79|0.63 |0.5 |
|RB.WdSylCnt |0.42 |0.96|0 |0.54 |0.18|0.5 |1.1 |
|RB.AvgDepsBl_nmod |0.42 |1.01|0 |0.52 |1.59|0 |0.9 |
|RB.AvgChainSpan |0.41 |1.04|0 |0.4 |1.58|0 |0.8 |
|RB.AvgDepsBl_nummod |0.41 |0.7 |0 |0.01 |0.21|1 |0 |
|RB.AvgDepsSen_expl |0.4 |0.41|0 |0.23 |0.06|1.1 |0 |
|RB.AvgPronBl_first_person |0.39 |0.71|0 |0.51 |0.5 |0.25 |1.41 |
|RB.AvgUnqVerbBl |0.38 |0.91|0 |0.06 |1.71|0 |0.6 |
|RB.AvgDepsBl_aux |0.37 |0.59|0 |0.19 |0.93|0.4 |0.5 |
|RB.AvgAdverbBl |0.33 |0.6 |0 |0.11 |1.31|0 |0.8 |
|RB.AvgDepsBl_punct |0.33 |1.26|0 |0.3 |1.17|0 |0.5 |
|RB.AvgNounSen |0.33 |0.99|0 |0.05 |0.22|0 |1.81 |
|RB.LxcSoph |0.32 |0.79|0 |0.3 |0.75|0 |1.2 |
|RB.CharEnt |0.31 |0.49|0 |1.05 |1.09|0.13 |0.4 |
|RB.AvgDepsSen_cop |0.31 |0.86|0 |0.55 |0.55|0 |1.2 |
|RB.AvgDepsBl_mark |0.31 |1.04|0 |0.56 |1.59|0 |0 |
|RB.AvgSenBlCoh_LDA |0.3 |0.82|0 |0.16 |1.15|0 |0.6 |
|RB.RdbltyFlesch |0.29 |0.47|0 |0.19 |0.17|0 |1.81 |
|RB.AvgCorefChain |0.28 |0.76|0 |0.2 |1.05|0 |0.6 |
|RB.AvgDepsBl_dobj |0.28 |0.92|0 |0.09 |1.36|0 |0.2 |
|RB.AvgDepsBl_cop |0.27 |0.59|0 |0.07 |0.97|0 |0.7 |
|RB.AvgDepsBl_det |0.27 |0.92|0 |0.09 |1.36|0 |0.1 |
|RB.AvgDepsSen_mark |0.27 |0.68|0 |0.19 |1.12|0 |0.5 |
|RB.AvgDepsBl_amod |0.26 |0.58|0 |0.27 |1.23|0 |0.3 |
|RB.AvgDepsBl_mwe |0.25 |0.8 |0 |0.09 |0.61|0.3 |0 |
|RB.AvgUnqAdverbBl |0.25 |0.6 |0 |0.03 |1.39|0 |0.1 |
|RB.AvgPrepositionSen |0.24 |0.44|0 |0.16 |0.91|0 |0.6 |
|RB.AvgConnBl_simple_subordinators |0.23 |0.76|0 |0.05 |1.22|0 |0 |
|RB.AvgAOASen_Kuperman |0.23 |0.53|0 |0.51 |0.39|0.2 |0.4 |
|RB.AvgDepsSen_compound |0.23 |1.22|0 |0.33 |0.33|0 |0.6 |
|RB.AvgDepsBl_ccomp |0.22 |0.51|0 |0.05 |0.54|0.2 |0.3 |
|RB.AvgUnqPronounBl |0.22 |0.46|0 |0 |1.33|0 |0 |
|RB.FrqRhythmId |0.22 |0.94|0 |0.3 |0.68|0.06 |0.2 |
|RB.AggPronSen_indefinite |0.22 |0.76|0 |0.37 |0.93|0 |0.2 |
|RB.AvgDepsSen_dobj |0.21 |0.98|0 |0.1 |0.49|0 |0.5 |
|RB.AggPronSen_second_person |0.2 |0.81|0 |0.23 |0.64|0 |0.3 |
|RB.AvgAOADoc_Shock |0.2 |0.98|0 |0.42 |0.82|0 |0 |
|RB.AvgConnSen_semi_coordinators |0.19 |0.59|0 |0.29 |0 |0.38 |0.1 |
|RB.AvgConnBl_addition |0.18 |0.7 |0 |0.23 |0.65|0 |0.2 |
|RB.AvgRhythmUnitStreesSyll |0.18 |0.89|0 |0.17 |0.47|0 |0.3 |
|RB.AvgDepsSen_ccomp |0.18 |0.31|0 |0.22 |0.94|0 |0.2 |
|RB.AvgAdverbSen |0.17 |0.38|0 |0.06 |0.99|0 |0 |
|RB.AvgCommaSen |0.17 |0.62|0 |0.25 |0.8 |0 |0 |
|RB.AvgAOEDoc_IndexAboveThreshold.0.3. |0.17 |0.72|0 |0.12 |0.36|0 |0.5 |
|RB.AvgConnBl_contrasts |0.17 |0.46|0 |0.08 |0.82|0 |0.2 |
|RB.AvgConnSen_simple_subordinators |0.16 |0.44|0 |0.13 |0.88|0 |0 |
|RB.AvgConnBl_reason_and_purpose |0.16 |0.73|0 |0.14 |0.62|0 |0.1 |
|RB.AvgAOADoc_Bird |0.16 |0.79|0 |0.14 |0.68|0 |0 |
|RB.AvgDepsSen_amod |0.16 |0.29|0 |0.25 |0.5 |0 |0.5 |
|RB.AvgConnBl_oppositions |0.16 |0.65|0 |0.05 |0.6 |0.02 |0.2 |
|RB.AvgAOABl_Kuperman |0.15 |0.11|0 |0.18 |0.45|0 |0.6 |
|RB.AvgDepsSen_xcomp |0.15 |0.63|0 |0.06 |0.73|0 |0 |
|RB.AvgPronounSen |0.14 |0.62|0 |0.03 |0.26|0 |0.4 |
|RB.AvgDepsBl_advcl |0.14 |0.21|0 |0.02 |0.89|0 |0 |
|RB.AvgInferenceDistChain |0.14 |0.56|0 |0.2 |0.45|0 |0.2 |
|RB.AvgNounNmdEntBl |0.14 |0.49|0 |0.87 |0.55|0 |0 |
|RB.AggPronSen_third_person |0.14 |0.65|0 |0.14 |0.64|0 |0 |
|RB.WdLettStdDev |0.14 |0.65|0 |0.18 |0.63|0 |0 |
|RB.AvgConnSen_addition |0.13 |0.47|0 |0.23 |0.63|0 |0 |
|RB.AvgNmdEntSen |0.13 |0.18|0 |0.36 |0.81|0 |0 |
|RB.WdDiffLemmaStem |0.12 |0.71|0 |0.26 |0.29|0 |0.1 |
|RB.AvgDepsSen_aux |0.12 |0.4 |0 |0.03 |0.64|0 |0 |
|RB.AvgCommaBl |0.12 |0.66|0 |0.04 |0.4 |0 |0.1 |
|RB.AvgAOASen_Shock |0.12 |0.28|0 |0.05 |0.73|0 |0 |
|RB.AvgDepsBl_acl |0.12 |0.47|0 |0.13 |0.6 |0 |0 |
|RB.AvgAOABl_Cortese |0.12 |0.28|0 |0.1 |0.64|0 |0.1 |
|RB.AvgDepsSen_advcl |0.12 |0.46|0 |0.25 |0.59|0 |0 |
|RB.AvgDepsBl_xcomp |0.12 |0.23|0 |0.09 |0.78|0 |0 |
|RB.AvgConnSen_temporal_connectors |0.11 |0.73|0 |0.06 |0.09|0.06 |0.1 |
|RB.AvgAOESen_InflectionPointPolynomial|0.11 |0.28|0 |0.11 |0.52|0 |0.1 |
|RB.AvgDepsSen_dep |0.11 |0.49|0 |0.17 |0.38|0 |0.1 |
|RB.AvgAOASen_Cortese |0.11 |0.22|0 |0.17 |0.66|0 |0 |
|RB.AvgDepsSen_det |0.11 |0.14|0 |0.12 |0.54|0 |0.2 |
|RB.AvgConnSen_reason_and_purpose |0.11 |0.39|0 |0.12 |0.58|0 |0 |
|RB.AvgAOABl_Bristol |0.1 |0.45|0 |0.15 |0.37|0 |0.1 |
|RB.AvgDepsBl_iobj |0.09 |0.74|0 |0.21 |0.17|0 |0 |
|RB.AvgDepsSen_mwe |0.09 |0.64|0 |0.44 |0.21|0 |0 |
|RB.AvgConnSen_order |0.08 |0.69|0 |0.67 |0.01|0 |0 |
|RB.AvgConnSen_oppositions |0.08 |0.63|0 |0.11 |0.09|0 |0.1 |
|RB.AvgConnBl_disjunctions |0.08 |0.5 |0 |0 |0.32|0 |0 |
|RB.AvgConnSen_contrasts |0.07 |0.6 |0 |0.11 |0.11|0 |0 |
|RB.AvgDepsBl_auxpass |0.07 |0.56|0 |0.01 |0.17|0 |0 |
|RB.AvgDepsSen_neg |0.07 |0.47|0 |0.31 |0 |0 |0.2 |
|RB.AvgConnBl_conditions |0.07 |0.49|0 |0.11 |0.23|0 |0 |
|RB.AvgDepsBl_neg |0.06 |0.19|0 |0.03 |0.23|0 |0.1 |
|RB.AvgPronBl_interrogative |0.06 |0.54|0 |0.04 |0.14|0 |0 |
|RB.SenAsson |0.05 |0.25|0 |0 |0.23|0 |0 |
|RB.AvgConnSen_disjunctions |0.05 |0.55|0 |0.03 |0.05|0 |0 |
|RB.AvgConnBl_semi_coordinators |0.04 |0.16|0 |0.1 |0.16|0 |0 |
|RB.AvgDepsSen_nummod |0.04 |0.46|0 |0.03 |0 |0 |0 |
|RB.AvgDepsSen_acl |0.04 |0.46|0 |0.01 |0.01|0 |0 |
|RB.AvgDepsBl_csubj |0.04 |0.4 |0 |0.01 |0.05|0 |0 |
|RB.AvgDepsBl_nsubjpass |0.04 |0.25|0 |0 |0.16|0 |0 |
|RB.AvgDepsBl_appos |0.04 |0.51|0 |0 |0.02|0 |0 |
|RB.AvgDepsBl_dep |0.02 |0 |0 |0.08 |0.15|0 |0 |
|RB.SenAllit |0.02 |0.3 |0 |0 |0 |0 |0 |
ReaderBench Model 3per
This modelwas trained on 15 min persuasive writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | svm | enet | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|:-------|
| -0.0141 | 0.0326 | 0.2043 | 0.2331 | 0.1507 | 0.3202 | 0.0801 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |mars |gbm |svm |enet |cube|
|--------------------------------------------------|-------|----|-----|-----|----|-----|----|
|RB.WdEnt |9.44 |1.97|0 |16.45|2.58|14.62|8.38|
|RB.AvgPrepositionBl |8.44 |1.96|20.57|11.55|2.48|2.8 |4.83|
|Sentences |6.71 |1.67|19.13|4.09 |1.41|4.14 |5.01|
|RB.AvgBlScore |5.39 |2 |0 |15.75|2.73|2.76 |5.92|
|RB.CAF |4.59 |1.59|19.13|1.27 |1.43|0 |2.73|
|RB.AvgSenScore |3.72 |0.51|8.2 |0.15 |0.47|5.74 |2 |
|RB.TCorefChainDoc |3.36 |1.97|0 |5.74 |2.16|4.69 |2.46|
|RB.AvgWdLen |2.49 |1.32|0 |4.98 |1.13|2.84 |3.28|
|RB.AvgAOADoc_Shock |2.39 |1.4 |8.2 |1.65 |1.21|0.2 |1.18|
|RB.AvgPronBl_indefinite |2.34 |1.76|0 |3.91 |2.01|2.57 |3.73|
|RB.RdbltyDaleChall |2.32 |0.79|0 |1.32 |0.68|5.11 |3.73|
|RB.AvgDepsBl_compound |2.28 |0.23|7.3 |0.29 |0.02|1.88 |2 |
|RB.AvgUnqNoundBl |2.11 |0.75|1.47 |0.3 |1.41|4.17 |2.64|
|RB.AvgConnBl_simple_subordinators |1.75 |1.76|0 |3.08 |1.98|2.11 |0.46|
|RB.AvgAOESen_InflectionPointPolynomial |1.61 |0.53|5.45 |0.22 |0.77|0.99 |0.36|
|RB.AvgPronBl_interrogative |1.23 |0.61|0 |0.53 |0.13|2.92 |2 |
|RB.AvgDepsBl_nsubj |1.12 |1.87|0 |1.87 |2.47|0 |3.37|
|RB.AvgDepsBl_mark |1.1 |1.63|0 |0.89 |1.7 |1.41 |1.91|
|RB.AvgNmdEntSen |1.07 |0.07|0 |0.25 |0.41|2.79 |0.91|
|RB.AvgDepsBl_amod |1.06 |0.82|0 |0.33 |0.41|2.72 |0.64|
|RB.AvgCorefChain |1.04 |0.95|0 |0.08 |1.1 |2.38 |1 |
|RB.AvgDepsSen_advmod |1.01 |0.09|0 |0.22 |0.22|2.68 |1 |
|RB.AvgPronBl_first_person |0.96 |0.7 |2.97 |0.06 |0.27|0.75 |0.64|
|RB.LangRhythmCoeff |0.95 |0.77|0 |1.43 |0.81|1.35 |0.73|
|RB.AvgAOABl_Bird |0.93 |0.47|3.59 |0.46 |0.63|0 |0 |
|RB.AvgDepsSen_aux |0.92 |0 |4 |0.21 |0.18|0 |0.55|
|RB.AvgSenAdjCoh_Path |0.87 |1.19|0 |2.13 |1.26|0.33 |0.73|
|RB.AvgDepsBl_det |0.83 |1.51|0 |0.54 |1.48|1.21 |0.73|
|RB.AvgConnSen_oppositions |0.8 |0.24|0 |0.41 |0.01|2.1 |0.46|
|RB.AvgAOASen_Shock |0.8 |0.62|0 |0.13 |1.02|1.67 |0.91|
|RB.LxcDiv |0.8 |1.45|0 |1.85 |1.4 |0 |1.55|
|RB.AvgUnqPronounBl |0.77 |1.68|0 |0.64 |1.73|0.63 |1.46|
|RB.AvgAOADoc_Cortese |0.69 |0.02|0 |0.3 |0.59|1.5 |0.73|
|RB.AvgUnqAdjectiveBl |0.69 |1.13|0 |0.03 |0.81|1.58 |0.36|
|RB.AvgDepsBl_nsubjpass |0.69 |0.48|0 |0.02 |0.16|2.05 |0.09|
|RB.AvgAOASen_Bird |0.64 |0.52|0 |0.31 |0.39|1.46 |0.46|
|RB.AvgDepsBl_cop |0.63 |1.09|0 |0.1 |0.8 |1.3 |0.55|
|RB.TCorefChainBigSpan |0.6 |1.53|0 |0.34 |1.38|0.62 |1 |
|RB.AvgChainSpan |0.59 |1.42|0 |1 |1.73|0 |0.73|
|RB.AvgDepsBl_aux |0.59 |1.42|0 |0.59 |1.36|0.5 |0.64|
|RB.AvgUnqPrepositionBl |0.58 |1.83|0 |0.66 |2.15|0 |0.64|
|RB.AggPronSen_second_person |0.54 |0.32|0 |0.04 |0.55|1.28 |0.46|
|RB.SynDiv |0.51 |1.15|0 |0.48 |1.14|0.52 |0.46|
|RB.CharEnt |0.49 |1.19|0 |1.21 |1.21|0 |0 |
|RB.AvgAOASen_Bristol |0.48 |0.35|0 |0.13 |0.36|1.1 |0.46|
|RB.AvgDepsBl_punct |0.47 |1.51|0 |0.72 |1.39|0 |0.64|
|RB.AvgDepsBl_nmod |0.46 |1.6 |0 |0.48 |1.72|0 |0.55|
|RB.AvgUnqVerbBl |0.43 |1.51|0 |0.41 |1.47|0 |0.91|
|RB.WdDiffLemmaStem |0.42 |0.86|0 |0.43 |0.9 |0.48 |0.18|
|RB.AvgDepsSen_mark |0.42 |0.28|0 |0.1 |0.29|0.68 |1.73|
|RB.WdDiffWdStem |0.42 |0.67|0 |0.31 |0.66|0.71 |0.18|
|RB.AvgPronounBl |0.41 |1.67|0 |0.06 |1.67|0 |1.18|
|RB.AvgAOASen_Cortese |0.41 |0.08|0 |0.15 |0.3 |0.88 |0.73|
|RB.AvgConnBl_temporal_connectors |0.41 |0.71|0 |0.02 |0.38|1.05 |0 |
|RB.AvgRhythmUnitStreesSyll |0.38 |0.09|0 |0.12 |0.17|0.87 |0.73|
|RB.LxcSoph |0.37 |0.75|0 |0.68 |0.65|0 |1.18|
|RB.AvgDepsBl_ccomp |0.34 |1.38|0 |0.08 |1.26|0.12 |0.64|
|RB.AvgDepsSen_neg |0.34 |0.23|0 |0.09 |0.52|0.74 |0 |
|RB.AvgPronBl_third_person |0.34 |1.34|0 |0.39 |1.16|0 |0.46|
|RB.AvgDepsBl_root |0.33 |0.09|0 |0.06 |0 |1 |0 |
|RB.TActCorefChainWd |0.33 |0.36|0 |0.36 |0.81|0.14 |0.91|
|RB.WdSylCnt |0.3 |0.76|0 |0.54 |0.7 |0 |0.64|
|RB.AvgUnqAdverbBl |0.29 |1.34|0 |0.09 |1.2 |0 |0.64|
|RB.AvgDepsSen_punct |0.27 |0.44|0 |0.09 |0.16|0.68 |0 |
|RB.AvgNmdEntBl |0.26 |1.25|0 |0.11 |1.05|0 |0.55|
|RB.AvgConnBl_addition |0.25 |1.13|0 |0.16 |0.9 |0 |0.55|
|RB.AvgDepsSen_compound |0.25 |0.5 |0 |0.19 |0.56|0 |1.37|
|RB.AggPronSen_indefinite |0.25 |0.42|0 |0.24 |0.9 |0 |0.64|
|RB.AvgDepsBl_dobj |0.25 |1.37|0 |0.02 |1.15|0 |0.46|
|RB.AvgConnBl_order |0.24 |0.56|0 |0.03 |0.21|0.59 |0 |
|RB.AvgAOADoc_Bristol |0.24 |0.8 |0 |0.28 |0.89|0 |0.27|
|RB.SenStdDevWd |0.24 |0.98|0 |0.18 |1.06|0 |0.18|
|RB.FrqRhythmId |0.23 |1.1 |0 |0.02 |0.72|0.21 |0.18|
|RB.AvgDepsBl_advmod |0.23 |1.21|0 |0.12 |0.96|0 |0.27|
|RB.AvgDepsBl_advcl |0.23 |1.36|0 |0.01 |1.26|0 |0 |
|RB.AvgAdverbBl |0.23 |1.25|0 |0.1 |1.01|0 |0.27|
|RB.AvgConnBl_logical_connectors |0.22 |1.14|0 |0.2 |0.89|0 |0.09|
|RB.AvgConnBl_semi_coordinators |0.21 |0.2 |0 |0.02 |0.03|0.56 |0.18|
|RB.AvgPronounSen |0.21 |0.33|0 |0.15 |0.72|0 |0.73|
|RB.AvgUnqNmdEntBl |0.21 |1 |0 |0.18 |0.65|0 |0.55|
|RB.AvgConnSen_simple_subordinators |0.2 |0.46|0 |0.17 |0.74|0 |0.46|
|RB.AvgSenBlCoh_LDA |0.2 |0.59|0 |0.06 |0.91|0.01 |0.36|
|RB.AvgConnBl_reason_and_purpose |0.2 |1.2 |0 |0.09 |0.96|0 |0 |
|RB.AvgDepsSen_amod |0.2 |0.18|0 |0.18 |0.62|0 |0.82|
|RB.AvgInferenceDistChain |0.19 |0.33|0 |0.23 |0.81|0 |0.09|
|RB.AvgAOESen_IndexPolynomialFitAboveThreshold.0.3.|0.19 |0.68|0 |0.32 |0.65|0 |0 |
|RB.AvgSenBlCoh_LSA |0.19 |0.97|0 |0.09 |0.86|0 |0.18|
|RB.AvgAOEDoc_InverseAverage |0.18 |0.62|0 |0.17 |0.82|0 |0 |
|RB.AvgAOEBl_IndexAboveThreshold.0.3. |0.18 |0.59|0 |0.17 |0.71|0.06 |0 |
|RB.SenAllit |0.18 |0.53|0 |0.01 |0.19|0.43 |0 |
|RB.AvgDepsSen_dep |0.18 |0.19|0 |0.2 |0.42|0 |0.91|
|RB.AvgDepsBl_nummod |0.17 |0.66|0 |0.11 |0.32|0.24 |0 |
|RB.AvgDepsSen_det |0.16 |0.21|0 |0.08 |0.89|0 |0 |
|RB.AvgDepsBl_conj |0.16 |1.02|0 |0.11 |0.63|0 |0.09|
|RB.AvgDepsSen_ccomp |0.16 |0.3 |0 |0.15 |0.55|0 |0.46|
|RB.AvgConnSen_addition |0.15 |0.01|0 |0.11 |0.52|0 |0.55|
|RB.AvgDepsSen_acl |0.15 |0.15|0 |0.13 |0.01|0.36 |0 |
|RB.AvgDepsBl_xcomp |0.14 |0.89|0 |0.04 |0.56|0 |0.18|
|RB.AvgPronBl_second_person |0.14 |0.91|0 |0.05 |0.55|0 |0.18|
|RB.AvgAOABl_Kuperman |0.14 |0.08|0 |0.23 |0.45|0 |0.18|
|RB.AvgNounSen |0.14 |0.2 |0 |0.03 |0.22|0 |1.18|
|RB.AvgConnBl_contrasts |0.14 |1.03|0 |0.05 |0.67|0 |0 |
|RB.WdLettStdDev |0.13 |0.6 |0 |0.19 |0.39|0 |0.09|
|RB.AvgDepsBl_neg |0.13 |0.3 |0 |0.03 |0.05|0.33 |0 |
|RB.AvgDepsSen_xcomp |0.13 |0.06|0 |0.05 |0.58|0 |0.36|
|RB.AvgDepsSen_advcl |0.13 |0.14|0 |0.13 |0.66|0 |0 |
|RB.AvgConnBl_oppositions |0.13 |0.98|0 |0.03 |0.54|0 |0.18|
|RB.AggPronSen_first_person |0.13 |0.06|0 |0.22 |0.56|0 |0 |
|RB.AggPronSen_third_person |0.12 |0.38|0 |0.02 |0.69|0 |0 |
|RB.AvgDepsSen_dobj |0.1 |0.07|0 |0.07 |0.31|0 |0.46|
|RB.AvgAdjectiveSen |0.1 |0.04|0 |0.09 |0.4 |0 |0.27|
|RB.AvgDepsSen_cop |0.1 |0.14|0 |0.04 |0.59|0 |0 |
|RB.AvgConnSen_reason_and_purpose |0.09 |0.05|0 |0.06 |0.25|0 |0.46|
|RB.AvgConnBl_conditions |0.09 |0.72|0 |0.04 |0.39|0 |0 |
|RB.LangRhythmDiameter |0.09 |0.29|0 |0.13 |0.06|0.15 |0 |
|RB.AvgDepsBl_acl |0.09 |0.74|0 |0.03 |0.39|0 |0.09|
|RB.AvgAOASen_Kuperman |0.08 |0.09|0 |0.11 |0.26|0 |0.18|
|RB.AvgConnBl_disjunctions |0.08 |0.52|0 |0.03 |0.21|0.08 |0 |
|RB.AvgDepsSen_nmod |0.08 |0.02|0 |0.15 |0.17|0 |0.27|
|RB.AvgCommaBl |0.08 |0.78|0 |0.02 |0.36|0 |0 |
|RB.AvgDepsBl_mwe |0.07 |0.67|0 |0.01 |0.33|0 |0 |
|RB.AvgDepsBl_dep |0.07 |0.64|0 |0.07 |0.22|0 |0.09|
|RB.AvgConnSen_semi_coordinators |0.06 |0.13|0 |0.01 |0.01|0.11 |0.27|
|RB.AvgConnSen_conditions |0.05 |0.01|0 |0.2 |0 |0 |0 |
|RB.AvgConnBl_conjuncts |0.04 |0.42|0 |0.01 |0.14|0 |0 |
|RB.LangRhythmId |0.04 |0.39|0 |0.04 |0.1 |0 |0 |
|RB.AvgDepsBl_csubj |0.03 |0.39|0 |0 |0.09|0 |0 |
|RB.AvgDepsBl_iobj |0.03 |0.29|0 |0.03 |0.09|0 |0 |
|RB.AvgDepsSen_nummod |0.03 |0.13|0 |0.09 |0.01|0 |0 |
|RB.AvgDepsBl_auxpass |0.03 |0.36|0 |0 |0.11|0 |0 |
|RB.AvgDepsBl_expl |0.03 |0.29|0 |0 |0.09|0.03 |0 |
|RB.SenAsson |0.03 |0.37|0 |0.02 |0.1 |0 |0 |
|RB.AvgCommaSen |0.03 |0.14|0 |0.05 |0.02|0 |0.18|
|RB.AvgDepsSen_csubj |0.01 |0.04|0 |0.02 |0 |0 |0 |
|RB.AvgConnSen_disjunctions |0.01 |0.07|0 |0.03 |0.01|0 |0 |
|RB.AvgDepsBl_parataxis |0.01 |0.2 |0 |0 |0.03|0 |0 |
|RB.AvgConnBl_complex_subordinators |0 |0.06|0 |0 |0.01|0 |0 |
|RB.AvgConnSen_temporal_connectors |0 |0.01|0 |0.01 |0 |0 |0 |
Coh-Metrix Model 1 {#cohmetrix-model-1}
General Description
Model 1 has been replaced by the greatly simplified Model 2. Model 2 is recommended for current use.
Coh-Metrix Model 1 is an ensemble (formed by averaging predicted quality scores) of six sub-models that are detailed below.
All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
This scoring model was evaluated in the following publications: [@Matta2022; @Keller-Margulis2021]
Coh-Metrix Model 1a
This model was trained on fall Coh-Metrix scores from data described in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:------|:-------|:------|
| -10.8465 | 0.0266 | 0.1506 | 0.2663 | -0.0302 | 0.296 | 0.2609 | 0.136 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------|
| DESWC | 19.42 | 42.06 | 5.86 | 4.76 | 26.22 | 7.84 | 45.19 | 26.59 |
| DESWLlt | 6.35 | 2.43 | 2.98 | 1.91 | 5.72 | 2.26 | 13.18 | 13.19 |
| LDMTLD | 4.88 | 8.17 | 4.01 | 3.11 | 6.4 | 4.13 | 4.19 | 10.99 |
| PCCONNp | 4.75 | 0.03 | 0 | 0.53 | 1.42 | 0.59 | 18.28 | 0 |
| PCNARp | 3.13 | 0 | 1.76 | 0.8 | 0 | 1.17 | 10.1 | 0 |
| WRDHYPn | 2.82 | 3.15 | 2.9 | 1.83 | 7.62 | 2.16 | 0 | 10.99 |
| PCVERBp | 2.35 | 0 | 0.99 | 0.6 | 0 | 1.23 | 7.35 | 0 |
| DESPL | 1.57 | 0.67 | 3.22 | 1.72 | 2.63 | 1.65 | 1.53 | 0 |
| SYNSTRUTa | 1.39 | 1.21 | 0.85 | 1.12 | 3.49 | 2.74 | 0 | 2.42 |
| PCDCp | 1.28 | 1.12 | 2 | 1.08 | 0 | 2.72 | 0 | 0.88 |
| DESWLsy | 1.26 | 0.48 | 2.09 | 1.23 | 1.3 | 1.34 | 0 | 3.08 |
| CNCTempx | 1.25 | 1.24 | 0.89 | 1.99 | 1.97 | 1.73 | 0 | 1.54 |
| LDTTRa | 1.25 | 0.48 | 2.01 | 0.89 | 2.69 | 1.65 | 0 | 3.08 |
| WRDFRQa | 1.23 | 1.38 | 1.86 | 1.58 | 1.86 | 0.82 | 0 | 3.08 |
| WRDVERB | 1.16 | 1.48 | 1.04 | 0.69 | 3.13 | 1.79 | 0 | 3.08 |
| LSASSpd | 1.11 | 0.1 | 1.95 | 1.43 | 3.46 | 1.39 | 0 | 1.54 |
| CNCTemp | 1.07 | 1.06 | 0.93 | 1.51 | 1.89 | 1.48 | 0 | 1.54 |
| CNCADC | 1 | 0.88 | 1.12 | 1.89 | 0 | 1.61 | 0 | 0 |
| SMINTEp | 0.99 | 1.86 | 0.75 | 1.33 | 3.22 | 1.3 | 0 | 1.54 |
| SMCAUSwn | 0.99 | 0.82 | 1.4 | 1.72 | 0 | 1.64 | 0 | 0 |
| DESWLsyd | 0.98 | 2.02 | 1.97 | 1.03 | 2.25 | 1.54 | 0 | 0.66 |
| CRFCWO1d | 0.97 | 1.03 | 1.67 | 1.5 | 0 | 1.64 | 0 | 0 |
| PCNARz | 0.95 | 0.74 | 1.29 | 0.88 | 3.68 | 1.5 | 0 | 1.54 |
| WRDHYPnv | 0.94 | 0.1 | 1.9 | 0.97 | 0 | 1.23 | 0 | 1.54 |
| WRDPRO | 0.94 | 1.26 | 1.77 | 1.28 | 0 | 1.68 | 0 | 0 |
| DESSLd | 0.93 | 0.15 | 1.67 | 1.09 | 0 | 1.79 | 0.18 | 0 |
| DESWLltd | 0.93 | 1.34 | 2.3 | 1.28 | 1.04 | 1.35 | 0 | 0 |
| DRPP | 0.93 | 0.99 | 2.18 | 1.12 | 0.47 | 1.62 | 0 | 0 |
| CNCLogic | 0.91 | 1.16 | 1.42 | 1.36 | 1.86 | 0.99 | 0 | 1.1 |
| LSAGN | 0.86 | 0.52 | 2.55 | 1.44 | 0 | 0.94 | 0 | 0 |
| PCCONNz | 0.83 | 1.35 | 1.09 | 1.09 | 0 | 1.21 | 0 | 0.88 |
| CRFCWOad | 0.82 | 0.32 | 1.51 | 1.44 | 0 | 1.24 | 0 | 0 |
| WRDADV | 0.81 | 0.07 | 1.3 | 1.24 | 0 | 1.51 | 0 | 0 |
| RDFRE | 0.81 | 0.86 | 1.29 | 0.98 | 2.81 | 1.65 | 0 | 0 |
| LSASS1d | 0.8 | 0.4 | 1.55 | 1.38 | 0.22 | 1.17 | 0 | 0 |
| WRDFRQmc | 0.8 | 1.22 | 1.35 | 0.32 | 0 | 1.92 | 0 | 0.66 |
| LDTTRc | 0.8 | 0.36 | 1.82 | 0.98 | 0.02 | 1.48 | 0 | 0 |
| PCVERBz | 0.8 | 0.11 | 1.55 | 0.87 | 0 | 0.93 | 0 | 1.54 |
| DRNP | 0.77 | 0.25 | 1.73 | 0.66 | 0 | 1.71 | 0 | 0 |
| WRDCNCc | 0.72 | 1.35 | 1.07 | 0.64 | 3.32 | 0 | 0 | 3.08 |
| CRFNOa | 0.72 | 0.4 | 0.74 | 1.33 | 0.63 | 1.25 | 0 | 0 |
| CNCPos | 0.71 | 0.33 | 0.88 | 1.02 | 0.09 | 0.64 | 0 | 1.54 |
| SYNMEDpos | 0.71 | 1.89 | 1.75 | 1.08 | 0 | 0.86 | 0 | 0 |
| LSAGNd | 0.68 | 0.78 | 1.5 | 1.1 | 0.83 | 0.94 | 0 | 0 |
| CNCCaus | 0.63 | 0.09 | 1.11 | 1.17 | 0 | 0.91 | 0 | 0 |
| DRVP | 0.63 | 1.02 | 0.82 | 0.6 | 0.03 | 0.7 | 0 | 1.54 |
| DRNEG | 0.63 | 0.4 | 0.76 | 1.13 | 0.03 | 1.09 | 0 | 0 |
| CRFCWOa | 0.61 | 0.32 | 0.46 | 1.25 | 0 | 0.98 | 0 | 0 |
| RDL2 | 0.61 | 0.3 | 0.31 | 0.87 | 0 | 1.45 | 0 | 0 |
| WRDPOLc | 0.61 | 0.56 | 0.85 | 1.57 | 0 | 0.5 | 0 | 0 |
| WRDIMGc | 0.61 | 0.19 | 0.14 | 0.74 | 0 | 0.87 | 0 | 1.54 |
| SMCAUSr | 0.6 | 0.32 | 1.58 | 0.23 | 0.02 | 1.5 | 0 | 0 |
| WRDFAMc | 0.6 | 1.32 | 1.09 | 0.86 | 1.25 | 0.96 | 0 | 0 |
| LSASSp | 0.59 | 0.14 | 0.79 | 1.05 | 0 | 0.99 | 0 | 0 |
| SMINTEr | 0.59 | 0.58 | 1.83 | 0.36 | 0 | 1.22 | 0 | 0 |
| SMCAUSlsa | 0.59 | 0.48 | 0.24 | 1.01 | 0.69 | 1.25 | 0 | 0 |
| WRDAOAc | 0.59 | 0.24 | 1.66 | 0.94 | 0.16 | 0.74 | 0 | 0 |
| SMCAUSvp | 0.54 | 0.02 | 0.93 | 1.33 | 0 | 0.46 | 0 | 0 |
| DRAP | 0.54 | 0.23 | 0.46 | 0.94 | 0 | 1.05 | 0 | 0 |
| WRDHYPv | 0.54 | 1.15 | 0.28 | 1.17 | 1.97 | 0.75 | 0 | 0 |
| PCTEMPp | 0.53 | 0.38 | 1.05 | 0.38 | 0 | 0.82 | 0 | 0.88 |
| SYNLE | 0.49 | 0.34 | 1.13 | 0.62 | 1.27 | 0.83 | 0 | 0 |
| PCDCz | 0.48 | 0 | 0 | 1.93 | 0 | 0 | 0 | 0 |
| CRFANPa | 0.46 | 0.18 | 0.76 | 0.77 | 0 | 0.78 | 0 | 0 |
| WRDNOUN | 0.44 | 0.54 | 0.56 | 0.53 | 0 | 0.97 | 0 | 0 |
| DESSC | 0.43 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 |
| CNCNeg | 0.43 | 0 | 0 | 1.74 | 0 | 0 | 0 | 0 |
| WRDPRP3s | 0.41 | 0.28 | 0.76 | 0.45 | 1.91 | 0.85 | 0 | 0 |
| PCREFp | 0.4 | 0 | 0.06 | 0.46 | 0 | 1.13 | 0 | 0 |
| PCREFz | 0.39 | 0.33 | 0.47 | 0.43 | 0 | 0.94 | 0 | 0 |
| WRDADJ | 0.38 | 0.08 | 0.99 | 0.59 | 0 | 0.51 | 0 | 0 |
| PCCNCz | 0.38 | 0.26 | 1.18 | 0.32 | 0 | 0.73 | 0 | 0 |
| CRFCWO1 | 0.37 | 0 | 0 | 1.5 | 0 | 0 | 0 | 0 |
| SMCAUSv | 0.35 | 0.38 | 0.39 | 0.53 | 0 | 0.68 | 0 | 0 |
| WRDMEAc | 0.34 | 1.04 | 0.21 | 0.57 | 2.34 | 0.56 | 0 | 0 |
| CNCAdd | 0.34 | 0 | 0 | 1.38 | 0 | 0 | 0 | 0 |
| WRDFRQc | 0.34 | 0.53 | 0.54 | 0.83 | 0 | 0.28 | 0 | 0 |
| PCCNCp | 0.32 | 0 | 0.57 | 0.13 | 0 | 0.93 | 0 | 0 |
| SYNSTRUTt | 0.32 | 0 | 0 | 1.28 | 0 | 0 | 0 | 0 |
| SYNNP | 0.32 | 0.14 | 0.03 | 0.26 | 0 | 1.02 | 0 | 0 |
| LSASS1 | 0.32 | 0 | 0 | 1.3 | 0 | 0 | 0 | 0 |
| CRFSOa | 0.31 | 0 | 0 | 1.25 | 0 | 0 | 0 | 0 |
| PCSYNp | 0.29 | 0.55 | 0.68 | 0 | 0.13 | 0.85 | 0 | 0 |
| SYNMEDlem | 0.28 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 |
| SYNMEDwrd | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 |
| RDFKGL | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 |
| WRDPRP3p | 0.25 | 0.01 | 0.84 | 0.02 | 0 | 0.66 | 0 | 0 |
| CNCAll | 0.22 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 |
| CRFAOa | 0.21 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| PCTEMPz | 0.21 | 0 | 0 | 0.85 | 0 | 0 | 0 | 0 |
| DESSL | 0.15 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 |
| SMTEMP | 0.13 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| CRFAO1 | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 |
| CRFANP1 | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| PCSYNz | 0.04 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 |
| CRFSO1 | 0.03 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 |
| CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1b
This model used Coh-Metrix scores from 7 min narrative writing samples in winter [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:--------|:-------|:-------|:-------|:-------|:-------|
| -9.468 | 0.2532 | -0.0876 | 0.2097 | 0.0554 | 0.2458 | 0.2979 | 0.0974 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------|
| DESWC | 20.79 | 30 | 7.75 | 3.44 | 27.38 | 7.68 | 37.87 | 27.6 |
| LDMTLD | 6.27 | 12.21 | 3.55 | 1.94 | 3.85 | 4.41 | 8.17 | 3.39 |
| DESWLlt | 6.11 | 3.24 | 1.99 | 1.08 | 6.03 | 2.23 | 14.98 | 13.8 |
| CRFCWO1d | 5.2 | 0.54 | 1.57 | 1.71 | 0.35 | 1.98 | 19.51 | 0 |
| LSAGN | 4.28 | 7.08 | 2.73 | 2.24 | 1.41 | 3.97 | 0 | 17.19 |
| RDL2 | 3.21 | 0.71 | 1.62 | 1.3 | 0 | 1.12 | 11.39 | 0 |
| DESPL | 2.56 | 0.19 | 3.82 | 1.84 | 6.29 | 2.53 | 5.18 | 0 |
| SYNLE | 1.48 | 3.13 | 0.87 | 1.41 | 1.45 | 2.12 | 0 | 0.48 |
| LSAGNd | 1.29 | 0.53 | 1.67 | 1.39 | 0 | 1.34 | 0 | 7.02 |
| WRDVERB | 1.28 | 1.66 | 1.91 | 1.32 | 3.02 | 2.19 | 0 | 0 |
| WRDPRP3s | 1.2 | 1.06 | 2.64 | 1.2 | 3.48 | 2.15 | 0 | 0 |
| RDFRE | 1.17 | 1.26 | 2.36 | 0.2 | 0 | 2.28 | 0 | 3.39 |
| WRDIMGc | 1.1 | 1.23 | 0.57 | 0.96 | 3.24 | 1.06 | 0 | 3.39 |
| DESSLd | 1.07 | 0.9 | 0.91 | 1.18 | 0 | 0.93 | 1.95 | 0 |
| LDTTRa | 1.06 | 0.96 | 3.04 | 0.83 | 0 | 1.12 | 0 | 3.39 |
| SYNMEDpos | 1.05 | 0.29 | 2.03 | 1.57 | 0 | 1.54 | 0 | 3.39 |
| WRDNOUN | 1.02 | 0.83 | 1.48 | 1.39 | 5.64 | 1.14 | 0 | 0 |
| PCCNCz | 1.01 | 0.78 | 1.56 | 1.26 | 0 | 1.22 | 0 | 3.39 |
| WRDHYPnv | 1.01 | 1.06 | 0.45 | 1.07 | 1.76 | 1.1 | 0 | 3.39 |
| DESWLltd | 0.95 | 1.21 | 0.88 | 1.2 | 2.76 | 1.52 | 0 | 0 |
| LSASSp | 0.91 | 0.91 | 1.57 | 1.34 | 0 | 1.92 | 0 | 0 |
| WRDHYPv | 0.88 | 0.72 | 1.87 | 1.14 | 2.79 | 1.32 | 0 | 0 |
| PCCNCp | 0.88 | 0 | 1.25 | 1.28 | 1.4 | 1.19 | 0 | 3.39 |
| SMCAUSwn | 0.87 | 1.08 | 1.35 | 1.6 | 2.18 | 0.78 | 0 | 0 |
| LSASS1d | 0.86 | 0.25 | 1.96 | 1.47 | 3.26 | 1.3 | 0 | 0 |
| CNCAdd | 0.84 | 0.74 | 1.13 | 0.8 | 1.85 | 0.53 | 0 | 3.39 |
| SYNNP | 0.82 | 1.25 | 1.32 | 0.72 | 4.14 | 0.68 | 0 | 0 |
| DESWLsy | 0.82 | 0.76 | 0.59 | 1.01 | 0 | 1.33 | 0.84 | 0 |
| DESWLsyd | 0.81 | 1.52 | 0.69 | 1.28 | 0 | 0.96 | 0.13 | 0 |
| WRDFRQmc | 0.75 | 0.84 | 0.92 | 0.99 | 2.76 | 1.03 | 0 | 0 |
| PCVERBz | 0.74 | 0.37 | 1.53 | 1.4 | 0 | 1.59 | 0 | 0 |
| PCREFp | 0.73 | 0 | 0.41 | 0.55 | 5.81 | 0.26 | 0 | 3.39 |
| CRFAOa | 0.72 | 0.22 | 1.88 | 1.32 | 0 | 1.6 | 0 | 0 |
| DRVP | 0.71 | 1.3 | 0.17 | 0.73 | 2.1 | 1.01 | 0 | 0 |
| CRFCWOad | 0.71 | 0.1 | 1.93 | 1.51 | 0 | 1.49 | 0 | 0 |
| SMCAUSv | 0.68 | 1.02 | 1.48 | 0.65 | 0 | 1.26 | 0 | 0 |
| PCSYNz | 0.68 | 0.41 | 1.89 | 0.69 | 0 | 1.78 | 0 | 0 |
| PCNARz | 0.67 | 0.59 | 1.31 | 1.29 | 0 | 1.15 | 0 | 0 |
| CRFCWO1 | 0.67 | 0.42 | 1.72 | 1.38 | 0 | 1.12 | 0 | 0 |
| CRFANPa | 0.66 | 0.22 | 1.25 | 1.25 | 0 | 1.57 | 0 | 0 |
| CRFNOa | 0.66 | 0.63 | 0.71 | 1.5 | 0 | 1.08 | 0 | 0 |
| DRPP | 0.66 | 0.69 | 1.38 | 0.81 | 2.9 | 0.69 | 0 | 0 |
| CNCTemp | 0.65 | 0.55 | 0.28 | 1.47 | 0.68 | 1.12 | 0 | 0 |
| SMINTEr | 0.63 | 0.72 | 1.23 | 0.55 | 0 | 1.56 | 0 | 0 |
| CNCNeg | 0.62 | 1.15 | 0.39 | 0.92 | 0 | 0.95 | 0 | 0 |
| LDTTRc | 0.62 | 0.65 | 1.34 | 1.03 | 0.22 | 0.99 | 0 | 0 |
| WRDADV | 0.62 | 0.62 | 0.79 | 0.93 | 0 | 1.4 | 0 | 0 |
| SMINTEp | 0.61 | 0.13 | 1.26 | 1.31 | 0 | 1.38 | 0 | 0 |
| WRDFRQa | 0.6 | 1.08 | 1.08 | 0.8 | 0 | 0.8 | 0 | 0 |
| SMCAUSlsa | 0.58 | 0.83 | 0.64 | 0.6 | 0 | 1.33 | 0 | 0 |
| SMCAUSvp | 0.58 | 0.33 | 1.47 | 1.12 | 0 | 1.08 | 0 | 0 |
| CNCAll | 0.58 | 0.92 | 1.12 | 0.75 | 1.02 | 0.64 | 0 | 0 |
| SYNSTRUTa | 0.56 | 0.4 | 1.51 | 0.94 | 0 | 1.03 | 0 | 0 |
| WRDADJ | 0.55 | 1.43 | 0.3 | 0.4 | 0.51 | 0.72 | 0 | 0 |
| WRDMEAc | 0.54 | 0.65 | 0.09 | 1.1 | 0 | 1.05 | 0 | 0 |
| DRNP | 0.53 | 0.53 | 1.42 | 0.45 | 0 | 1.25 | 0 | 0 |
| PCTEMPp | 0.52 | 0.51 | 1.21 | 0.8 | 0 | 0.95 | 0 | 0 |
| PCVERBp | 0.52 | 0 | 0.91 | 1.17 | 0 | 1.3 | 0 | 0 |
| SMCAUSr | 0.52 | 0.25 | 1.49 | 0.08 | 0 | 1.87 | 0 | 0 |
| PCSYNp | 0.51 | 0.01 | 1.27 | 0.44 | 0 | 1.82 | 0 | 0 |
| CNCCaus | 0.5 | 0.62 | 0.42 | 0.72 | 0 | 1.12 | 0 | 0 |
| WRDPRO | 0.49 | 0.65 | 0.75 | 0.93 | 0.14 | 0.62 | 0 | 0 |
| WRDAOAc | 0.48 | 0.84 | 0.97 | 0.55 | 0 | 0.7 | 0 | 0 |
| PCNARp | 0.46 | 0 | 1.14 | 1.07 | 0 | 0.96 | 0 | 0 |
| CNCTempx | 0.45 | 0.46 | 0.67 | 0.68 | 0 | 0.99 | 0 | 0 |
| WRDFRQc | 0.42 | 0.49 | 0.25 | 0.78 | 0 | 0.85 | 0 | 0 |
| PCCONNp | 0.39 | 0.8 | 0.87 | 0.29 | 0 | 0.57 | 0 | 0 |
| DRNEG | 0.37 | 0.17 | 0.47 | 0.85 | 0.01 | 0.75 | 0 | 0 |
| WRDPOLc | 0.36 | 0.19 | 0.72 | 0.73 | 0 | 0.72 | 0 | 0 |
| CNCLogic | 0.35 | 0.26 | 0.34 | 0.7 | 1.58 | 0.35 | 0 | 0 |
| DESSC | 0.34 | 0 | 0 | 1.84 | 0 | 0 | 0 | 0 |
| PCREFz | 0.33 | 0.51 | 0.84 | 0.55 | 0 | 0.3 | 0 | 0 |
| SYNMEDwrd | 0.32 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 |
| WRDFAMc | 0.31 | 0.46 | 0 | 0.71 | 0 | 0.42 | 0 | 0 |
| LSASSpd | 0.3 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 |
| PCDCp | 0.29 | 0.25 | 0.78 | 0.71 | 0 | 0.25 | 0 | 0 |
| SYNMEDlem | 0.29 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 |
| WRDHYPn | 0.29 | 0.36 | 1.3 | 0.66 | 0 | 0 | 0 | 0 |
| CRFSOa | 0.28 | 0 | 0 | 1.53 | 0 | 0 | 0 | 0 |
| DRAP | 0.26 | 0.3 | 0.38 | 0.69 | 0 | 0.21 | 0 | 0 |
| LSASS1 | 0.26 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 |
| CRFCWOa | 0.25 | 0 | 0 | 1.37 | 0 | 0 | 0 | 0 |
| WRDCNCc | 0.21 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 |
| SYNSTRUTt | 0.21 | 0 | 0 | 1.13 | 0 | 0 | 0 | 0 |
| PCTEMPz | 0.21 | 0 | 0 | 1.14 | 0 | 0 | 0 | 0 |
| SMTEMP | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 |
| CRFANP1 | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 |
| PCDCz | 0.18 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 |
| WRDPRP3p | 0.17 | 0 | 0.55 | 0 | 0 | 0.71 | 0 | 0 |
| CNCADC | 0.15 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 |
| CNCPos | 0.14 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 |
| PCCONNz | 0.12 | 0 | 0 | 0.65 | 0 | 0 | 0 | 0 |
| CRFAO1 | 0.1 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 |
| DESSL | 0.08 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 |
| RDFKGL | 0.06 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 |
| CRFSO1 | 0.04 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 |
| CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1c
This model used Coh-Metrix scores from 7 min narrative writing samples in spring [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:--------|:-------|:--------|
| -4.8423 | 0.5169 | 0.1348 | 0.6009 | -0.2375 | -0.4134 | 0.4001 | -0.0098 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------|
| DESWC | 20.66 | 36.45 | 5.76 | 2.78 | 21.32 | 6.39 | 47.19 | 16.14 |
| WRDVERB | 5.51 | 3.32 | 1.88 | 1.07 | 0.58 | 2.06 | 22.99 | 2.79 |
| WRDHYPn | 4.27 | 2.98 | 2.38 | 1.13 | 3.13 | 1.75 | 14.74 | 1.28 |
| DESSLd | 3.23 | 1.17 | 0.87 | 1.74 | 3.05 | 2.13 | 10.33 | 0 |
| PCNARp | 2.65 | 2.58 | 2.72 | 1.77 | 10.29 | 2.17 | 0 | 4.99 |
| DESPL | 2.51 | 2.16 | 3.29 | 1.6 | 2.76 | 2.2 | 4.27 | 2.56 |
| DESWLltd | 1.8 | 4.36 | 1.75 | 1.09 | 1.39 | 1.51 | 0 | 6.97 |
| WRDNOUN | 1.59 | 1.69 | 2.13 | 0.8 | 5.92 | 1.42 | 0 | 5.81 |
| WRDFRQmc | 1.59 | 2.27 | 2.04 | 1.46 | 2.86 | 1.51 | 0 | 5.34 |
| DESWLlt | 1.55 | 0.7 | 2.05 | 0.77 | 6.89 | 1.97 | 0 | 3.72 |
| CRFANPa | 1.45 | 1.93 | 1.23 | 1.64 | 0.05 | 2.84 | 0 | 0 |
| LSASS1d | 1.43 | 2.3 | 1.5 | 1.82 | 0.08 | 1.92 | 0 | 0 |
| LDMTLD | 1.43 | 2.6 | 1.79 | 1.39 | 0 | 2.14 | 0 | 0 |
| CRFCWOa | 1.4 | 2.06 | 1.9 | 1.74 | 0 | 2.09 | 0 | 0 |
| WRDHYPv | 1.31 | 1.51 | 2.37 | 0.97 | 2.75 | 1.62 | 0 | 1.63 |
| PCDCz | 1.31 | 2.21 | 1.05 | 1.47 | 0 | 2 | 0 | 1.97 |
| WRDPRP3s | 1.28 | 1.96 | 1.36 | 0.66 | 3.27 | 1.43 | 0 | 0.81 |
| SMCAUSwn | 1.26 | 1.16 | 2.26 | 1.16 | 3.45 | 1.13 | 0 | 2.9 |
| SMCAUSvp | 1.25 | 2.1 | 0.97 | 1.55 | 0 | 1.8 | 0 | 0 |
| SYNSTRUTa | 1.17 | 1.47 | 1.74 | 1.81 | 0 | 1.38 | 0 | 3.72 |
| PCDCp | 1.16 | 0 | 1.81 | 1.28 | 3.29 | 2.05 | 0 | 4.53 |
| LSAGN | 1.12 | 0.71 | 1.72 | 1.81 | 0 | 2.08 | 0 | 2.09 |
| RDL2 | 1.04 | 0.86 | 2.17 | 0.93 | 2.07 | 1.45 | 0 | 1.28 |
| SMCAUSlsa | 1.01 | 0.63 | 1.59 | 1.07 | 3.13 | 0.94 | 0 | 3.72 |
| DRPP | 0.98 | 1.61 | 1.78 | 0.67 | 0.69 | 1.47 | 0 | 1.28 |
| LSAGNd | 0.97 | 0.08 | 2.29 | 2 | 0 | 1.63 | 0 | 0 |
| SYNMEDpos | 0.94 | 0.3 | 1.78 | 1.48 | 0 | 2.09 | 0 | 0.93 |
| CNCTemp | 0.9 | 1.06 | 0.85 | 0.91 | 1.61 | 1.18 | 0 | 0 |
| WRDADV | 0.89 | 1.56 | 1.9 | 0.67 | 0 | 1.4 | 0 | 0 |
| CNCPos | 0.87 | 0.61 | 0.82 | 0.74 | 1.97 | 1.58 | 0 | 2.44 |
| SMCAUSv | 0.86 | 1.16 | 0.91 | 1.27 | 0 | 1.22 | 0 | 0 |
| PCTEMPp | 0.85 | 0.19 | 1.68 | 1.09 | 2.36 | 0.96 | 0 | 2.09 |
| PCVERBz | 0.85 | 0.24 | 1.73 | 1.52 | 0 | 1.6 | 0 | 3.37 |
| LDTTRc | 0.84 | 1.39 | 1.26 | 0.82 | 0 | 1.37 | 0 | 0 |
| PCREFz | 0.78 | 0.25 | 1.2 | 0.56 | 2.5 | 1.41 | 0 | 0 |
| RDFKGL | 0.77 | 0.36 | 2.06 | 0.91 | 0 | 1.86 | 0 | 0 |
| LSASSp | 0.76 | 0.04 | 1.97 | 1.62 | 0 | 1.16 | 0 | 0.93 |
| PCVERBp | 0.75 | 0.3 | 1.09 | 1.28 | 0 | 1.55 | 0 | 0 |
| CRFCWO1d | 0.73 | 0.13 | 1.54 | 1.52 | 0 | 1.18 | 0 | 0 |
| DRNP | 0.7 | 0.54 | 1.62 | 0.83 | 0 | 1.51 | 0 | 0 |
| WRDPRO | 0.69 | 0.48 | 0.73 | 0.6 | 1.85 | 0.96 | 0 | 4.18 |
| SMCAUSr | 0.69 | 0.9 | 0.05 | 0.72 | 0.63 | 1.33 | 0 | 0 |
| WRDAOAc | 0.69 | 0.92 | 0.69 | 0.84 | 0.51 | 0.98 | 0 | 0 |
| LDTTRa | 0.68 | 0.05 | 2.31 | 0.95 | 0 | 1.57 | 0 | 0 |
| WRDCNCc | 0.67 | 0.53 | 1.3 | 0.38 | 3.63 | 0 | 0 | 1.28 |
| PCSYNz | 0.62 | 0.24 | 1.76 | 0.77 | 0 | 1.45 | 0 | 1.28 |
| CNCCaus | 0.61 | 0.72 | 0.73 | 0.73 | 0.12 | 1.1 | 0 | 0 |
| WRDMEAc | 0.57 | 0.51 | 0.68 | 0.45 | 2.11 | 0.46 | 0 | 0 |
| CNCTempx | 0.57 | 0.45 | 0.26 | 1.36 | 0.06 | 0.53 | 0 | 0 |
| DESWLsy | 0.56 | 0.34 | 0.85 | 0.55 | 0.35 | 0.89 | 0.49 | 1.28 |
| WRDPOLc | 0.56 | 0.48 | 0.89 | 0.63 | 0 | 1.31 | 0 | 0 |
| SYNLE | 0.55 | 0.35 | 0.09 | 0.62 | 0 | 1.67 | 0 | 0 |
| PCCNCz | 0.54 | 0.14 | 1.87 | 0.96 | 0 | 0.72 | 0 | 3.25 |
| DRNEG | 0.54 | 0.05 | 0.73 | 0.81 | 1.44 | 0.74 | 0 | 0 |
| WRDFRQa | 0.52 | 0.4 | 0.36 | 0.71 | 0 | 1.2 | 0 | 1.28 |
| WRDFRQc | 0.52 | 0.97 | 0.3 | 0.33 | 0 | 1.09 | 0 | 1.28 |
| DRVP | 0.5 | 0.11 | 0.79 | 0.77 | 0.61 | 0.88 | 0 | 0.81 |
| SYNNP | 0.5 | 0.21 | 0.65 | 0.88 | 0 | 1.04 | 0 | 0 |
| LSASSpd | 0.5 | 0 | 0 | 1.94 | 0 | 0 | 0 | 0 |
| CNCLogic | 0.49 | 0.55 | 0.41 | 0.68 | 0 | 0.9 | 0 | 0 |
| CNCADC | 0.49 | 0.38 | 1.61 | 0.46 | 0.24 | 0.93 | 0 | 0 |
| SYNSTRUTt | 0.48 | 0 | 0 | 1.83 | 0 | 0 | 0 | 0 |
| PCNARz | 0.46 | 0 | 0 | 1.77 | 0 | 0 | 0 | 0 |
| PCCNCp | 0.46 | 0 | 1.15 | 0.87 | 0 | 0.93 | 0 | 0 |
| WRDADJ | 0.45 | 0.29 | 1.12 | 0.72 | 0 | 0.76 | 0 | 0 |
| CRFCWOad | 0.44 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 |
| CRFAOa | 0.43 | 0 | 0 | 1.64 | 0 | 0 | 0 | 0 |
| PCSYNp | 0.43 | 0 | 1.45 | 0.62 | 0 | 1.01 | 0 | 0 |
| DESSC | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 |
| CRFCWO1 | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 |
| SYNMEDwrd | 0.42 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 |
| CRFAO1 | 0.41 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 |
| LSASS1 | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 |
| SYNMEDlem | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 |
| DRAP | 0.4 | 0.23 | 1.23 | 0.64 | 0 | 0.58 | 0 | 0 |
| SMINTEp | 0.38 | 0.36 | 1.06 | 0.68 | 0 | 0.29 | 0 | 0.81 |
| PCTEMPz | 0.38 | 0 | 0 | 1.45 | 0 | 0 | 0 | 0 |
| WRDPRP3p | 0.38 | 0.08 | 0.44 | 0.01 | 1.83 | 0.83 | 0 | 0 |
| SMTEMP | 0.37 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 |
| WRDHYPnv | 0.36 | 0.3 | 0.02 | 0.78 | 0 | 0.5 | 0 | 0 |
| CRFANP1 | 0.35 | 0 | 0 | 1.35 | 0 | 0 | 0 | 0 |
| CRFNOa | 0.34 | 0 | 0 | 1.32 | 0 | 0 | 0 | 0 |
| CRFSOa | 0.3 | 0 | 0 | 1.17 | 0 | 0 | 0 | 0 |
| PCREFp | 0.28 | 0 | 0.32 | 0.32 | 0 | 0.97 | 0 | 0 |
| DESWLsyd | 0.27 | 0.32 | 0.32 | 0.45 | 0 | 0.34 | 0 | 1.28 |
| PCCONNp | 0.26 | 0.07 | 1.18 | 0.05 | 0 | 0.87 | 0 | 0 |
| PCCONNz | 0.26 | 0.18 | 0.53 | 0.23 | 0 | 0.69 | 0 | 0 |
| DESSL | 0.24 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 |
| CRFNO1 | 0.22 | 0 | 0.98 | 0.06 | 0 | 0.81 | 0 | 0 |
| RDFRE | 0.21 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 |
| WRDFAMc | 0.21 | 0.29 | 0 | 0.07 | 1.19 | 0.02 | 0 | 0 |
| SMINTEr | 0.21 | 0.07 | 0.37 | 0.32 | 0 | 0.5 | 0 | 0 |
| CNCNeg | 0.15 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 |
| CNCAll | 0.12 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 |
| CNCAdd | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 |
| WRDIMGc | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 |
| CRFSO1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1d
This model used principal components scores from 7 min narrative writing samples in fall [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:-------|:------|:-------|
| -20.0773 | 0.0971 | 0.7558 | 0.5784 | -0.4401 | -4e-04 | 0.002 | 0.0227 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:------|:------|:------|:----|:-----|:------|:------|:------|
| PC3 | 16.71 | 31.88 | 19.04 | 0 | 8.86 | 20.39 | 26.66 | 17.12 |
| PC5 | 12.28 | 18.78 | 13.53 | 0 | 8.6 | 14.58 | 19.92 | 8.98 |
| PC1 | 11.48 | 6.86 | 17.08 | 0 | 3.49 | 5.37 | 14.52 | 8.98 |
| PC8 | 7.3 | 4.81 | 7.77 | 0 | 7.07 | 4.77 | 9.88 | 8.47 |
| PC9 | 5 | 7.06 | 4.69 | 0 | 4.66 | 7.51 | 11.76 | 8.47 |
| PC4 | 4.73 | 2.76 | 5.98 | 0 | 3.09 | 2.6 | 0 | 7.97 |
| PC11 | 4.57 | 2.64 | 4.52 | 0 | 5.05 | 1.91 | 0 | 8.47 |
| PC7 | 2.91 | 4.92 | 2.9 | 0 | 2.16 | 5.93 | 0 | 8.47 |
| PC34 | 2.6 | 0.6 | 0.87 | 0 | 5.78 | 0.91 | 0 | 7.97 |
| PC14 | 2.37 | 0.66 | 2.18 | 0 | 3.18 | 0.56 | 0 | 2.2 |
| PC16 | 2.3 | 0.66 | 1.87 | 0 | 3.52 | 1.18 | 0 | 1.69 |
| PC21 | 2.27 | 0.65 | 1.5 | 0 | 3.75 | 1.41 | 0 | 6.78 |
| PC10 | 2.26 | 1.45 | 2.34 | 0 | 2.39 | 1.39 | 0 | 1.69 |
| PC30 | 2.07 | 0.38 | 0.88 | 0 | 4.61 | 1.45 | 0 | 0.51 |
| PC15 | 1.81 | 0 | 1.61 | 0 | 2.7 | 1.24 | 0 | 0.51 |
| PC31 | 1.8 | 1.2 | 0.69 | 0 | 3.86 | 1.42 | 0 | 1.19 |
| PC6 | 1.7 | 0.76 | 2.11 | 0 | 1.37 | 1.29 | 0 | 0 |
| PC35 | 1.7 | 0.15 | 0.51 | 0 | 4.07 | 1.11 | 5.51 | 0 |
| PC17 | 1.5 | 0.35 | 1.24 | 0 | 2.35 | 1.14 | 0 | 0 |
| PC12 | 1.4 | 0.88 | 1.45 | 0 | 1.59 | 0.23 | 0 | 0 |
| PC24 | 1.39 | 0.83 | 0.85 | 0 | 2.51 | 1.02 | 0 | 0 |
| PC13 | 1.3 | 0.62 | 1.24 | 0 | 1.66 | 0.98 | 0 | 0 |
| PC22 | 1.27 | 1.34 | 0.81 | 0 | 2.08 | 2.74 | 0 | 0 |
| PC19 | 1.11 | 2.25 | 0.72 | 0 | 1.51 | 4.51 | 0 | 0.51 |
| PC32 | 1.09 | 1.42 | 0.37 | 0 | 2.27 | 0.42 | 0 | 0 |
| PC2 | 0.89 | 1.16 | 1.19 | 0 | 0.33 | 2.11 | 0.87 | 0 |
| PC26 | 0.67 | 0.85 | 0.34 | 0 | 1.2 | 2.9 | 0 | 0 |
| PC33 | 0.64 | 0.61 | 0.17 | 0 | 1.32 | 0.89 | 7.71 | 0 |
| PC18 | 0.63 | 0.17 | 0.5 | 0 | 0.93 | 0.58 | 3.18 | 0 |
| PC29 | 0.6 | 0.71 | 0.24 | 0 | 1.18 | 2.26 | 0 | 0 |
| PC28 | 0.51 | 0.26 | 0.23 | 0 | 1.08 | 0.92 | 0 | 0 |
| PC20 | 0.43 | 0.87 | 0.27 | 0 | 0.63 | 0 | 0 | 0 |
| PC25 | 0.34 | 0.22 | 0.19 | 0 | 0.64 | 0.93 | 0 | 0 |
| PC27 | 0.33 | 0.86 | 0.12 | 0 | 0.52 | 2.07 | 0 | 0 |
| PC23 | 0.04 | 0.37 | 0 | 0 | 0 | 1.27 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components
are displayed.
| Variable | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 |
|:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----|
| SS loadings | 14.67 | 14.32 | 5.99 | 5.95 | 5.14 | 4.89 | 4.76 | 4.01 | 4.00 | 3.01 |
| Proportion Var | 0.15 | 0.15 | 0.06 | 0.06 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 |
| Cumulative Var | 0.15 | 0.30 | 0.36 | 0.42 | 0.47 | 0.53 | 0.57 | 0.62 | 0.66 | 0.69 |
| Proportion Explained | 0.22 | 0.21 | 0.09 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 |
| Cumulative Proportion | 0.22 | 0.43 | 0.52 | 0.61 | 0.69 | 0.76 | 0.83 | 0.90 | 0.95 | 1.00 |
Varimax Rotated Loadings
| Metric | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 |
|:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------|
| DESSC | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 |
| DESWC | 0.06 | 0.11 | 0.34 | 0.23 | -0.09 | -0.06 | -0.07 | 0.1 | 0.78 | -0.03 |
| DESPL | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 |
| DESSL | -0.37 | -0.76 | -0.03 | 0.23 | 0.06 | 0.01 | -0.06 | 0.01 | 0.3 | 0.14 |
| DESSLd | 0.52 | -0.1 | 0.04 | 0.32 | 0.1 | -0.04 | -0.04 | -0.01 | 0.25 | -0.45 |
| DESWLsy | 0.08 | 0.09 | 0.71 | -0.01 | -0.06 | 0.24 | 0.25 | -0.07 | 0.02 | -0.02 |
| DESWLsyd | 0.08 | 0.07 | 0.66 | 0.01 | 0.01 | 0.22 | 0.18 | -0.13 | 0 | 0.13 |
| DESWLlt | 0.07 | 0.29 | 0.71 | 0.12 | 0.11 | 0.05 | 0.19 | 0.11 | 0.01 | -0.08 |
| DESWLltd | 0.22 | 0.24 | 0.69 | 0.08 | 0.06 | 0.21 | -0.09 | -0.06 | 0.02 | 0.2 |
| PCNARz | 0.74 | 0.23 | -0.02 | 0.1 | -0.22 | 0.02 | -0.51 | -0.1 | 0.01 | 0.07 |
| PCNARp | 0.61 | 0.35 | 0.13 | 0.06 | -0.2 | 0.07 | -0.48 | -0.03 | 0.06 | -0.01 |
| PCSYNz | -0.09 | 0.88 | 0.13 | -0.06 | -0.04 | -0.1 | 0.01 | 0.14 | -0.33 | 0.08 |
| PCSYNp | -0.19 | 0.84 | 0.16 | 0.01 | -0.01 | -0.04 | -0.03 | 0.15 | -0.22 | 0.12 |
| PCCNCz | -0.35 | -0.48 | -0.08 | 0.04 | 0.61 | -0.35 | 0.08 | 0.08 | -0.02 | 0.27 |
| PCCNCp | -0.12 | -0.35 | -0.02 | 0.03 | 0.57 | -0.38 | 0.1 | 0.14 | -0.08 | 0.25 |
| PCREFz | 0.71 | -0.34 | -0.25 | -0.08 | 0.02 | 0.04 | -0.03 | 0.14 | 0.07 | 0.48 |
| PCREFp | 0.45 | -0.36 | -0.26 | -0.03 | -0.03 | 0.04 | -0.1 | 0.13 | 0.16 | 0.52 |
| PCDCz | 0.12 | 0.17 | 0.12 | 0.9 | -0.09 | -0.15 | -0.02 | 0.22 | 0.04 | 0.04 |
| PCDCp | 0.12 | 0.13 | 0.14 | 0.85 | -0.14 | -0.14 | 0.02 | 0.16 | -0.01 | 0.08 |
| PCVERBz | -0.63 | -0.44 | -0.46 | 0.05 | -0.06 | 0.12 | 0.2 | 0.21 | 0.12 | 0.05 |
| PCVERBp | -0.47 | -0.21 | -0.5 | 0.06 | -0.01 | 0.13 | 0.21 | 0.3 | 0.11 | -0.08 |
| PCCONNz | 0.04 | 0.09 | 0.24 | 0.09 | -0.02 | 0.87 | -0.06 | -0.06 | 0.05 | 0.05 |
| PCCONNp | -0.08 | 0.01 | 0.02 | 0.01 | 0.01 | 0.81 | 0 | -0.1 | -0.12 | 0.01 |
| PCTEMPz | 0.67 | 0.64 | 0.17 | 0.06 | 0.06 | 0.01 | 0.01 | 0 | 0.01 | -0.28 |
| PCTEMPp | 0.4 | 0.37 | 0.04 | 0.04 | 0.17 | -0.16 | 0.01 | 0 | 0.14 | -0.35 |
| CRFNO1 | 0.61 | -0.16 | 0.11 | -0.12 | 0.06 | 0.06 | 0.49 | 0.2 | 0.17 | 0.09 |
| CRFAO1 | 0.88 | 0.21 | 0.13 | 0.02 | -0.01 | 0.08 | -0.06 | 0.09 | 0 | 0.08 |
| CRFSO1 | 0.64 | -0.13 | 0.12 | -0.06 | 0.04 | 0.01 | 0.51 | 0.18 | 0.19 | 0.01 |
| CRFNOa | 0.64 | -0.21 | 0.11 | -0.12 | 0.1 | 0.06 | 0.48 | 0.16 | 0.13 | 0.08 |
| CRFAOa | 0.91 | 0.21 | 0.07 | 0.06 | 0.06 | 0.06 | -0.09 | 0.01 | -0.07 | 0.06 |
| CRFSOa | 0.65 | -0.16 | 0.12 | -0.07 | 0.09 | 0.01 | 0.53 | 0.15 | 0.13 | 0.02 |
| CRFCWO1 | 0.88 | 0.18 | -0.08 | -0.03 | 0 | 0.07 | -0.06 | 0.14 | -0.01 | 0.2 |
| CRFCWO1d | 0.2 | 0.62 | 0.03 | 0.16 | 0.01 | 0.06 | -0.13 | -0.03 | 0.29 | 0.02 |
| CRFCWOa | 0.9 | 0.14 | -0.11 | -0.03 | 0.04 | 0.04 | -0.08 | 0.07 | -0.1 | 0.14 |
| CRFCWOad | 0.2 | 0.76 | -0.03 | 0.09 | 0.03 | 0.04 | 0.04 | -0.02 | 0.24 | -0.06 |
| CRFANP1 | 0.82 | 0.29 | 0.09 | 0.06 | 0 | 0.02 | -0.22 | -0.02 | -0.02 | 0 |
| CRFANPa | 0.85 | 0.09 | 0.01 | 0.12 | 0.05 | -0.03 | -0.25 | -0.06 | -0.13 | 0.04 |
| LSASS1 | 0.83 | 0.06 | 0.02 | -0.02 | 0.05 | -0.01 | 0.11 | -0.01 | 0.14 | -0.01 |
| LSASS1d | 0.16 | 0.7 | 0.09 | 0.05 | -0.01 | 0.05 | 0.08 | -0.05 | 0.26 | 0.06 |
| LSASSp | 0.85 | 0.01 | 0.03 | 0 | 0.08 | -0.01 | 0.11 | -0.04 | 0.08 | -0.04 |
| LSASSpd | 0.24 | 0.76 | 0.12 | 0.05 | 0 | 0.08 | 0.13 | 0.02 | 0.28 | 0.05 |
| LSAGN | 0.56 | 0.66 | 0.2 | -0.01 | -0.03 | 0 | 0.08 | 0.02 | 0.3 | -0.02 |
| LSAGNd | 0.78 | 0.41 | 0.12 | 0 | 0.08 | 0.03 | 0.07 | -0.02 | 0.08 | -0.15 |
| LDTTRc | -0.06 | -0.01 | 0.07 | -0.11 | 0.02 | 0.03 | -0.35 | -0.41 | -0.59 | 0.03 |
| LDTTRa | -0.16 | 0.12 | 0.12 | -0.07 | -0.11 | 0.11 | -0.04 | -0.17 | -0.76 | -0.16 |
| LDMTLD | -0.08 | 0.2 | 0.52 | 0.21 | -0.08 | 0.15 | -0.17 | 0.08 | 0.09 | -0.19 |
| CNCAll | -0.12 | -0.12 | -0.21 | 0.51 | 0.01 | -0.77 | 0.05 | -0.03 | -0.03 | 0.07 |
| CNCCaus | 0 | 0.01 | 0.08 | 0.86 | -0.02 | 0.08 | 0 | 0.06 | -0.09 | 0.01 |
| CNCLogic | 0.02 | 0 | 0.05 | 0.64 | -0.3 | -0.25 | -0.04 | 0.3 | 0.01 | 0.16 |
| CNCADC | 0.07 | 0.02 | 0.16 | 0.01 | -0.37 | -0.25 | -0.16 | 0.43 | 0.06 | 0.04 |
| CNCTemp | 0.01 | 0.11 | 0.09 | 0.31 | -0.1 | -0.28 | -0.01 | 0.11 | 0.03 | 0.22 |
| CNCTempx | 0.11 | 0.11 | -0.06 | 0.29 | 0.05 | 0.1 | 0.06 | 0 | 0.06 | 0.28 |
| CNCAdd | -0.11 | -0.19 | -0.33 | -0.04 | 0.09 | -0.83 | 0.05 | -0.08 | -0.06 | -0.04 |
| CNCPos | -0.13 | -0.1 | -0.22 | 0.53 | 0.12 | -0.67 | 0.09 | -0.14 | -0.06 | 0.09 |
| CNCNeg | 0.03 | 0.01 | 0.13 | 0.02 | -0.35 | -0.26 | -0.19 | 0.44 | 0.03 | 0.01 |
| SMCAUSv | -0.02 | 0.62 | 0.09 | 0.06 | 0.06 | -0.02 | -0.2 | 0.16 | -0.33 | 0.12 |
| SMCAUSvp | -0.02 | 0.44 | 0.11 | 0.52 | 0.02 | -0.01 | -0.17 | 0.2 | -0.32 | 0.05 |
| SMINTEp | 0.06 | 0.59 | 0.19 | 0.14 | 0.1 | -0.06 | -0.1 | 0.05 | -0.2 | 0.35 |
| SMCAUSr | -0.02 | -0.34 | 0 | 0.72 | -0.07 | 0.07 | 0.02 | 0.05 | 0.27 | -0.14 |
| SMINTEr | -0.07 | -0.31 | 0.01 | 0.72 | -0.05 | 0.1 | 0.06 | 0.01 | 0.26 | -0.13 |
| SMCAUSlsa | 0.03 | 0.03 | -0.29 | -0.15 | -0.03 | 0.34 | 0.35 | 0.48 | 0.1 | -0.02 |
| SMCAUSwn | 0.04 | 0.1 | -0.14 | 0.32 | 0.08 | 0.08 | 0.02 | 0.73 | 0.11 | 0.02 |
| SMTEMP | 0.67 | 0.64 | 0.17 | 0.04 | 0.07 | 0.02 | 0.01 | 0.01 | 0 | -0.26 |
| SYNLE | 0.01 | -0.19 | -0.02 | 0.34 | 0.03 | 0.02 | -0.06 | 0.07 | 0.34 | -0.19 |
| SYNNP | -0.05 | -0.04 | 0.03 | 0 | 0.14 | -0.09 | 0.61 | 0 | 0.1 | -0.32 |
| SYNMEDpos | 0.63 | 0.56 | 0.2 | 0.04 | 0.02 | 0.05 | 0.01 | -0.05 | -0.05 | -0.38 |
| SYNMEDwrd | 0.61 | 0.64 | 0.24 | 0.05 | 0.04 | 0.05 | -0.01 | -0.01 | -0.03 | -0.33 |
| SYNMEDlem | 0.61 | 0.63 | 0.24 | 0.05 | 0.04 | 0.03 | -0.02 | -0.01 | -0.02 | -0.34 |
| SYNSTRUTa | 0.06 | 0.8 | -0.01 | -0.05 | 0.08 | 0.19 | 0.1 | -0.06 | -0.16 | 0.1 |
| SYNSTRUTt | 0.08 | 0.83 | 0.03 | -0.03 | 0.05 | 0.2 | 0.07 | -0.08 | -0.2 | 0.08 |
| DRNP | 0.02 | -0.01 | -0.27 | -0.05 | 0.27 | 0.43 | 0.02 | 0.07 | -0.19 | 0.07 |
| DRVP | 0.15 | -0.03 | 0.14 | -0.08 | -0.09 | 0.18 | -0.63 | 0.14 | 0.02 | -0.02 |
| DRAP | -0.11 | 0.06 | 0.26 | 0.12 | -0.57 | -0.19 | 0.03 | -0.01 | -0.16 | 0.23 |
| DRPP | 0.13 | 0.23 | 0.27 | 0.09 | 0.2 | 0.09 | 0.04 | -0.16 | 0.27 | 0.25 |
| DRNEG | -0.13 | -0.23 | 0.1 | -0.1 | -0.33 | -0.01 | -0.18 | -0.01 | 0.28 | 0.05 |
| WRDNOUN | -0.15 | 0.11 | 0.07 | -0.07 | 0.29 | 0.09 | 0.7 | -0.09 | -0.09 | -0.03 |
| WRDVERB | -0.05 | -0.05 | 0.03 | -0.05 | 0.13 | -0.1 | -0.54 | 0.18 | 0.12 | -0.12 |
| WRDADJ | -0.01 | -0.04 | -0.13 | 0.01 | -0.11 | 0.1 | 0.33 | 0.01 | -0.16 | -0.5 |
| WRDADV | -0.11 | 0.01 | 0.29 | 0.2 | -0.7 | -0.18 | -0.04 | 0.02 | -0.01 | 0.17 |
| WRDPRO | 0.13 | -0.08 | -0.25 | -0.09 | -0.1 | 0.16 | -0.6 | -0.03 | -0.21 | 0.13 |
| WRDPRP3s | -0.14 | -0.02 | -0.1 | 0.13 | -0.01 | 0.02 | -0.17 | 0.12 | 0.05 | -0.17 |
| WRDPRP3p | -0.06 | 0.04 | 0.12 | 0.25 | 0.04 | -0.08 | 0 | -0.01 | 0.13 | -0.1 |
| WRDFRQc | -0.1 | -0.09 | -0.49 | 0.07 | -0.54 | 0.25 | -0.02 | -0.07 | 0.09 | 0.11 |
| WRDFRQa | -0.11 | -0.21 | -0.59 | -0.13 | -0.21 | -0.01 | 0.1 | -0.44 | 0.07 | 0.04 |
| WRDFRQmc | 0.09 | 0.64 | 0.01 | 0.13 | -0.03 | 0.07 | -0.16 | 0.25 | 0 | -0.08 |
| WRDAOAc | 0.1 | -0.02 | 0.39 | 0.12 | 0.03 | 0 | -0.19 | 0.27 | 0.11 | -0.04 |
| WRDFAMc | 0 | 0.02 | -0.28 | 0.15 | -0.19 | 0.19 | 0.02 | -0.15 | 0.31 | 0.18 |
| WRDCNCc | 0.09 | 0.1 | 0.24 | -0.08 | 0.77 | -0.05 | 0.11 | 0.15 | 0.14 | 0.15 |
| WRDIMGc | 0.09 | 0.04 | 0.22 | -0.14 | 0.86 | -0.03 | 0.12 | 0.06 | 0.03 | 0.02 |
| WRDMEAc | 0.06 | 0.04 | 0.22 | 0.03 | 0.73 | -0.04 | 0.03 | -0.06 | -0.09 | 0.06 |
| WRDPOLc | -0.01 | 0.08 | -0.23 | 0.32 | -0.03 | -0.03 | -0.13 | 0.66 | -0.04 | -0.05 |
| WRDHYPn | 0.12 | 0.15 | 0.38 | 0.13 | 0.13 | 0 | -0.1 | 0.66 | 0.13 | -0.02 |
| WRDHYPv | 0.01 | -0.01 | 0.02 | 0.14 | 0.27 | -0.19 | -0.22 | 0.39 | -0.03 | 0.26 |
| WRDHYPnv | 0.01 | 0.08 | 0.37 | 0.05 | 0.38 | 0.07 | 0.22 | 0.6 | 0.09 | 0.06 |
| RDFRE | 0.35 | 0.74 | -0.17 | -0.23 | -0.05 | -0.08 | -0.02 | 0.02 | -0.29 | -0.13 |
| RDFKGL | -0.36 | -0.76 | 0.04 | 0.23 | 0.06 | 0.03 | -0.03 | 0.01 | 0.3 | 0.14 |
| RDL2 | 0.5 | 0.5 | -0.34 | -0.01 | -0.27 | 0.29 | 0.01 | 0.01 | -0.04 | 0.24 |
Coh-Metrix Model 1e
This model used principal components scores from 7 min narrative writing samples in winter [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:------|:------|:-------|:--------|:-------|:-------|:--------|
| -10.9566 | 0.338 | 0.385 | 0.5608 | -0.2444 | 0.0443 | 0.0047 | -0.0256 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:------|:------|:------|:----|:-----|:------|:------|:------|
| PC1 | 15.14 | 27.79 | 9.8 | 0 | 4.09 | 18.64 | 24.42 | 21.1 |
| PC4 | 9.3 | 10.16 | 11.41 | 0 | 4.87 | 7.73 | 13.26 | 10.85 |
| PC7 | 8.4 | 8.15 | 9.97 | 0 | 6.48 | 6.21 | 19.43 | 10.26 |
| PC11 | 6.67 | 7.23 | 6.54 | 0 | 5.69 | 5.83 | 11.19 | 10.26 |
| PC9 | 6.08 | 3.67 | 9.2 | 0 | 6.76 | 2 | 1.16 | 1.97 |
| PC10 | 5.49 | 3.61 | 7.03 | 0 | 5.42 | 3.37 | 8.76 | 9.47 |
| PC28 | 4.63 | 2.51 | 3.16 | 0 | 9.22 | 3.54 | 0 | 10.26 |
| PC16 | 3.93 | 2.03 | 4.15 | 0 | 5.33 | 2.65 | 0 | 10.26 |
| PC12 | 3.92 | 2.11 | 4.88 | 0 | 4.59 | 1.8 | 0 | 8.88 |
| PC33 | 3.09 | 2.27 | 1.71 | 0 | 7.03 | 2.57 | 0 | 1.97 |
| PC17 | 2.46 | 1.81 | 2.53 | 0 | 3.42 | 2.91 | 0 | 0.59 |
| PC31 | 2.32 | 2.4 | 1.32 | 0 | 4.7 | 1.59 | 0 | 0 |
| PC20 | 2.3 | 4.01 | 1.31 | 0 | 1.99 | 2.63 | 0 | 0 |
| PC18 | 2.29 | 1.13 | 2.55 | 0 | 3.68 | 2.3 | 16.31 | 0.59 |
| PC14 | 2.21 | 2.99 | 1.75 | 0 | 1.83 | 3.19 | 0 | 0.79 |
| PC22 | 2.1 | 0.93 | 2.13 | 0 | 3.83 | 2.78 | 0 | 0 |
| PC25 | 2.05 | 0.96 | 2.05 | 0 | 4.55 | 0 | 0 | 1.38 |
| PC19 | 2.02 | 1.4 | 2.05 | 0 | 2.95 | 2.31 | 0 | 0.99 |
| PC6 | 1.88 | 2.76 | 1.76 | 0 | 0.89 | 2.87 | 0 | 0 |
| PC3 | 1.79 | 0.55 | 3.72 | 0 | 0.94 | 1 | 0 | 0 |
| PC2 | 1.49 | 0.77 | 2.52 | 0 | 0.49 | 2.87 | 0.24 | 0 |
| PC15 | 1.38 | 0 | 2.17 | 0 | 2.43 | 0.98 | 0 | 0 |
| PC8 | 1.38 | 1.43 | 1.51 | 0 | 0.9 | 2.69 | 0 | 0 |
| PC13 | 1.31 | 1.74 | 1.23 | 0 | 1.17 | 1.25 | 0 | 0 |
| PC26 | 1.13 | 1.03 | 0.71 | 0 | 1.66 | 2.56 | 0 | 0 |
| PC32 | 1.03 | 2.12 | 0.2 | 0 | 0.73 | 2.06 | 0 | 0 |
| PC29 | 0.93 | 0.43 | 0.75 | 0 | 2.25 | 0.69 | 0 | 0 |
| PC34 | 0.68 | 1.1 | 0.12 | 0 | 0.53 | 2.38 | 0 | 0 |
| PC21 | 0.58 | 0.44 | 0.44 | 0 | 0.59 | 1.47 | 5.24 | 0.39 |
| PC5 | 0.5 | 0.16 | 0.73 | 0 | 0.16 | 1.73 | 0 | 0 |
| PC27 | 0.44 | 0.88 | 0.15 | 0 | 0.22 | 0.95 | 0 | 0 |
| PC23 | 0.37 | 0.46 | 0.17 | 0 | 0.16 | 1.67 | 0 | 0 |
| PC24 | 0.36 | 0.15 | 0.28 | 0 | 0.43 | 1.4 | 0 | 0 |
| PC30 | 0.36 | 0.82 | 0 | 0 | 0 | 1.41 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components
are displayed.
| Variable | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 |
|:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----|
| SS loadings | 17.16 | 14.47 | 6.31 | 5.42 | 5.10 | 5.09 | 4.07 | 3.84 | 3.40 | 3.20 |
| Proportion Var | 0.18 | 0.15 | 0.07 | 0.06 | 0.05 | 0.05 | 0.04 | 0.04 | 0.04 | 0.03 |
| Cumulative Var | 0.18 | 0.33 | 0.39 | 0.45 | 0.50 | 0.55 | 0.59 | 0.63 | 0.67 | 0.70 |
| Proportion Explained | 0.25 | 0.21 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 | 0.05 |
| Cumulative Proportion | 0.25 | 0.46 | 0.56 | 0.64 | 0.71 | 0.79 | 0.85 | 0.90 | 0.95 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 |
|:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------|
| DESSC | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 |
| DESWC | 0.18 | 0.16 | -0.09 | 0.21 | 0.05 | -0.09 | 0.19 | 0.21 | 0.63 | 0.38 |
| DESPL | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 |
| DESSL | -0.81 | -0.25 | -0.04 | 0.15 | 0 | -0.07 | 0.1 | 0.12 | 0.34 | 0.01 |
| DESSLd | -0.04 | 0.46 | -0.08 | 0.14 | 0.1 | -0.15 | 0.07 | 0.17 | 0.01 | 0.51 |
| DESWLsy | 0.14 | 0.21 | 0.02 | -0.1 | -0.16 | 0.84 | -0.05 | -0.06 | -0.06 | -0.05 |
| DESWLsyd | 0.13 | 0.17 | -0.07 | -0.04 | -0.14 | 0.79 | -0.08 | 0.03 | -0.14 | -0.04 |
| DESWLlt | 0.22 | 0.12 | 0.21 | -0.04 | -0.01 | 0.83 | -0.1 | 0.02 | 0.02 | -0.05 |
| DESWLltd | 0.18 | 0.13 | -0.05 | -0.07 | -0.17 | 0.67 | -0.11 | 0.11 | -0.13 | -0.09 |
| PCNARz | 0.33 | 0.64 | -0.37 | 0.18 | 0 | -0.25 | 0.34 | 0.28 | -0.03 | 0.05 |
| PCNARp | 0.42 | 0.53 | -0.34 | 0.22 | -0.04 | -0.17 | 0.23 | 0.28 | 0 | 0.04 |
| PCSYNz | 0.9 | -0.07 | 0.06 | -0.01 | 0.11 | 0.14 | -0.05 | -0.09 | -0.14 | -0.18 |
| PCSYNp | 0.85 | -0.16 | 0.07 | 0.14 | 0.08 | 0.17 | -0.04 | -0.07 | 0.01 | -0.15 |
| PCCNCz | -0.56 | -0.3 | 0.65 | -0.14 | 0.17 | -0.11 | -0.1 | 0.03 | -0.04 | -0.19 |
| PCCNCp | -0.42 | -0.11 | 0.69 | 0.05 | 0.09 | -0.03 | -0.06 | 0.13 | 0.09 | -0.16 |
| PCREFz | -0.36 | 0.73 | -0.06 | -0.07 | -0.02 | -0.19 | -0.04 | 0.08 | 0.16 | -0.46 |
| PCREFp | -0.42 | 0.45 | -0.14 | -0.17 | -0.08 | -0.26 | 0 | 0.07 | 0.18 | -0.46 |
| PCDCz | 0.14 | 0.24 | -0.15 | 0.86 | 0.18 | 0.01 | 0.13 | 0.04 | 0.15 | 0.06 |
| PCDCp | 0.19 | 0.2 | -0.14 | 0.82 | 0.21 | 0.02 | 0.09 | -0.01 | 0.09 | 0.03 |
| PCVERBz | -0.56 | -0.65 | -0.12 | 0.01 | -0.04 | -0.12 | -0.39 | 0.14 | 0.05 | 0 |
| PCVERBp | -0.33 | -0.56 | -0.15 | 0.09 | 0.05 | -0.14 | -0.51 | 0.17 | -0.07 | 0.13 |
| PCCONNz | 0.05 | -0.04 | -0.1 | 0.13 | -0.93 | 0.06 | -0.11 | -0.02 | 0.07 | -0.03 |
| PCCONNp | 0.04 | -0.03 | -0.02 | -0.08 | -0.75 | 0 | -0.13 | -0.09 | -0.13 | -0.04 |
| PCTEMPz | 0.68 | 0.62 | -0.03 | 0.12 | 0.05 | 0.15 | 0 | 0.11 | -0.07 | 0.25 |
| PCTEMPp | 0.42 | 0.46 | 0.13 | 0.08 | 0.04 | 0.03 | 0.02 | 0.21 | -0.11 | 0.17 |
| CRFNO1 | -0.11 | 0.73 | 0.26 | 0.11 | -0.01 | 0.12 | -0.16 | -0.08 | 0.1 | 0.08 |
| CRFAO1 | 0.33 | 0.86 | -0.07 | 0.05 | 0.01 | 0.1 | 0.01 | 0.09 | -0.03 | -0.04 |
| CRFSO1 | -0.1 | 0.77 | 0.27 | 0.09 | -0.03 | 0.16 | -0.1 | -0.12 | 0.12 | 0.14 |
| CRFNOa | -0.16 | 0.77 | 0.25 | 0.07 | 0.05 | 0.07 | -0.12 | -0.07 | 0.13 | 0.12 |
| CRFAOa | 0.31 | 0.88 | -0.08 | 0.03 | 0.01 | 0.1 | 0.03 | 0.1 | -0.06 | -0.01 |
| CRFSOa | -0.14 | 0.78 | 0.25 | 0.08 | 0.02 | 0.13 | -0.1 | -0.11 | 0.11 | 0.14 |
| CRFCWO1 | 0.24 | 0.86 | -0.11 | 0.03 | 0.05 | 0.05 | -0.05 | 0.08 | 0 | -0.19 |
| CRFCWO1d | 0.8 | 0.14 | -0.04 | 0.01 | -0.06 | 0.1 | 0.01 | 0.07 | 0.14 | 0.03 |
| CRFCWOa | 0.21 | 0.89 | -0.14 | 0.01 | 0.06 | 0.01 | -0.02 | 0.09 | -0.01 | -0.16 |
| CRFCWOad | 0.82 | 0.2 | -0.03 | -0.02 | -0.07 | 0.02 | 0.05 | 0.09 | 0.17 | 0.13 |
| CRFANP1 | 0.47 | 0.66 | -0.19 | 0.05 | 0.05 | 0.06 | 0.02 | 0.18 | -0.04 | -0.07 |
| CRFANPa | 0.28 | 0.71 | -0.21 | 0.03 | 0.04 | 0.11 | 0.03 | 0.18 | -0.16 | -0.07 |
| LSASS1 | 0.22 | 0.83 | 0.09 | 0 | -0.01 | 0.04 | 0.03 | 0.04 | 0.17 | 0.03 |
| LSASS1d | 0.71 | 0.21 | 0.01 | -0.04 | -0.09 | 0.03 | 0.04 | 0.08 | 0.26 | 0.14 |
| LSASSp | 0.17 | 0.87 | 0.06 | -0.01 | 0.04 | 0 | 0.06 | 0 | 0.17 | 0.02 |
| LSASSpd | 0.72 | 0.28 | 0.04 | -0.08 | -0.09 | 0.07 | 0.06 | 0.05 | 0.32 | 0.1 |
| LSAGN | 0.75 | 0.52 | 0.02 | -0.01 | -0.02 | 0.04 | 0.1 | 0.06 | 0.28 | 0.06 |
| LSAGNd | 0.48 | 0.78 | 0.02 | 0.03 | 0.03 | 0.05 | 0.06 | 0.05 | 0.16 | 0.14 |
| LDTTRc | -0.07 | -0.35 | 0.08 | -0.11 | -0.11 | 0.09 | 0.1 | 0.03 | -0.74 | 0.19 |
| LDTTRa | 0.05 | -0.3 | 0.3 | -0.02 | -0.14 | 0.4 | 0.06 | -0.1 | -0.66 | 0.04 |
| LDMTLD | 0.24 | -0.14 | 0.17 | 0.28 | -0.11 | 0.51 | 0.27 | 0.03 | 0.02 | 0.41 |
| CNCAll | -0.12 | 0.07 | -0.06 | 0.32 | 0.86 | -0.16 | 0.08 | -0.15 | 0.02 | -0.07 |
| CNCCaus | 0.03 | 0.08 | 0.03 | 0.84 | -0.1 | -0.02 | -0.14 | 0.04 | -0.06 | 0.06 |
| CNCLogic | -0.07 | -0.09 | -0.31 | 0.62 | 0.35 | -0.01 | 0.33 | -0.07 | 0.19 | -0.13 |
| CNCADC | -0.04 | -0.12 | -0.05 | 0.1 | 0.36 | 0.13 | 0.52 | 0.33 | -0.02 | 0.06 |
| CNCTemp | 0.14 | 0.09 | -0.31 | 0.08 | 0.34 | -0.01 | 0.32 | -0.21 | 0.2 | -0.23 |
| CNCTempx | 0.15 | -0.11 | 0.03 | 0.39 | 0.07 | 0.02 | 0.05 | -0.02 | -0.22 | -0.1 |
| CNCAdd | -0.14 | 0.01 | 0.04 | -0.19 | 0.89 | -0.14 | -0.04 | -0.08 | -0.04 | -0.01 |
| CNCPos | -0.08 | 0.11 | -0.05 | 0.33 | 0.73 | -0.2 | -0.16 | -0.26 | 0 | -0.1 |
| CNCNeg | -0.07 | -0.13 | -0.11 | 0.03 | 0.37 | 0.11 | 0.55 | 0.31 | 0.02 | 0.01 |
| SMCAUSv | 0.64 | 0.04 | 0.07 | 0.05 | -0.09 | 0.14 | -0.05 | 0.06 | -0.15 | -0.33 |
| SMCAUSvp | 0.49 | 0.12 | 0.11 | 0.55 | -0.14 | 0.06 | -0.03 | 0.09 | -0.06 | -0.17 |
| SMINTEp | 0.66 | 0.08 | 0.13 | 0.05 | -0.18 | 0.11 | 0.09 | 0.03 | 0.12 | -0.27 |
| SMCAUSr | -0.36 | 0.16 | -0.05 | 0.58 | -0.13 | -0.13 | 0.05 | 0.08 | 0.18 | 0.3 |
| SMINTEr | -0.43 | 0.12 | -0.08 | 0.68 | -0.06 | -0.07 | -0.06 | 0.04 | 0.12 | 0.25 |
| SMCAUSlsa | -0.01 | 0.06 | -0.17 | 0.13 | 0.12 | 0.04 | -0.54 | 0.07 | 0.29 | -0.01 |
| SMCAUSwn | 0.28 | 0.09 | -0.01 | 0.23 | 0.02 | 0.22 | -0.35 | 0.54 | 0.22 | 0.22 |
| SMTEMP | 0.69 | 0.62 | -0.03 | 0.11 | 0.04 | 0.15 | -0.01 | 0.11 | -0.07 | 0.24 |
| SYNLE | -0.13 | -0.11 | -0.1 | -0.14 | 0.05 | 0.15 | 0.11 | 0.03 | 0.18 | -0.06 |
| SYNNP | 0 | -0.05 | 0.29 | -0.17 | 0 | 0.38 | -0.21 | -0.5 | 0.09 | 0.35 |
| SYNMEDpos | 0.66 | 0.51 | -0.06 | 0.11 | 0.04 | 0.13 | 0.05 | 0.1 | -0.11 | 0.4 |
| SYNMEDwrd | 0.71 | 0.53 | -0.04 | 0.12 | 0.03 | 0.17 | 0.01 | 0.09 | -0.07 | 0.35 |
| SYNMEDlem | 0.7 | 0.54 | -0.04 | 0.12 | 0.03 | 0.16 | 0.02 | 0.09 | -0.08 | 0.35 |
| SYNSTRUTa | 0.81 | 0.14 | -0.08 | 0.08 | -0.12 | 0.17 | -0.06 | 0.05 | -0.05 | -0.13 |
| SYNSTRUTt | 0.85 | 0.15 | -0.05 | 0.06 | -0.12 | 0.16 | -0.05 | 0.02 | -0.04 | -0.15 |
| DRNP | -0.04 | -0.1 | 0.15 | 0.02 | -0.43 | -0.03 | -0.26 | -0.02 | -0.12 | -0.28 |
| DRVP | -0.05 | 0.07 | -0.08 | -0.14 | -0.05 | -0.14 | 0 | 0.65 | 0 | -0.02 |
| DRAP | 0.09 | 0 | -0.21 | 0.22 | 0.26 | -0.09 | 0.42 | -0.01 | 0.06 | -0.36 |
| DRPP | 0.11 | -0.02 | 0.05 | 0.09 | -0.01 | 0.06 | -0.45 | -0.03 | 0 | 0.05 |
| DRNEG | 0.03 | 0.03 | -0.14 | 0.08 | 0.1 | -0.15 | 0.53 | -0.11 | 0.03 | 0.09 |
| WRDNOUN | 0.04 | -0.06 | 0.49 | -0.12 | -0.16 | 0.28 | -0.41 | -0.4 | 0.02 | 0.06 |
| WRDVERB | 0.17 | 0.17 | 0.05 | -0.13 | -0.06 | 0 | 0.17 | 0.65 | -0.12 | 0.03 |
| WRDADJ | 0.05 | 0.01 | -0.09 | 0.04 | -0.06 | 0.37 | 0.01 | -0.26 | -0.03 | 0.29 |
| WRDADV | 0.04 | 0.04 | -0.24 | 0.38 | 0.29 | 0 | 0.47 | -0.08 | 0.22 | -0.19 |
| WRDPRO | -0.11 | 0.01 | -0.27 | -0.03 | -0.23 | -0.54 | 0.37 | 0.2 | -0.13 | -0.19 |
| WRDPRP3s | 0.05 | 0.01 | -0.09 | 0.14 | -0.03 | -0.03 | 0.45 | -0.03 | 0.1 | 0.22 |
| WRDPRP3p | -0.23 | -0.17 | -0.12 | 0.23 | 0.04 | 0.13 | -0.02 | 0.04 | 0.19 | 0.04 |
| WRDFRQc | -0.12 | -0.25 | -0.58 | 0.26 | -0.15 | -0.24 | -0.05 | 0.31 | 0.05 | 0.06 |
| WRDFRQa | -0.23 | -0.19 | -0.49 | -0.11 | 0.06 | -0.47 | -0.23 | -0.06 | -0.26 | 0.07 |
| WRDFRQmc | 0.66 | -0.02 | -0.21 | 0.05 | 0.11 | 0.1 | -0.08 | 0.15 | -0.07 | 0.18 |
| WRDAOAc | 0.12 | 0.14 | 0.17 | 0.16 | 0.04 | 0.32 | 0.03 | -0.09 | 0.07 | 0.13 |
| WRDFAMc | -0.09 | -0.12 | -0.47 | 0.22 | -0.18 | -0.23 | -0.08 | 0.22 | 0.12 | -0.06 |
| WRDCNCc | 0.03 | 0.02 | 0.83 | -0.12 | 0.04 | -0.03 | -0.06 | 0.02 | -0.09 | 0.05 |
| WRDIMGc | 0.08 | 0.07 | 0.88 | -0.06 | -0.05 | 0 | -0.11 | -0.02 | -0.15 | 0.03 |
| WRDMEAc | 0.06 | 0.04 | 0.71 | 0.06 | -0.19 | 0.03 | -0.13 | 0.17 | -0.2 | 0.03 |
| WRDPOLc | 0.05 | 0.15 | 0.08 | 0.3 | -0.06 | 0.01 | -0.27 | 0.67 | 0.18 | 0.11 |
| WRDHYPn | 0.24 | 0.16 | 0.25 | 0.27 | -0.11 | -0.07 | 0.07 | 0.3 | 0.17 | -0.18 |
| WRDHYPv | 0.21 | 0 | 0.23 | 0.1 | -0.12 | 0.2 | -0.02 | 0.54 | 0.31 | -0.01 |
| WRDHYPnv | 0.13 | 0.08 | 0.61 | 0.11 | -0.23 | 0.22 | -0.34 | 0.12 | 0.24 | -0.05 |
| RDFRE | 0.82 | 0.19 | 0.04 | -0.11 | 0.05 | -0.19 | -0.09 | -0.1 | -0.32 | 0 |
| RDFKGL | -0.81 | -0.23 | -0.04 | 0.15 | -0.02 | 0.02 | 0.1 | 0.11 | 0.34 | 0.01 |
| RDL2 | 0.44 | 0.48 | -0.44 | 0.2 | -0.1 | -0.03 | -0.09 | 0.25 | 0.01 | -0.15 |
Coh-Metrix Model 1f
This model used principal component scores from 7 min narrative writing samples in spring [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
gbm = stochastic gradient boosted trees
pls = partial least squares regression
svm = support vector machines
enet = elastic net regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube |
|:----------|:-------|:-------|:-------|:--------|:--------|:------|:--------|
| -16.5845 | 0.1071 | 0.5091 | 0.6984 | -0.2708 | -0.0323 | 0.036 | -0.0221 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based
pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube |
|:-------|:------|:------|:------|:----|:------|:------|:------|:-----|
| PC7 | 14.02 | 14.78 | 14.62 | 0 | 12.36 | 11.93 | 19.81 | 10 |
| PC1 | 13.63 | 30.49 | 12.95 | 0 | 6.54 | 26.18 | 26.11 | 20 |
| PC8 | 12.82 | 12.75 | 13.42 | 0 | 11.91 | 8.44 | 15.73 | 10 |
| PC6 | 10.25 | 12.69 | 10.89 | 0 | 7.93 | 8.88 | 10.09 | 16.6 |
| PC3 | 5.8 | 2.48 | 8 | 0 | 3.23 | 3.41 | 0.8 | 10 |
| PC9 | 4.73 | 3.43 | 5.34 | 0 | 4.67 | 2.71 | 0 | 6.6 |
| PC4 | 4.32 | 2.1 | 5.78 | 0 | 2.94 | 0.55 | 0.09 | 6.6 |
| PC16 | 3.92 | 1.27 | 3.57 | 0 | 6.22 | 0.83 | 0 | 3.4 |
| PC20 | 3.79 | 3.76 | 2.76 | 0 | 6.1 | 2.84 | 0 | 6.6 |
| PC24 | 3.35 | 2.04 | 2.26 | 0 | 6.11 | 5.13 | 0 | 3.4 |
| PC21 | 2.5 | 0 | 1.85 | 0 | 3.81 | 0.86 | 7.63 | 3.4 |
| PC32 | 2.2 | 0.74 | 1.2 | 0 | 5.05 | 1.77 | 0 | 0 |
| PC18 | 2.06 | 0.28 | 1.83 | 0 | 3.34 | 0.48 | 0 | 3.4 |
| PC15 | 2.06 | 0.86 | 2.12 | 0 | 2.84 | 1.45 | 0 | 0 |
| PC14 | 1.87 | 0.22 | 2.13 | 0 | 2.42 | 0.49 | 0 | 0 |
| PC19 | 1.76 | 0.83 | 1.58 | 0 | 2.82 | 1.1 | 0 | 0 |
| PC10 | 1.73 | 0.88 | 2.1 | 0 | 1.55 | 3.01 | 0 | 0 |
| PC31 | 1.69 | 0.48 | 1.03 | 0 | 3.81 | 0.38 | 0 | 0 |
| PC12 | 1.2 | 0.41 | 1.48 | 0 | 1.15 | 1.21 | 0 | 0 |
| PC30 | 1.06 | 0.58 | 0.76 | 0 | 1.97 | 1.51 | 0 | 0 |
| PC22 | 1.02 | 0.47 | 0.99 | 0 | 1.56 | 0.27 | 0 | 0 |
| PC17 | 0.83 | 1.24 | 0.22 | 0 | 0 | 3.14 | 13.07 | 0 |
| PC26 | 0.72 | 0.25 | 0.7 | 0 | 1.14 | 0.26 | 0 | 0 |
| PC5 | 0.58 | 1.09 | 0.47 | 0 | 0 | 2.15 | 4.42 | 0 |
| PC25 | 0.49 | 0.83 | 0.54 | 0 | 0.47 | 0.05 | 0 | 0 |
| PC13 | 0.46 | 0.3 | 0.67 | 0 | 0.07 | 2.34 | 0 | 0 |
| PC2 | 0.35 | 0.99 | 0.31 | 0 | 0 | 0 | 2.25 | 0 |
| PC11 | 0.23 | 1.78 | 0 | 0 | 0 | 3.42 | 0 | 0 |
| PC28 | 0.21 | 0.07 | 0.31 | 0 | 0 | 1.49 | 0 | 0 |
| PC29 | 0.11 | 0.55 | 0.08 | 0 | 0 | 0.75 | 0 | 0 |
| PC23 | 0.09 | 0.33 | 0.01 | 0 | 0 | 2.16 | 0 | 0 |
| PC33 | 0.07 | 0.57 | 0.03 | 0 | 0 | 0.32 | 0 | 0 |
| PC27 | 0.05 | 0.43 | 0 | 0 | 0 | 0.5 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components
are displayed.
| Variable | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 |
|:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----|
| SS loadings | 18.09 | 16.04 | 6.14 | 4.86 | 4.81 | 4.44 | 4.37 | 4.20 | 3.86 | 2.98 |
| Proportion Var | 0.19 | 0.17 | 0.06 | 0.05 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 |
| Cumulative Var | 0.19 | 0.35 | 0.42 | 0.47 | 0.51 | 0.56 | 0.61 | 0.65 | 0.69 | 0.72 |
| Proportion Explained | 0.26 | 0.23 | 0.09 | 0.07 | 0.07 | 0.06 | 0.06 | 0.06 | 0.06 | 0.04 |
| Cumulative Proportion | 0.26 | 0.49 | 0.58 | 0.65 | 0.72 | 0.78 | 0.84 | 0.90 | 0.96 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 |
|:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------|
| DESSC | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 |
| DESWC | 0.17 | 0.32 | 0.15 | 0.44 | -0.02 | 0.18 | 0.04 | -0.17 | 0.38 | 0.43 |
| DESPL | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 |
| DESSL | -0.75 | -0.24 | 0.1 | 0.12 | -0.09 | 0.12 | -0.11 | -0.15 | 0.25 | 0.25 |
| DESSLd | 0.02 | 0.66 | 0.16 | 0.1 | -0.03 | 0.1 | 0.07 | 0.06 | -0.06 | 0.2 |
| DESWLsy | 0.09 | -0.08 | 0 | -0.05 | -0.14 | -0.16 | 0.86 | 0.02 | 0.06 | -0.13 |
| DESWLsyd | 0.03 | -0.07 | 0 | -0.04 | -0.07 | 0.06 | 0.82 | -0.1 | -0.04 | -0.09 |
| DESWLlt | 0.14 | -0.17 | -0.05 | -0.01 | -0.12 | -0.32 | 0.72 | 0.1 | 0.17 | -0.07 |
| DESWLltd | 0.12 | 0.06 | 0.09 | -0.05 | 0.15 | -0.05 | 0.74 | 0.06 | -0.13 | 0.18 |
| PCNARz | 0.35 | 0.7 | 0.16 | 0.21 | 0.45 | 0.21 | -0.14 | 0.03 | -0.03 | 0.17 |
| PCNARp | 0.47 | 0.57 | 0.1 | 0.27 | 0.41 | 0.2 | -0.13 | -0.02 | -0.02 | 0.16 |
| PCSYNz | 0.89 | -0.07 | -0.03 | -0.17 | 0.08 | -0.17 | 0.09 | 0.06 | -0.07 | -0.23 |
| PCSYNp | 0.88 | -0.13 | 0.05 | -0.15 | 0.01 | -0.14 | 0.02 | 0.04 | 0.08 | -0.11 |
| PCCNCz | -0.64 | -0.38 | -0.18 | -0.4 | -0.06 | -0.31 | -0.04 | -0.22 | 0.07 | 0.23 |
| PCCNCp | -0.48 | -0.13 | -0.1 | -0.45 | -0.04 | -0.36 | 0.03 | -0.14 | 0.05 | 0.22 |
| PCREFz | -0.4 | 0.72 | 0.01 | -0.14 | 0.29 | 0.09 | -0.24 | -0.13 | 0.1 | -0.1 |
| PCREFp | -0.49 | 0.4 | -0.04 | -0.04 | 0.4 | 0.15 | -0.24 | -0.1 | 0.05 | -0.05 |
| PCDCz | 0.17 | 0.3 | 0.9 | 0.06 | 0.06 | 0.07 | 0 | 0.06 | 0.13 | 0.02 |
| PCDCp | 0.18 | 0.32 | 0.79 | 0.15 | 0.09 | 0.08 | 0.05 | 0.04 | 0.09 | 0.07 |
| PCVERBz | -0.62 | -0.62 | -0.01 | -0.1 | -0.18 | 0.3 | -0.09 | -0.05 | 0.2 | -0.09 |
| PCVERBp | -0.41 | -0.49 | 0.06 | -0.17 | -0.22 | 0.43 | -0.1 | -0.03 | 0.16 | -0.11 |
| PCCONNz | 0.24 | 0.02 | 0.09 | -0.06 | -0.06 | 0.08 | -0.01 | 0.9 | 0.11 | 0.03 |
| PCCONNp | 0.13 | -0.07 | 0.03 | -0.24 | 0.12 | -0.05 | -0.12 | 0.75 | 0.11 | -0.14 |
| PCTEMPz | 0.72 | 0.62 | 0.09 | 0.09 | 0.01 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 |
| PCTEMPp | 0.47 | 0.54 | 0.14 | 0.17 | -0.11 | 0.02 | 0.17 | 0.07 | 0 | 0.17 |
| CRFNO1 | 0.07 | 0.7 | 0.15 | 0.07 | -0.37 | -0.13 | -0.22 | -0.03 | 0.11 | -0.07 |
| CRFAO1 | 0.36 | 0.85 | 0.06 | 0.07 | 0.17 | 0.02 | 0.11 | 0 | 0.06 | 0.06 |
| CRFSO1 | 0.09 | 0.77 | 0.1 | -0.01 | -0.32 | -0.16 | -0.22 | -0.07 | 0.06 | -0.11 |
| CRFNOa | -0.03 | 0.77 | 0.08 | 0.01 | -0.32 | -0.08 | -0.21 | -0.05 | 0.08 | -0.1 |
| CRFAOa | 0.3 | 0.86 | 0.07 | 0.08 | 0.25 | 0.07 | 0.12 | 0.02 | 0.05 | 0.06 |
| CRFSOa | -0.01 | 0.81 | 0.07 | -0.04 | -0.28 | -0.12 | -0.2 | -0.09 | 0.05 | -0.1 |
| CRFCWO1 | 0.27 | 0.89 | 0.08 | -0.08 | 0.17 | 0.05 | -0.01 | -0.04 | 0.08 | -0.03 |
| CRFCWO1d | 0.81 | 0.27 | -0.05 | 0.07 | -0.03 | 0.02 | 0.02 | -0.04 | 0.03 | 0.06 |
| CRFCWOa | 0.18 | 0.9 | 0.08 | -0.07 | 0.19 | 0.07 | 0 | -0.04 | 0.07 | -0.04 |
| CRFCWOad | 0.83 | 0.33 | -0.02 | 0.12 | -0.01 | 0.02 | 0.02 | 0.02 | 0.03 | 0.1 |
| CRFANP1 | 0.38 | 0.8 | 0.09 | 0.08 | 0.29 | 0.05 | 0.12 | 0.03 | 0.03 | 0.08 |
| CRFANPa | 0.18 | 0.84 | 0.11 | 0.04 | 0.29 | 0.11 | 0.15 | 0.05 | 0.02 | 0.04 |
| LSASS1 | 0.29 | 0.81 | 0.07 | -0.08 | 0.11 | -0.13 | -0.14 | -0.03 | 0.03 | 0.01 |
| LSASS1d | 0.67 | 0.34 | 0.01 | 0.13 | -0.09 | -0.13 | -0.16 | 0.02 | 0.03 | 0.1 |
| LSASSp | 0.19 | 0.85 | 0.06 | -0.1 | 0.11 | -0.07 | -0.09 | -0.03 | 0.03 | -0.02 |
| LSASSpd | 0.76 | 0.38 | -0.02 | 0.09 | -0.12 | -0.1 | -0.1 | 0.06 | 0.05 | 0.09 |
| LSAGN | 0.73 | 0.58 | -0.01 | 0.13 | 0.09 | -0.05 | -0.01 | -0.04 | 0.12 | 0.04 |
| LSAGNd | 0.59 | 0.72 | 0.03 | 0.02 | -0.05 | -0.08 | -0.07 | 0 | 0.04 | 0.02 |
| LDTTRc | -0.11 | -0.48 | 0.03 | -0.19 | 0.15 | -0.09 | 0.22 | 0.27 | -0.56 | 0.16 |
| LDTTRa | -0.03 | -0.46 | 0.01 | -0.15 | -0.02 | -0.31 | 0.18 | 0.5 | -0.42 | -0.11 |
| LDMTLD | 0.19 | -0.13 | 0.15 | 0.25 | -0.03 | -0.16 | 0.28 | 0.37 | -0.19 | 0.32 |
| CNCAll | -0.3 | 0.07 | 0.62 | -0.12 | 0.17 | -0.01 | -0.06 | -0.62 | -0.02 | -0.09 |
| CNCCaus | 0.04 | 0.02 | 0.86 | -0.19 | 0 | 0.12 | 0.13 | 0.03 | -0.01 | -0.08 |
| CNCLogic | -0.09 | 0.11 | 0.78 | 0.31 | 0.08 | 0.06 | -0.09 | -0.01 | -0.05 | -0.13 |
| CNCADC | 0.05 | -0.11 | -0.1 | 0.71 | -0.04 | 0.06 | 0.12 | -0.24 | -0.16 | 0.05 |
| CNCTemp | 0.05 | 0.24 | 0.35 | 0.16 | 0.09 | -0.15 | -0.26 | 0.14 | 0.14 | 0.03 |
| CNCTempx | -0.02 | 0.11 | 0.14 | 0.09 | -0.16 | 0.33 | 0.08 | -0.01 | -0.19 | 0.25 |
| CNCAdd | -0.36 | -0.05 | 0 | -0.12 | 0.11 | -0.07 | -0.09 | -0.84 | -0.11 | -0.05 |
| CNCPos | -0.27 | 0.11 | 0.61 | -0.31 | 0.19 | -0.05 | -0.1 | -0.53 | 0.03 | -0.1 |
| CNCNeg | 0 | -0.14 | -0.08 | 0.63 | -0.06 | 0.08 | 0.07 | -0.22 | -0.17 | 0.02 |
| SMCAUSv | 0.76 | -0.13 | 0.04 | -0.2 | 0.05 | -0.1 | 0.23 | 0.11 | 0.13 | 0.06 |
| SMCAUSvp | 0.58 | -0.1 | 0.55 | -0.22 | 0.01 | -0.03 | 0.22 | 0.15 | 0.13 | -0.03 |
| SMINTEp | 0.62 | 0.03 | -0.13 | -0.29 | 0.33 | -0.08 | -0.06 | 0.16 | 0.08 | -0.01 |
| SMCAUSr | -0.25 | 0.24 | 0.61 | 0.15 | -0.05 | 0.2 | -0.11 | 0.04 | 0.1 | 0.13 |
| SMINTEr | -0.23 | 0.03 | 0.66 | 0.03 | -0.13 | 0.26 | 0.11 | -0.11 | 0.13 | 0.18 |
| SMCAUSlsa | -0.16 | 0.03 | -0.08 | -0.11 | -0.21 | 0.43 | 0.17 | -0.07 | 0.38 | -0.38 |
| SMCAUSwn | 0.11 | 0.06 | 0.17 | -0.08 | 0.17 | 0.12 | 0.02 | 0.09 | 0.71 | 0.06 |
| SMTEMP | 0.73 | 0.62 | 0.07 | 0.08 | 0 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 |
| SYNLE | -0.17 | -0.03 | -0.01 | 0.03 | 0.02 | -0.18 | 0.08 | 0.23 | 0.35 | 0.02 |
| SYNNP | -0.03 | -0.11 | -0.04 | -0.14 | -0.65 | -0.04 | 0.11 | 0.03 | 0 | 0.01 |
| SYNMEDpos | 0.74 | 0.57 | 0.07 | 0.11 | 0.01 | 0.02 | 0.12 | 0.04 | 0.04 | 0.12 |
| SYNMEDwrd | 0.76 | 0.56 | 0.07 | 0.11 | 0 | 0 | 0.14 | 0.07 | 0.05 | 0.13 |
| SYNMEDlem | 0.76 | 0.55 | 0.07 | 0.11 | -0.01 | 0 | 0.14 | 0.08 | 0.05 | 0.14 |
| SYNSTRUTa | 0.77 | 0.18 | -0.04 | -0.11 | 0.04 | 0.05 | 0.05 | 0.15 | 0.07 | -0.01 |
| SYNSTRUTt | 0.82 | 0.16 | -0.08 | -0.11 | 0.04 | 0.04 | 0.01 | 0.18 | 0.08 | 0.01 |
| DRNP | -0.17 | 0.01 | -0.27 | -0.2 | -0.15 | 0.04 | -0.25 | 0.33 | -0.36 | 0.1 |
| DRVP | 0.16 | 0.04 | 0.09 | -0.05 | 0.64 | -0.04 | 0.03 | -0.04 | 0.22 | -0.22 |
| DRAP | -0.1 | 0.09 | 0.25 | 0.55 | 0.08 | -0.01 | -0.11 | -0.01 | 0.25 | -0.07 |
| DRPP | 0.04 | -0.06 | -0.09 | -0.13 | -0.13 | 0 | 0.02 | 0.18 | 0 | 0.66 |
| DRNEG | 0.03 | 0.03 | -0.11 | 0.5 | 0.05 | 0.06 | -0.08 | 0.13 | -0.12 | -0.06 |
| WRDNOUN | 0.05 | -0.06 | -0.13 | -0.26 | -0.71 | -0.31 | 0.03 | 0.24 | 0.05 | 0.02 |
| WRDVERB | 0.23 | 0.1 | 0.08 | -0.06 | 0.61 | -0.04 | 0.07 | 0.07 | 0.21 | 0.07 |
| WRDADJ | 0.09 | -0.11 | -0.09 | 0.09 | -0.24 | 0.1 | 0.31 | 0.1 | -0.1 | -0.56 |
| WRDADV | -0.09 | 0.14 | 0.33 | 0.71 | 0.06 | 0.06 | -0.1 | -0.05 | 0.19 | -0.08 |
| WRDPRO | -0.04 | 0.14 | -0.08 | 0 | 0.61 | 0.27 | -0.34 | 0.06 | -0.27 | 0.02 |
| WRDPRP3s | 0.03 | 0.12 | 0 | 0.21 | 0.06 | -0.03 | -0.04 | 0.08 | 0.01 | 0.09 |
| WRDPRP3p | -0.06 | -0.18 | -0.05 | -0.03 | 0 | -0.17 | 0.07 | 0.18 | 0.04 | -0.3 |
| WRDFRQc | -0.08 | -0.06 | 0.23 | 0.22 | 0.16 | 0.75 | -0.34 | 0.07 | 0.04 | -0.14 |
| WRDFRQa | -0.26 | -0.09 | 0.17 | 0.07 | 0.13 | 0.58 | -0.43 | -0.17 | -0.25 | 0.22 |
| WRDFRQmc | 0.79 | 0.09 | 0.01 | 0.04 | 0.12 | 0.07 | 0.05 | 0 | 0.09 | 0.08 |
| WRDAOAc | 0.22 | -0.06 | 0.11 | 0.24 | -0.11 | -0.2 | 0.07 | 0.1 | 0.07 | -0.14 |
| WRDFAMc | -0.02 | -0.03 | 0.12 | 0.02 | 0.12 | 0.67 | -0.08 | 0.09 | 0.19 | 0.09 |
| WRDCNCc | 0.04 | 0.03 | -0.31 | -0.41 | -0.07 | -0.5 | 0.03 | 0.02 | 0.1 | 0.51 |
| WRDIMGc | 0.01 | -0.02 | -0.38 | -0.48 | -0.16 | -0.48 | 0.12 | -0.01 | 0.05 | 0.38 |
| WRDMEAc | 0.11 | -0.11 | -0.36 | -0.47 | -0.13 | -0.18 | 0.25 | -0.1 | 0.09 | 0.37 |
| WRDPOLc | 0.15 | 0.08 | 0.25 | -0.21 | 0.13 | 0.2 | -0.04 | 0.15 | 0.67 | -0.04 |
| WRDHYPn | 0.11 | 0.25 | 0.03 | 0.04 | 0.22 | -0.32 | -0.03 | 0.07 | 0.48 | 0.07 |
| WRDHYPv | 0.14 | 0.04 | -0.01 | -0.07 | 0.57 | -0.12 | 0.01 | -0.03 | 0.46 | 0.06 |
| WRDHYPnv | 0.04 | 0.16 | -0.1 | -0.23 | -0.23 | -0.52 | -0.02 | 0.14 | 0.49 | 0.03 |
| RDFRE | 0.79 | 0.26 | -0.08 | -0.09 | 0.14 | -0.09 | -0.13 | 0.11 | -0.23 | -0.18 |
| RDFKGL | -0.75 | -0.25 | 0.1 | 0.11 | -0.11 | 0.11 | -0.03 | -0.15 | 0.26 | 0.24 |
| RDL2 | 0.48 | 0.59 | 0.15 | 0.01 | 0.2 | 0.44 | -0.16 | 0.08 | 0.1 | -0.1 |
Coh-Metrix Model 2 {#cohmetrix-model-2}
General Description
Coh-Metrix Model 2 is a simplified version of Model 1. Model 2 is recommended for use over Model 1.
Coh-Metrix Model 2 is an ensemble (formed by averaging predicted quality scores) of the three sub-models described below.
Highly correlated Coh-Metrix metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Coh-Metrix Model 2a
This model was trained on fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
pls = partial least squares regression
rf = random forest regression
mars = bagged multivariate adaptive regression splines
gbm = stochastic gradient boosted trees
svm = support vector machines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | cube |
|:----------|:-------|:-------|:-------|:-------|:------|:-------|
| -11.0081 | 0.1741 | 0.0413 | 0.1875 | 0.2353 | 0.206 | 0.2108 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | gbm | svm | cube |
|:----------|:--------|:-----|:------|:------|:------|:-----|:------|
| DESWC | 29.91 | 5.86 | 14.51 | 55.78 | 44.87 | 5.96 | 36.49 |
| DESWLlt | 8.67 | 2.98 | 2.69 | 18.79 | 3.07 | 2.39 | 17.89 |
| LDMTLD | 7.35 | 4.01 | 5.57 | 0 | 9.55 | 3.9 | 17.89 |
| WRDHYPn | 7.16 | 2.9 | 2.68 | 9.55 | 3.85 | 2.29 | 17.89 |
| LDTTRa | 2.84 | 2.01 | 0.97 | 11.14 | 0.26 | 1.12 | 1.05 |
| CNCPos | 1.36 | 0.88 | 1.11 | 4.75 | 0.05 | 1.28 | 0.35 |
| DESWLsy | 1.3 | 2.09 | 1.9 | 0 | 1.66 | 1.54 | 1.05 |
| CNCTempx | 1.11 | 0.89 | 1.03 | 0 | 1.63 | 2.48 | 0.35 |
| CNCLogic | 1.09 | 1.42 | 1.39 | 0 | 1.81 | 1.7 | 0.35 |
| PCDCp | 1.08 | 2 | 2.56 | 0 | 1.71 | 1.36 | 0 |
| DESPL | 1.04 | 3.22 | 2.18 | 0 | 0.04 | 2.15 | 0 |
| DESWLltd | 1.02 | 2.3 | 1.13 | 0 | 1.26 | 1.6 | 0 |
| WRDFRQa | 0.96 | 1.86 | 0.58 | 0 | 0.17 | 1.97 | 1.05 |
| DESWLsyd | 0.92 | 1.97 | 1.87 | 0 | 1.21 | 1.29 | 0 |
| DESSLd | 0.9 | 1.67 | 2 | 0 | 1.27 | 1.36 | 0 |
| CNCTemp | 0.89 | 0.93 | 1.39 | 0 | 1.08 | 1.89 | 0.35 |
| LSAGN | 0.87 | 2.55 | 1.18 | 0 | 0.22 | 1.8 | 0 |
| LSASSpd | 0.86 | 1.95 | 0.86 | 0 | 0.36 | 1.79 | 0.35 |
| CNCADC | 0.85 | 1.12 | 1.41 | 0 | 0.67 | 2.37 | 0 |
| DRPP | 0.84 | 2.18 | 2.11 | 0 | 0.57 | 1.4 | 0 |
| PCCONNz | 0.82 | 1.09 | 0.4 | 0 | 1.61 | 1.36 | 0 |
| WRDPRO | 0.82 | 1.77 | 1.44 | 0 | 0.73 | 1.61 | 0 |
| SYNSTRUTa | 0.8 | 0.85 | 3.07 | 0 | 0.58 | 1.4 | 0.7 |
| CRFCWO1d | 0.79 | 1.67 | 1.39 | 0 | 0.43 | 1.88 | 0 |
| LSASS1d | 0.78 | 1.55 | 0.48 | 0 | 0.78 | 1.72 | 0 |
| SMCAUSwn | 0.76 | 1.4 | 1.04 | 0 | 0.31 | 2.16 | 0 |
| SYNMEDpos | 0.75 | 1.75 | 0.6 | 0 | 0.77 | 1.36 | 0 |
| SMINTEp | 0.73 | 0.75 | 0.85 | 0 | 0.79 | 1.66 | 0.35 |
| LDTTRc | 0.73 | 1.82 | 0.7 | 0 | 0.72 | 1.23 | 0 |
| CRFCWOad | 0.73 | 1.51 | 1.12 | 0 | 0.38 | 1.81 | 0 |
| WRDVERB | 0.71 | 1.04 | 0.93 | 0 | 0.58 | 0.86 | 1.05 |
| WRDFAMc | 0.7 | 1.09 | 0.5 | 0 | 1.32 | 1.08 | 0 |
| WRDHYPnv | 0.69 | 1.9 | 0.91 | 0 | 0.13 | 1.22 | 0.35 |
| WRDFRQmc | 0.69 | 1.35 | 2.5 | 0 | 1.3 | 0.4 | 0 |
| WRDCNCc | 0.69 | 1.07 | 0.44 | 0 | 0.58 | 0.8 | 1.05 |
| PCNARz | 0.68 | 1.29 | 1.29 | 0 | 0.58 | 1.1 | 0.35 |
| WRDPOLc | 0.67 | 0.85 | 0.77 | 0 | 0.51 | 1.97 | 0 |
| RDFRE | 0.66 | 1.29 | 0.91 | 0 | 0.76 | 1.23 | 0 |
| CRFNOa | 0.63 | 0.74 | 1.34 | 0 | 0.56 | 1.67 | 0 |
| PCVERBz | 0.63 | 1.55 | 0.67 | 0 | 0.28 | 1.09 | 0.35 |
| LSAGNd | 0.61 | 1.5 | 0.95 | 0 | 0.24 | 1.38 | 0 |
| SMCAUSvp | 0.59 | 0.93 | 0.73 | 0 | 0.36 | 1.67 | 0 |
| WRDADV | 0.58 | 1.3 | 0.95 | 0 | 0.13 | 1.55 | 0 |
| WRDAOAc | 0.58 | 1.66 | 0.46 | 0 | 0.27 | 1.18 | 0 |
| DRNP | 0.57 | 1.73 | 1.5 | 0 | 0.27 | 0.82 | 0 |
| WRDHYPv | 0.56 | 0.28 | 1.01 | 0 | 0.83 | 1.47 | 0 |
| DRVP | 0.56 | 0.82 | 0.62 | 0 | 0.82 | 0.75 | 0.35 |
| PCNARp | 0.54 | 1.76 | 1.29 | 0 | 0 | 1 | 0 |
| CNCCaus | 0.51 | 1.11 | 0.58 | 0 | 0.11 | 1.46 | 0 |
| CRFCWOa | 0.49 | 0.46 | 0.56 | 0 | 0.37 | 1.57 | 0 |
| SMCAUSr | 0.47 | 1.58 | 1.07 | 0 | 0.47 | 0.29 | 0 |
| SMCAUSlsa | 0.46 | 0.24 | 0.86 | 0 | 0.63 | 1.26 | 0 |
| LSASSp | 0.46 | 0.79 | 1.01 | 0 | 0.15 | 1.31 | 0 |
| SMINTEr | 0.45 | 1.83 | 0.9 | 0 | 0.13 | 0.45 | 0 |
| DRAP | 0.44 | 0.46 | 1.01 | 0 | 0.41 | 1.17 | 0 |
| DRNEG | 0.44 | 0.76 | 0.43 | 0 | 0.09 | 1.41 | 0 |
| WRDNOUN | 0.43 | 0.56 | 0.96 | 0 | 0.75 | 0.66 | 0 |
| WRDFRQc | 0.43 | 0.54 | 0.67 | 0 | 0.48 | 1.04 | 0 |
| WRDPRP3s | 0.43 | 0.76 | 0.84 | 0 | 0.74 | 0.56 | 0 |
| CRFANPa | 0.41 | 0.76 | 1.36 | 0 | 0.18 | 0.96 | 0 |
| SYNLE | 0.4 | 1.13 | 0.95 | 0 | 0.1 | 0.77 | 0 |
| PCTEMPp | 0.4 | 1.05 | 0.22 | 0 | 0.55 | 0.47 | 0 |
| WRDADJ | 0.39 | 0.99 | 0.94 | 0 | 0.2 | 0.74 | 0 |
| RDL2 | 0.39 | 0.31 | 1.06 | 0 | 0.39 | 1.09 | 0 |
| WRDIMGc | 0.37 | 0.14 | 0.74 | 0 | 0.3 | 0.93 | 0.35 |
| PCVERBp | 0.36 | 0.99 | 1.39 | 0 | 0 | 0.75 | 0 |
| PCCNCz | 0.35 | 1.18 | 0.1 | 0 | 0.32 | 0.39 | 0 |
| WRDMEAc | 0.29 | 0.21 | 0.45 | 0 | 0.44 | 0.71 | 0 |
| PCREFz | 0.29 | 0.47 | 1.07 | 0 | 0.3 | 0.53 | 0 |
| SMCAUSv | 0.25 | 0.39 | 0.77 | 0 | 0.13 | 0.66 | 0 |
| SYNNP | 0.22 | 0.03 | 0.64 | 0 | 0.58 | 0.33 | 0 |
| PCSYNp | 0.21 | 0.68 | 1.55 | 0 | 0.15 | 0 | 0 |
| PCCONNp | 0.17 | 0 | 0.33 | 0 | 0.11 | 0.66 | 0 |
| PCCNCp | 0.16 | 0.57 | 0.78 | 0 | 0 | 0.16 | 0 |
| PCREFp | 0.15 | 0.06 | 0.74 | 0 | 0 | 0.58 | 0 |
| WRDPRP3p | 0.14 | 0.84 | 0 | 0 | 0 | 0.03 | 0 |
Coh-Metrix Model 2b
This model was trained on winter data [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
mars = bagged multivariate adaptive regression splines
gbm = stochastic gradient boosted trees
svm = support vector machines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | mars | gbm | svm | cube |
|:----------|:-------|:-------|:-------|:-------|
| -7.2585 | 0.2289 | 0.5300 | 0.1527 | 0.1150 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | mars | gbm | svm | cube |
|:----------|:--------|:------|:-----|:-----|:------|
| DESWC | 30.39 | 45.46 | 34.5 | 4.37 | 16.04 |
| LSAGN | 7.18 | 0 | 9.31 | 2.85 | 17.43 |
| DESWLlt | 6.73 | 19 | 2.42 | 1.37 | 9.31 |
| LDMTLD | 5.59 | 0 | 8.91 | 2.47 | 5.54 |
| SYNLE | 5.43 | 13.58 | 3.65 | 1.79 | 2.18 |
| WRDIMGc | 3.6 | 9.36 | 1.84 | 1.22 | 3.37 |
| WRDNOUN | 3.19 | 6.64 | 1.38 | 1.77 | 6.53 |
| CNCAdd | 1.87 | 5.96 | 0.45 | 1.02 | 1.39 |
| WRDVERB | 1.42 | 0 | 2.02 | 1.67 | 1.19 |
| SMCAUSwn | 1.25 | 0 | 1.83 | 2.04 | 0 |
| DESWLltd | 1.16 | 0 | 1.21 | 1.53 | 2.77 |
| CRFCWO1d | 1.09 | 0 | 1.18 | 2.17 | 1.39 |
| WRDHYPnv | 1.02 | 0 | 0.98 | 1.36 | 2.77 |
| CRFNOa | 0.99 | 0 | 1.37 | 1.9 | 0 |
| RDFRE | 0.99 | 0 | 1.54 | 0.25 | 1.39 |
| LSAGNd | 0.85 | 0 | 0.53 | 1.76 | 2.77 |
| DESWLsy | 0.77 | 0 | 1.11 | 1.28 | 0 |
| SYNMEDpos | 0.76 | 0 | 0.29 | 2 | 2.77 |
| WRDPRP3s | 0.74 | 0 | 0.91 | 1.53 | 0.4 |
| DESPL | 0.73 | 0 | 0.44 | 2.34 | 1.39 |
| CNCAll | 0.72 | 0 | 0.44 | 0.96 | 3.17 |
| RDL2 | 0.71 | 0 | 0.6 | 1.65 | 1.39 |
| PCCNCz | 0.71 | 0 | 0.62 | 1.6 | 1.39 |
| PCVERBz | 0.69 | 0 | 0.82 | 1.78 | 0 |
| WRDPRO | 0.69 | 0 | 0.69 | 1.18 | 1.39 |
| CNCTemp | 0.65 | 0 | 0.71 | 1.87 | 0 |
| WRDFRQc | 0.65 | 0 | 0.98 | 0.99 | 0 |
| WRDFRQmc | 0.63 | 0 | 0.56 | 1.25 | 1.39 |
| DRVP | 0.62 | 0 | 0.93 | 0.92 | 0 |
| LSASS1d | 0.61 | 0 | 0.65 | 1.87 | 0 |
| SMCAUSlsa | 0.6 | 0 | 0.94 | 0.77 | 0 |
| CRFCWOad | 0.6 | 0 | 0.61 | 1.92 | 0 |
| WRDMEAc | 0.59 | 0 | 0.53 | 1.4 | 0.99 |
| PCTEMPp | 0.56 | 0 | 0.78 | 1.02 | 0 |
| SMCAUSv | 0.56 | 0 | 0.85 | 0.83 | 0 |
| PCCNCp | 0.56 | 0 | 0.02 | 1.62 | 2.77 |
| WRDFRQa | 0.53 | 0 | 0.73 | 1.01 | 0 |
| LSASSp | 0.53 | 0 | 0.54 | 1.71 | 0 |
| LDTTRc | 0.52 | 0 | 0.64 | 1.3 | 0 |
| DRPP | 0.51 | 0 | 0.7 | 1.02 | 0 |
| PCREFp | 0.5 | 0 | 0 | 0.7 | 3.56 |
| CRFCWO1 | 0.47 | 0 | 0.4 | 1.75 | 0 |
| SMCAUSvp | 0.46 | 0 | 0.47 | 1.42 | 0 |
| PCNARz | 0.45 | 0 | 0.26 | 1.64 | 0.59 |
| SYNNP | 0.45 | 0 | 0.3 | 0.91 | 1.39 |
| PCSYNz | 0.45 | 0 | 0.32 | 0.87 | 1.39 |
| SMINTEp | 0.45 | 0 | 0.4 | 1.66 | 0 |
| LDTTRa | 0.43 | 0 | 0.23 | 1.06 | 1.39 |
| DESWLsyd | 0.42 | 0 | 0.35 | 1.62 | 0 |
| CRFANPa | 0.41 | 0 | 0.34 | 1.59 | 0 |
| SMINTEr | 0.41 | 0 | 0.6 | 0.69 | 0 |
| CNCLogic | 0.4 | 0 | 0.51 | 0.89 | 0 |
| WRDAOAc | 0.4 | 0 | 0.58 | 0.69 | 0 |
| WRDHYPv | 0.4 | 0 | 0.36 | 1.45 | 0 |
| CNCNeg | 0.4 | 0 | 0.32 | 1.17 | 0.59 |
| CNCCaus | 0.38 | 0 | 0.47 | 0.92 | 0 |
| WRDFAMc | 0.37 | 0 | 0.46 | 0.91 | 0 |
| SYNSTRUTa | 0.36 | 0 | 0.36 | 1.2 | 0 |
| CRFAOa | 0.35 | 0 | 0.19 | 1.68 | 0 |
| WRDADV | 0.34 | 0 | 0.32 | 1.19 | 0 |
| SMCAUSr | 0.33 | 0 | 0.61 | 0.1 | 0 |
| DESSLd | 0.32 | 0 | 0.18 | 1.5 | 0 |
| PCCONNp | 0.29 | 0 | 0.46 | 0.37 | 0 |
| WRDPOLc | 0.28 | 0 | 0.28 | 0.93 | 0 |
| WRDADJ | 0.28 | 0 | 0.4 | 0.51 | 0 |
| DRAP | 0.26 | 0 | 0.24 | 0.87 | 0 |
| DRNP | 0.26 | 0 | 0.33 | 0.57 | 0 |
| WRDHYPn | 0.26 | 0 | 0.27 | 0.84 | 0 |
| CNCTempx | 0.25 | 0 | 0.23 | 0.86 | 0 |
| DRNEG | 0.23 | 0 | 0.13 | 1.08 | 0 |
| PCREFz | 0.23 | 0 | 0.24 | 0.7 | 0 |
| PCVERBp | 0.22 | 0 | 0 | 1.48 | 0 |
| PCNARp | 0.2 | 0 | 0.01 | 1.35 | 0 |
| PCDCp | 0.19 | 0 | 0.11 | 0.9 | 0 |
| PCSYNp | 0.09 | 0 | 0.02 | 0.56 | 0 |
| WRDPRP3p | 0 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 2c
This model was trained on spring data [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations:
all = ensemble model
pls = partial least squares regression
mars = bagged multivariate adaptive regression splines
gbm = stochastic gradient boosted trees
* svm = support vector machines
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | svm |
|:----------|:-------|:-------|:------|:-------|
| -10.8192 | 0.0374 | 0.2735 | 0.243 | 0.5377 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | mars | gbm | svm |
|:----------|:--------|:-----|:------|:------|:-----|
| DESWC | 22.58 | 5.76 | 48.45 | 37.37 | 3.91 |
| WRDVERB | 7.27 | 1.88 | 23.22 | 2.92 | 1.5 |
| DESWLltd | 5.72 | 1.75 | 16.17 | 3.85 | 1.53 |
| CRFCWOa | 4.57 | 1.9 | 12.16 | 1.14 | 2.44 |
| PCNARp | 2.04 | 2.72 | 0 | 3.23 | 2.49 |
| PCDCz | 1.78 | 1.05 | 0 | 3.24 | 2.07 |
| CRFANPa | 1.67 | 1.23 | 0 | 2.21 | 2.3 |
| LSASS1d | 1.67 | 1.5 | 0 | 1.62 | 2.56 |
| WRDHYPn | 1.59 | 2.38 | 0 | 3.29 | 1.58 |
| SYNSTRUTa | 1.59 | 1.74 | 0 | 1.28 | 2.54 |
| LDMTLD | 1.57 | 1.79 | 0 | 2.47 | 1.95 |
| LSAGN | 1.57 | 1.72 | 0 | 1.17 | 2.55 |
| SMCAUSvp | 1.53 | 0.97 | 0 | 1.91 | 2.17 |
| DESSLd | 1.52 | 0.87 | 0 | 1.28 | 2.45 |
| LSAGNd | 1.49 | 2.29 | 0 | 0.11 | 2.81 |
| WRDFRQmc | 1.41 | 2.04 | 0 | 1.45 | 2.06 |
| DESPL | 1.41 | 3.29 | 0 | 0.85 | 2.25 |
| PCVERBz | 1.25 | 1.73 | 0 | 0.62 | 2.13 |
| SYNMEDpos | 1.24 | 1.78 | 0 | 0.72 | 2.08 |
| LSASSp | 1.22 | 1.97 | 0 | 0.13 | 2.28 |
| SMCAUSv | 1.16 | 0.91 | 0 | 1.16 | 1.78 |
| CRFCWO1d | 1.15 | 1.54 | 0 | 0.21 | 2.14 |
| SMCAUSwn | 1.09 | 2.26 | 0 | 0.94 | 1.63 |
| CNCTempx | 1.08 | 0.26 | 0 | 0.57 | 1.92 |
| WRDHYPv | 1.06 | 2.37 | 0 | 1.41 | 1.36 |
| SMCAUSlsa | 1.03 | 1.59 | 0 | 1.07 | 1.5 |
| WRDNOUN | 1 | 2.13 | 0 | 1.68 | 1.11 |
| PCDCp | 1 | 1.81 | 0 | 0.23 | 1.8 |
| PCVERBp | 0.93 | 1.09 | 0 | 0.05 | 1.79 |
| PCTEMPp | 0.92 | 1.68 | 0 | 0.51 | 1.52 |
| LDTTRc | 0.9 | 1.26 | 0 | 1.34 | 1.14 |
| WRDPRP3s | 0.87 | 1.36 | 0 | 1.67 | 0.92 |
| DESWLlt | 0.85 | 2.05 | 0 | 1.12 | 1.07 |
| CNCTemp | 0.85 | 0.85 | 0 | 0.9 | 1.27 |
| RDL2 | 0.84 | 2.17 | 0 | 0.54 | 1.31 |
| DRPP | 0.83 | 1.78 | 0 | 1.37 | 0.94 |
| PCCNCz | 0.8 | 1.87 | 0 | 0.31 | 1.35 |
| DRNP | 0.8 | 1.62 | 0 | 0.79 | 1.16 |
| LDTTRa | 0.79 | 2.31 | 0 | 0.24 | 1.34 |
| WRDAOAc | 0.78 | 0.69 | 0 | 0.78 | 1.17 |
| RDFKGL | 0.73 | 2.06 | 0 | 0.13 | 1.27 |
| SYNNP | 0.68 | 0.65 | 0 | 0.24 | 1.23 |
| CNCPos | 0.68 | 0.82 | 0 | 0.67 | 1.03 |
| CNCCaus | 0.67 | 0.73 | 0 | 0.63 | 1.02 |
| WRDADV | 0.67 | 1.9 | 0 | 0.66 | 0.93 |
| PCSYNz | 0.66 | 1.76 | 0 | 0.33 | 1.07 |
| WRDPRO | 0.64 | 0.73 | 0 | 0.9 | 0.84 |
| CNCLogic | 0.64 | 0.41 | 0 | 0.71 | 0.95 |
| PCCNCp | 0.64 | 1.15 | 0 | 0 | 1.22 |
| DRVP | 0.63 | 0.79 | 0 | 0.32 | 1.08 |
| WRDADJ | 0.62 | 1.12 | 0 | 0.38 | 1 |
| SMINTEp | 0.62 | 1.06 | 0 | 0.53 | 0.95 |
| DRNEG | 0.6 | 0.73 | 0 | 0.06 | 1.13 |
| WRDPOLc | 0.6 | 0.89 | 0 | 0.61 | 0.88 |
| WRDHYPnv | 0.58 | 0.02 | 0 | 0.17 | 1.09 |
| WRDFRQa | 0.58 | 0.36 | 0 | 0.35 | 0.99 |
| SYNLE | 0.57 | 0.09 | 0 | 0.62 | 0.87 |
| SMCAUSr | 0.56 | 0.05 | 0 | 0.3 | 1 |
| DRAP | 0.55 | 1.23 | 0 | 0.32 | 0.89 |
| PCREFz | 0.53 | 1.2 | 0 | 0.46 | 0.78 |
| DESWLsy | 0.5 | 0.85 | 0 | 0.43 | 0.76 |
| WRDMEAc | 0.49 | 0.68 | 0 | 0.7 | 0.63 |
| PCSYNp | 0.48 | 1.45 | 0 | 0.02 | 0.86 |
| CNCADC | 0.46 | 1.61 | 0 | 0.41 | 0.63 |
| WRDFRQc | 0.42 | 0.3 | 0 | 0.83 | 0.45 |
| WRDCNCc | 0.39 | 1.3 | 0 | 0.39 | 0.53 |
| DESWLsyd | 0.39 | 0.32 | 0 | 0.33 | 0.63 |
| SMINTEr | 0.26 | 0.37 | 0 | 0.11 | 0.44 |
| PCCONNz | 0.23 | 0.53 | 0 | 0.23 | 0.32 |
| PCREFp | 0.23 | 0.32 | 0 | 0.01 | 0.44 |
| WRDFAMc | 0.1 | 0 | 0 | 0.24 | 0.09 |
| CRFNO1 | 0.09 | 0.98 | 0 | 0.1 | 0.08 |
| PCCONNp | 0.08 | 1.18 | 0 | 0.04 | 0.06 |
| WRDPRP3p | 0.02 | 0.44 | 0 | 0.03 | 0 |
Coh-Metrix Model 3 {#cohmetrix-model-3}
General Description
Coh-Metrix Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below.
The models were trained on Coh-Metrix scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons).
Highly correlated CohMetrix metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
More details on the sample will be provided once peer review is complete on the main study using this model.
CohMetrix Model 3narr
This model was trained on CohMetrix scores from 15 min narrative writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
mars = bagged multivariate adaptive regression splines
enet = elastic net regression
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | enet | cube |
|:----------|:-------|:-------|:-------|:-------|:-------|
| 0.0000 | 0.1419 | 0.3143 | 0.0729 | 0.0816 | 0.1792 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |gbm |mars |enet |cube |
|---------|-------|----|-----|-----|-----|-----|
|DESWC |24.87 |4.56|36.54|23.26|28.45|14.11|
|WRDHYPn |8.87 |1.84|2.7 |14.41|7.18 |4.52 |
|WRDNOUN |7.12 |2.27|2.89 |10.61|7.1 |4.37 |
|DESSL |7 |1.2 |0.27 |14.41|0 |0.77 |
|SYNNP |5.71 |1.81|3.49 |7.61 |8.68 |2.38 |
|DESWLlt |5.25 |0.86|1.94 |7.61 |5.68 |4.14 |
|LDVOCD |4.17 |2.88|3.99 |6.34 |0 |0.69 |
|LDTTRa |2.6 |3.58|4.54 |0 |7.74 |3.99 |
|SMCAUSwn |2.18 |2 |5.72 |0 |3.96 |2.15 |
|SYNLE |2.05 |0.5 |1.59 |3.39 |0 |0.31 |
|WRDPRP1s |1.71 |0.5 |0.83 |2.89 |0 |0.84 |
|WRDHYPnv |1.55 |0.44|0.28 |2.61 |0 |1.61 |
|PCDCp |1.44 |1.37|0.04 |2.61 |0.04 |1 |
|PCREFp |1.41 |0.55|0 |2.65 |0 |1 |
|PCNARz |1.32 |2.01|3.19 |0 |0 |3.14 |
|CRFANPa |1.3 |1.59|3.69 |0 |0 |2.3 |
|LSAGN |1.29 |2.26|2.08 |0 |2.54 |2.99 |
|WRDFRQmc |0.94 |1.95|0.78 |0 |2.77 |2.68 |
|CNCLogic |0.91 |0.73|0.37 |1.61 |0 |0.23 |
|PCDCz |0.91 |1.12|3.1 |0 |0 |0.77 |
|SYNMEDpos|0.9 |1.78|0.13 |0 |4.2 |2.61 |
|SMCAUSlsa|0.84 |0.78|0.76 |0 |1.34 |3.37 |
|PCCONNz |0.75 |1.46|1.11 |0 |2.09 |1.46 |
|DRPP |0.65 |1.2 |0.36 |0 |2.51 |1.76 |
|WRDAOAc |0.65 |1.77|1.13 |0 |1.24 |1.23 |
|WRDPRO |0.63 |1.35|1.42 |0 |0 |1.53 |
|DESPL |0.61 |2.71|0.51 |0 |1.6 |1.46 |
|PCREFz |0.59 |1.06|0.56 |0 |2.75 |0.92 |
|WRDMEAc |0.53 |0.99|1.17 |0 |0.82 |0.84 |
|LDTTRc |0.52 |2.28|0.15 |0 |0 |2.68 |
|DRNP |0.49 |0.62|0.24 |0 |1.25 |1.92 |
|DESWLsy |0.49 |1.25|0.33 |0 |0.72 |1.99 |
|WRDPOLc |0.44 |1.45|0.91 |0 |0 |1.07 |
|SMINTEr |0.4 |0.35|0.26 |0 |2.03 |0.84 |
|PCSYNz |0.39 |0.91|0.59 |0 |0 |1.46 |
|PCCONNp |0.37 |2.07|0.2 |0 |1.9 |0.31 |
|SMINTEp |0.35 |0.56|1.24 |0 |0 |0.15 |
|DRPVAL |0.34 |1.5 |0.33 |0 |1.65 |0.23 |
|LSASS1d |0.33 |1.49|0.06 |0 |0 |1.76 |
|PCCNCz |0.32 |1.44|0.1 |0 |0 |1.61 |
|CRFCWOa |0.3 |1.73|0.04 |0 |0 |1.46 |
|RDL2 |0.29 |1.91|0.31 |0 |0 |0.92 |
|CNCPos |0.28 |0.22|0.87 |0 |0 |0.38 |
|PCVERBz |0.28 |1.5 |0.14 |0 |0 |1.23 |
|LSAGNd |0.28 |1.86|0.19 |0 |0 |1.07 |
|CRFCWO1d |0.25 |1.57|0.74 |0 |0 |0 |
|WRDFRQa |0.25 |0.44|0.67 |0 |0 |0.46 |
|CNCCaus |0.25 |0.36|0.29 |0 |0 |1.15 |
|CRFAOa |0.24 |1.74|0.04 |0 |0 |1.07 |
|DESWLltd |0.23 |0.2 |0.49 |0 |0 |0.69 |
|WRDADJ |0.22 |0.25|0.43 |0 |0 |0.69 |
|WRDPRP3p |0.21 |1.25|0.51 |0 |0 |0.23 |
|LDMTLD |0.21 |0.26|0.42 |0 |0 |0.69 |
|CRFCWOad |0.21 |1.68|0.35 |0 |0 |0.38 |
|CNCTemp |0.2 |0.55|0.06 |0 |1.45 |0.15 |
|WRDIMGc |0.19 |0.4 |0.22 |0 |0 |0.84 |
|DESWLsyd |0.19 |0.69|0.26 |0 |0 |0.69 |
|WRDVERB |0.18 |0.55|0.44 |0 |0 |0.31 |
|CRFNOa |0.15 |1.06|0.07 |0 |0 |0.61 |
|CNCADC |0.14 |1.46|0.12 |0 |0 |0.31 |
|WRDHYPv |0.14 |0.35|0.32 |0 |0 |0.31 |
|WRDFRQc |0.14 |0.58|0.38 |0 |0 |0.15 |
|WRDCNCc |0.14 |0.37|0.51 |0 |0 |0 |
|LSASSp |0.14 |1.54|0.08 |0 |0 |0.38 |
|LSASSpd |0.14 |1.55|0.05 |0 |0 |0.46 |
|PCTEMPp |0.13 |1.36|0.06 |0 |0 |0.38 |
|DRVP |0.12 |0.92|0.15 |0 |0 |0.31 |
|WRDPRP2 |0.12 |1.54|0.13 |0 |0.23 |0 |
|DRGERUND |0.12 |0.49|0.15 |0 |0 |0.46 |
|SMCAUSvp |0.11 |0.54|0.28 |0 |0 |0.15 |
|PCSYNp |0.1 |0.68|0.15 |0 |0 |0.23 |
|DRINF |0.1 |0.68|0.11 |0 |0 |0.31 |
|DRAP |0.09 |0.43|0.29 |0 |0 |0 |
|WRDADV |0.09 |0.6 |0.28 |0 |0 |0 |
|DESSLd |0.09 |1 |0.12 |0 |0.1 |0.08 |
|SYNSTRUTa|0.08 |1.31|0.09 |0 |0 |0 |
|PCCNCp |0.07 |1.21|0 |0 |0 |0.15 |
|WRDFAMc |0.07 |0.63|0.2 |0 |0 |0 |
|SMCAUSv |0.06 |0.53|0.14 |0 |0 |0 |
|CNCTempx |0.05 |0 |0.16 |0 |0 |0.08 |
|SMCAUSr |0.04 |0.77|0.02 |0 |0 |0 |
|PCVERBp |0.04 |0.9 |0 |0 |0 |0 |
|WRDPRP1p |0.03 |0.68|0.02 |0 |0 |0 |
|WRDPRP3s |0.02 |0.4 |0 |0 |0 |0 |
|DRNEG |0.01 |0.24|0.02 |0 |0 |0 |
Coh-Metrix Model 3exp
This model was trained on Coh-Metrix scores from 15 min expository writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
gbm = stochastic gradient boosted trees
mars = bagged multivariate adaptive regression splines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | cube |
|:----------|:-------|:-------|:-------|:-------|
| -0.0577 | 0.1306 | 0.3136 | 0.3991 | 0.1752 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|mars |pls |gbm |cube |
|---------|-------|-----|----|-----|-----|
|DESWC |26.13 |25.33|5.56|47.08|15.82|
|LSAGN |3.77 |0 |2.75|5.56 |4.35 |
|LDTTRa |3.68 |0 |4.12|4.55 |3.68 |
|DESSLd |3.13 |9.32 |1.26|2.42 |3.51 |
|DESWLlt |2.98 |10.88|0.99|1.35 |4.35 |
|WRDPRP2 |2.25 |0 |2.09|2.72 |3.18 |
|LDVOCD |2.16 |2.46 |3.82|0.71 |2.26 |
|DRPP |2.11 |0 |2.07|3.28 |1.09 |
|WRDPOLc |2.1 |12.49|0.93|0.31 |0.5 |
|LDTTRc |1.96 |0 |3.31|1.89 |1.17 |
|SMCAUSwn |1.62 |0 |0.98|2.12 |2.85 |
|WRDNOUN |1.61 |3.23 |1.38|0.53 |3.26 |
|WRDPRP1s |1.55 |5.37 |1.05|0.83 |1.26 |
|WRDPRP1p |1.5 |5.2 |0.08|0.9 |2.68 |
|CNCTemp |1.39 |7.18 |1.01|0.27 |0.33 |
|PCNARz |1.37 |0 |2.21|0.63 |2.59 |
|LSASS1d |1.29 |4.14 |1.67|0.51 |0.25 |
|PCREFz |1.27 |5.37 |1.14|0.18 |0.92 |
|PCCONNz |1.25 |0 |1.16|2.11 |0.42 |
|DRNP |1.22 |3.67 |0.17|1.18 |1.34 |
|WRDMEAc |1.2 |5.37 |0.64|0.47 |0.75 |
|WRDFRQa |1.19 |0 |1.04|1.52 |1.59 |
|PCCONNp |1.19 |0 |2.2 |0.62 |1.59 |
|SYNMEDpos|1.19 |0 |1.84|0.24 |3.1 |
|WRDHYPn |1.18 |0 |0.89|1.18 |2.59 |
|DESPL |1.03 |0 |2.71|0.38 |0.25 |
|WRDHYPnv |1.02 |0 |0.83|0.73 |2.76 |
|PCCNCz |0.94 |0 |1.35|0.26 |2.43 |
|RDL2 |0.93 |0 |1.89|0.83 |0.17 |
|LSASSp |0.9 |0 |1.68|0.69 |0.67 |
|PCVERBz |0.9 |0 |1.38|0.38 |1.92 |
|WRDHYPv |0.87 |0 |0.92|0.94 |1.26 |
|LSASSpd |0.84 |0 |1.66|0.21 |1.42 |
|WRDADJ |0.82 |0 |1.39|0.89 |0.25 |
|CRFCWOa |0.81 |0 |1.79|0.37 |0.67 |
|CRFANPa |0.78 |0 |1.28|0.52 |1.09 |
|LSAGNd |0.77 |0 |2.06|0.08 |0.59 |
|PCREFp |0.77 |0 |0.97|0 |2.76 |
|CRFAOa |0.74 |0 |1.87|0.02 |0.92 |
|DESSL |0.74 |0 |0.94|0.46 |1.59 |
|CRFCWO1d |0.69 |0 |1.77|0.29 |0.17 |
|SYNNP |0.69 |0 |1.25|0.31 |1.09 |
|CRFNOa |0.64 |0 |1.2 |0.47 |0.5 |
|PCTEMPp |0.62 |0 |1.26|0.49 |0.25 |
|WRDAOAc |0.61 |0 |1.21|0.47 |0.33 |
|CNCNeg |0.6 |0 |1.7 |0.2 |0 |
|DRAP |0.58 |0 |1.08|0.21 |0.92 |
|DRGERUND |0.57 |0 |0.97|0.69 |0 |
|PCDCz |0.55 |0 |1.33|0.16 |0.42 |
|PCDCp |0.53 |0 |1.35|0.09 |0.42 |
|DESWLsy |0.53 |0 |0.58|0.34 |1.26 |
|PCSYNz |0.51 |0 |0.75|0.12 |1.34 |
|DESWLltd |0.5 |0 |0.25|0.53 |1.26 |
|SMCAUSr |0.47 |0 |1.38|0.11 |0 |
|WRDFRQmc |0.47 |0 |1.21|0.11 |0.33 |
|WRDIMGc |0.45 |0 |0.32|0.27 |1.42 |
|LDMTLD |0.45 |0 |0.51|0.64 |0.25 |
|SMCAUSlsa|0.44 |0 |0.33|0.24 |1.42 |
|SYNSTRUTa|0.43 |0 |1.13|0.22 |0 |
|WRDADV |0.42 |0 |0.56|0.12 |1.17 |
|CNCTempx |0.37 |0 |1.07|0.09 |0 |
|WRDFRQc |0.37 |0 |0.68|0.16 |0.59 |
|CNCLogic |0.36 |0 |0.95|0.17 |0 |
|SMINTEr |0.36 |0 |1.12|0.03 |0 |
|DRINF |0.36 |0 |0.89|0.21 |0 |
|DESWLsyd |0.36 |0 |0.28|0.55 |0.33 |
|SMINTEp |0.35 |0 |0.99|0.07 |0.08 |
|SMCAUSvp |0.33 |0 |0.99|0.06 |0 |
|CNCPos |0.33 |0 |0.17|0.61 |0.25 |
|PCVERBp |0.31 |0 |0.49|0.01 |0.92 |
|WRDPRP3s |0.29 |0 |0.42|0.26 |0.33 |
|PCCNCp |0.29 |0 |0.91|0.03 |0 |
|WRDPRO |0.29 |0 |0.64|0.14 |0.25 |
|WRDFAMc |0.26 |0 |0.55|0.24 |0 |
|DRVP |0.25 |0 |0.57|0.18 |0 |
|SMCAUSv |0.23 |0 |0.69|0.04 |0 |
|SYNLE |0.2 |0 |0.03|0.33 |0.33 |
|PCSYNp |0.2 |0 |0.53|0.02 |0.17 |
|WRDPRP3p |0.19 |0 |0.25|0.29 |0 |
|CNCCaus |0.18 |0 |0.47|0.09 |0 |
|WRDVERB |0.17 |0 |0.1 |0.34 |0 |
|DRNEG |0.03 |0 |0 |0.08 |0 |
Coh-Metrix Model 3per
This model was trained on Coh-Metrix scores from 15 min persuasive writing samples.
Algorithm Weightings in Ensemble
Abbreviations:
overall = ensemble model
pls = partial least squares regression
gbm = stochastic gradient boosted trees
mars = bagged multivariate adaptive regression splines
* cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | cube |
|:----------|:-------|:-------|:-------|:-------|
| -0.0381 | 0.0558 | 0.4924 | 0.4425 | 0.0259 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |mars |gbm |cube |
|---------|-------|----|-----|-----|-----|
|DESWC |32.09 |4.68|34.34|33.8 |19.05|
|WRDHYPn |10.41 |2.03|17.13|4.3 |5.17 |
|LDVOCD |9.59 |3.43|0 |21.44|2.53 |
|DESWLlt |8.27 |1.29|13.16|3.77 |7.4 |
|LSAGN |6.13 |2.92|8.63 |3.88 |4.05 |
|WRDNOUN |4.46 |1.39|7.36 |1.66 |3.95 |
|WRDADV |3.05 |1.18|5.53 |0.52 |3.24 |
|WRDFRQa |2.13 |0.25|3.86 |0.55 |0.2 |
|SMCAUSwn |2.11 |1.08|3.7 |0.54 |1.01 |
|CNCAdd |1.76 |0.66|3.37 |0.19 |0.2 |
|WRDADJ |1.67 |0.59|2.93 |0.26 |4.15 |
|LDTTRa |1.47 |4.05|0 |2.58 |4.76 |
|DESSC |1.38 |3.27|0 |2.6 |2.74 |
|LDTTRc |1.17 |3.17|0 |2.21 |1.32 |
|DESWLltd |0.63 |1.31|0 |1.2 |1.42 |
|WRDPRO |0.57 |1.57|0 |1.1 |0.3 |
|PCDCz |0.5 |1.55|0 |0.94 |0.3 |
|DESSLd |0.47 |1.61|0 |0.84 |0.61 |
|CRFCWO1d |0.47 |2.36|0 |0.74 |0.81 |
|DRNEG |0.46 |1.75|0 |0.72 |1.82 |
|WRDPOLc |0.46 |0.78|0 |0.89 |1.22 |
|SYNNP |0.45 |0.82|0 |0.92 |0 |
|CNCCaus |0.44 |0.24|0 |0.98 |0 |
|WRDPRP3p |0.4 |0.59|0 |0.82 |0.3 |
|WRDHYPv |0.39 |0.94|0 |0.76 |0.2 |
|LDMTLD |0.35 |0.38|0 |0.67 |1.72 |
|WRDAOAc |0.34 |1.9 |0 |0.52 |0.3 |
|CNCPos |0.32 |0.48|0 |0.66 |0 |
|WRDMEAc |0.32 |1.11|0 |0.56 |0.61 |
|CRFANPa |0.31 |1.37|0 |0.52 |0.2 |
|DRVP |0.31 |0.47|0 |0.58 |1.22 |
|WRDFRQc |0.31 |0.22|0 |0.65 |0.71 |
|SMCAUSlsa|0.26 |0.65|0 |0.5 |0 |
|LSASS1d |0.26 |2.04|0 |0.34 |0 |
|CRFAO1 |0.25 |2.03|0 |0.3 |0.2 |
|WRDHYPnv |0.25 |0.33|0 |0.48 |0.71 |
|CNCTempx |0.23 |0.5 |0 |0.32 |2.23 |
|SYNSTRUTa|0.22 |0.75|0 |0.39 |0.41 |
|SYNMEDpos|0.21 |2.04|0 |0.14 |1.42 |
|WRDPRP1s |0.2 |0.81|0 |0.26 |1.62 |
|PCVERBz |0.2 |1.75|0 |0.15 |1.62 |
|CRFCWO1 |0.19 |1.81|0 |0.18 |0.41 |
|RDL2 |0.19 |1.54|0 |0.22 |0.51 |
|PCNARz |0.18 |2.01|0 |0.05 |1.72 |
|LSAGNd |0.18 |2.08|0 |0.1 |0.91 |
|RDFKGL |0.17 |0.52|0 |0.19 |2.13 |
|PCSYNz |0.17 |0.67|0 |0.19 |2.13 |
|LSASSpd |0.15 |1.95|0 |0.08 |0.2 |
|SMCAUSv |0.15 |0.37|0 |0.26 |0.61 |
|SYNLE |0.14 |0.44|0 |0.26 |0 |
|RDFRE |0.14 |0.45|0 |0.14 |2.13 |
|DRNP |0.14 |0.41|0 |0.28 |0 |
|PCDCp |0.14 |1.89|0 |0 |1.62 |
|SMCAUSr |0.13 |1.29|0 |0.13 |0 |
|LSASS1 |0.13 |1.84|0 |0.04 |0.41 |
|CRFCWOad |0.13 |1.93|0 |0.04 |0.3 |
|WRDPRP2 |0.12 |1.32|0 |0.1 |0 |
|WRDFAMc |0.12 |0.32|0 |0.23 |0.1 |
|PCREFz |0.12 |1.13|0 |0.06 |1.42 |
|DESWLsy |0.11 |0.61|0 |0.13 |0.51 |
|CNCLogic |0.11 |0.55|0 |0.17 |0.1 |
|CRFNO1 |0.11 |1.61|0 |0.04 |0.2 |
|DRGERUND |0.1 |0.38|0 |0.16 |0.41 |
|DRPP |0.1 |0.34|0 |0.19 |0 |
|PCTEMPp |0.1 |1.16|0 |0.07 |0.2 |
|SMINTEr |0.1 |1.5 |0 |0.03 |0.2 |
|PCCNCz |0.1 |1.31|0 |0.05 |0.41 |
|DESWLsyd |0.1 |0.76|0 |0.13 |0.3 |
|CNCNeg |0.09 |0.55|0 |0.13 |0.3 |
|WRDVERB |0.08 |0.23|0 |0.15 |0 |
|SMCAUSvp |0.08 |0.24|0 |0.14 |0.2 |
|PCCONNz |0.08 |0.49|0 |0.11 |0.3 |
|PCCONNp |0.07 |1.11|0 |0.01 |0 |
|DRAP |0.07 |0.12|0 |0.14 |0 |
|WRDCNCc |0.07 |0.03|0 |0.14 |0.2 |
|WRDFRQmc |0.07 |1.06|0 |0.03 |0 |
|PCVERBp |0.07 |1.19|0 |0 |0.3 |
|PCREFp |0.06 |0.92|0 |0 |0.3 |
|PCCNCp |0.05 |0.83|0 |0 |0 |
|WRDPRP1p |0.05 |0.16|0 |0.09 |0 |
|WRDIMGc |0.05 |0 |0 |0.09 |0.51 |
|SMINTEp |0.05 |0.71|0 |0.03 |0 |
|DRINF |0.05 |0.3 |0 |0.06 |0.3 |
|CNCADC |0.05 |0.44|0 |0.05 |0.2 |
|CNCTemp |0.04 |0.58|0 |0.02 |0 |
|PCSYNp |0.04 |0.42|0 |0.01 |0.71 |
|DRPVAL |0.01 |0.07|0 |0.01 |0 |
Automated Written Expression CBM (aWE-CBM) Model 1 {#awecbm-model-1}
General Description
Total Words Written (TWW) scores are generated directly from the GAMET word count score. Words Spelled Correctly (WSC) scores are generated by subtracting the GAMET misspelling score from the GAMET word count score.
Correct Word Sequences (CWS) and Correct Minus Incorrect Word Sequences (CIWS) scores are based on ensemble models originally trained to predict CBM scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019]. More details on the sample are available in [@Mercer2019].
The CWS and CIWS models are detailed below (from Mercer et al., 2021).
Correct Word Sequences Model
| Metric | Overall | GBM | SVM | ENET | MARS |
|:------------|:--------|:------|:------|:------|:------|
| Word Count | 75.48 | 86.79 | 67.10 | 77.17 | 77.84 |
| Spelling | 14.26 | 0.62 | 0.00 | 21.41 | 22.05 |
| %Spelling | 8.78 | 12.28 | 27.95 | 0.40 | 0.11 |
| Grammar | 0.85 | 0.05 | 2.77 | 0.11 | 0.00 |
| %Grammar | 0.01 | 0.06 | 0.01 | 0.00 | 0.00 |
| Duplication | 0.04 | 0.12 | 0.12 | 0.00 | 0.00 |
| Typography | 0.38 | 0.08 | 1.33 | 0.00 | 0.00 |
| White Space | 0.20 | 0.00 | 0.71 | 0.92 | 0.00 |
Note. The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following regression equation was used to weight the algorithms in the CWS ensemble model: .162 + .074 * GBM + .281 * SVM + .001 * ENET + .642 * MARS.
Correct Minus Incorrect Word Sequences Model
| Metric | Overall | GBM | SVM | ENET | MARS |
|:------------|:--------|:------|:------|:------|:------|
| Word Count | 55.60 | 55.76 | 47.57 | 61.43 | 61.35 |
| Spelling | 19.25 | 1.48 | 6.57 | 35.80 | 35.04 |
| %Spelling | 22.31 | 41.99 | 42.74 | 0.00 | 0.00 |
| Grammar | 0.82 | 0.00 | 1.69 | 0.00 | 0.62 |
| %Grammar | 0.04 | 0.23 | 0.00 | 0.00 | 0.00 |
| Duplication | 0.28 | 0.10 | 0.76 | 0.00 | 0.00 |
| Typography | 1.37 | 0.41 | 0.07 | 1.55 | 2.97 |
| White Space | 0.34 | 0.04 | 0.60 | 1.22 | 0.00 |
Note. The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following equation was used for the CIWS model: -.170 + .180 * GBM + .346 * SVM + .100 * ENET + .375 * MARS.
References
Try the writeAlizer package in your browser
Any scripts or data that you put into this service are public.
writeAlizer documentation built on Sept. 17, 2026, 1:08 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
This reference describes how the scoring models were developed. For installation, importing files, and generating scores, start with the getting-started guide.
How to use this reference
Use the contents list to jump to your model. Each model section describes its training data, the algorithms combined in its predictions, and the features those algorithms used. An ensemble combines predictions from several algorithms. Weight tables describe that combination; feature-importance tables describe relative contributions, not causal effects.
These are historical model-development summaries. writeAlizer loads saved models rather than retraining them. During scoring, Models 2 and 3 standardize predictors within the submitted group of texts, so changing that group can change scores. Training-time feature selection described below is distinct from this scoring step.
The detailed tables are wide. On a small screen, scroll a table horizontally to see all algorithms.
Recommended Models for Use
ReaderBench-Model-3 and Coh-Metrix-Model-3 are the recommended models for generating predicted writing quality scores, and aWE-CBM-Model-1 is the best available model for generating automated written expression curriculum-based measurement scores.
Scoring Model Development {#scoring-model-development}
The general process used to generate all scoring models is presented below.
Predictive Algorithms and R Packages Used
The caret and caretEnsemble packages were used as wrappers for the following predictive algorithms:
-
Random forest regression (package
randomForest) -
Cubist regression (package
Cubist) -
Support vector machines with a radial kernel (package
kernlab) -
Bagged multivariate adaptive regression splines (package
earth) -
Stochastic gradient boosted trees (package
gbm) -
Partial least squares regression (package
pls) -
Elasticnet regression (package
elasticnet)
These algorithms are described in detail in the following references (among others):
-
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer. https://doi.org/10.1007/b94608
-
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer. https://doi.org/10.1007/978-1-4614-6849-3
Steps
The following flowchart provides an overview of the scoring model development workflow, with more details on some steps provided below.
1. Import Data
Depending on the specific scoring model, ReaderBench, Coh-Metrix, and/or GAMET output files were imported into R using functions similar to the import_XXXX.R functions in writeAlizer (see https://shmercer.github.io/writeAlizer/reference/index.html)
2. Pre-Process Data
Automated data pre-processing was done using the preProcess() function in caret:
-
Predictors from the output file with near zero variance (defined based on defaults in the
nearZeroVar()function) were removed, and the remaining predictors were standardized. -
Highly correlated | r > .90 | predictors were identified, with the predictor that had the highest mean correlation with all of the other predictors removed.
-
The reduced set of predictors was submitted to the next step of the analysis.
3. Determine Optimal Tuning Parameters
The following tuning hyperparameters were optimized based on resampling (repeated 10 fold) in caret. Each algorithm was tuned separately. Full descriptions of the tuning parameters are available in each package's documentation.
-
Random forest regression:
mtry -
Cubist regression:
committees,neighbors -
Support vector machines:
sigma,C -
Bagged multivariate adaptive regression splines:
nprune,degree -
Stochastic gradient boosted trees:
n.trees,interaction.depth,shrinkage,n.minobsinnode -
Partial least squares regression:
ncomp -
Elastic net regression:
fraction,lambda
4. Final/Optimal Model for each Algorithm
A model for each algorithm was fit with the hyperparameters set to the optimal values found in Step 3, with bootstrapped (1000 samples) resampling-based cross-validation so that an ensemble model (weighting each algorithm) could be built based on the resamples. This step was done with the caretList() function of the caretEnsemble package. This process is illustrated in more detail in the caretEnsemble vignette: https://zachmayer.github.io/caretEnsemble/articles/caretEnsemble-intro.html
5. Estimate an Ensemble Model to Combine the Algorithms
The caretEnsemble() function was used to determine the optimal linear weighting of the algorithms that minimized RMSE (i.e., discrepancy between actual writing quality scores and predicted quality scores) in the resamples from Step 4. The tables below report the fitted algorithm weights, including small or negative coefficients where present.
The varImp() function of caretEnsemble was used to generate estimates of relative predictor importance for the overall ensemble model and for each individual algorithm.
6. Generate Predicted Quality Scores from each Ensemble
The predict() function of caretEnsemble was used to generate/store predicted quality scores for the ensemble models.
7. Average Scores to get Final Predicted Quality Scores
The predicted scores from each ensemble were averaged to produce the final predictions.
ReaderBench Model 1 {#readerbench-model-1}
General Description
Model 1 has been replaced by the greatly simplified Model 2 that better handles multi-paragraph compositions. Model 2 is recommended over Model 1.
Model 1 is an ensemble (formed by averaging predicted quality scores) of the six sub-models described below.
All of these sub-models used ReaderBench scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
This scoring model was evaluated in the following publications: [@Matta2022; @Mercer2022; @Keller-Margulis2021]
ReaderBench Model 1a
This model was trained on fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:--------|:-------|:--------|:--------|:-------|:-------|:-------| | -3.9077 | -0.1323 | 0.4789 | -0.0963 | -0.0361 | 0.3985 | 0.1297 | 0.3442 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------------------------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | WdEnt | 14.61 | 40.66 | 2.49 | 1.93 | 13.17 | 3.97 | 62.12 | 19.56 | | LxcDiv | 3.02 | 4.3 | 2.2 | 1.52 | 2.69 | 2.77 | 0 | 5.55 | | AvgUnqPrepositionBl | 2.88 | 3.21 | 2.18 | 1.42 | 6.67 | 1.98 | 0 | 5.84 | | AvgNmdEntBl | 2.71 | 0.47 | 1.13 | 0.4 | 0.79 | 0.89 | 18.1 | 2.92 | | AvgUnqVerbBl | 2.4 | 2.36 | 2.21 | 1.48 | 6.53 | 2.33 | 0 | 3.5 | | WdDiffLemmaStem | 2.07 | 1.07 | 0.82 | 0.91 | 0 | 1.35 | 13 | 1.46 | | RdbltyFlesch | 1.91 | 0.7 | 0.41 | 0.41 | 4.1 | 1.04 | 6.51 | 3.94 | | AvgChainSpan | 1.84 | 3.5 | 1.77 | 1.16 | 0 | 2.11 | 0 | 2.04 | | AvgDepsBl_det | 1.64 | 2.01 | 1.63 | 0.79 | 3.96 | 1.6 | 0 | 2.19 | | AvgBlScore | 1.55 | 0.87 | 2.01 | 1.32 | 0 | 1.76 | 0 | 1.75 | | AvgPronBl_first_person | 1.52 | 0.7 | 1.56 | 0.75 | 2.38 | 1.39 | 0 | 2.63 | | AvgDepsBl_nsubj | 1.5 | 1.34 | 2.18 | 1.45 | 0 | 1.9 | 0 | 0.88 | | AvgUnqAdverbBl | 1.48 | 1.86 | 1.88 | 1.06 | 1.3 | 2.12 | 0 | 0.73 | | AvgDepsBl_punct | 1.45 | 1.45 | 1.77 | 0.93 | 1.9 | 2.1 | 0 | 0.88 | | WdDiffWdStem | 1.43 | 2.02 | 1.47 | 0.84 | 1.36 | 1.04 | 0 | 2.34 | | AvgDepsSen_punct | 1.41 | 0.64 | 1.33 | 0.65 | 4.76 | 1.04 | 0 | 2.63 | | AvgDepsSen_dep | 1.34 | 0.04 | 0.65 | 0.33 | 5.78 | 0.53 | 0 | 4.09 | | AvgSenScore | 1.28 | 0.39 | 0.27 | 0.22 | 0 | 0.53 | 0 | 4.82 | | AvgPronounBl | 1.23 | 0.2 | 1.8 | 1.08 | 0 | 1.38 | 0 | 1.31 | | TCorefChainDoc | 1.2 | 0.41 | 1.59 | 0.77 | 0 | 0.88 | 0 | 2.04 | | AvgDepsBl_nmod | 1.12 | 0.52 | 1.9 | 1.09 | 0 | 1.57 | 0 | 0.29 | | AvgAOASen_Shock | 1.09 | 1.04 | 0.77 | 0.52 | 3.75 | 1.04 | 0 | 1.9 | | AvgSenBl | 1.03 | 0.42 | 1.51 | 0.68 | 0 | 1.29 | 0 | 0.88 | | WdLettStdDev | 1.01 | 0.85 | 1.27 | 0.8 | 0 | 0.84 | 0 | 1.46 | | AvgWdLen | 0.97 | 1.33 | 1.4 | 0.75 | 0 | 1.26 | 0 | 0.44 | | AvgDepsBl_nummod | 0.95 | 0.75 | 1.16 | 0.42 | 0.04 | 1.23 | 0 | 1.02 | | AvgCorefChain | 0.94 | 0.56 | 1.01 | 0.34 | 2.87 | 0.32 | 0 | 2.04 | | TActCorefChainWd | 0.93 | 0.78 | 0.83 | 0.55 | 0.99 | 1.15 | 0 | 1.31 | | RdbltyDaleChall | 0.92 | 0.93 | 0.95 | 0.38 | 0 | 1.07 | 0.27 | 1.17 | | AvgDepsBl_advmod | 0.89 | 0.2 | 1.65 | 0.84 | 1 | 1.26 | 0 | 0 | | AvgAOABl_Bristol | 0.86 | 0.73 | 0.93 | 0.41 | 0 | 0.52 | 0 | 1.75 | | AvgUnqNoundBl | 0.85 | 0.28 | 1.64 | 0.82 | 0.65 | 0.88 | 0 | 0.29 | | AvgDepsBl_dobj | 0.85 | 0.19 | 1.49 | 0.68 | 1.5 | 0.93 | 0 | 0.44 | | AvgAOABl_Shock | 0.83 | 2.04 | 1.3 | 0.63 | 0.12 | 0.96 | 0 | 0 | | SenStdDevWd | 0.77 | 0.67 | 1.07 | 0.56 | 0.83 | 1.42 | 0 | 0 | | AvgDepsBl_mark | 0.76 | 0.15 | 1.31 | 0.53 | 1.23 | 0.7 | 0 | 0.58 | | LexChainMaxSp | 0.75 | 0.31 | 1.57 | 0.75 | 0 | 0.85 | 0 | 0 | | WdAvgDpthHypernymTree | 0.75 | 0.42 | 0.63 | 0.36 | 0.57 | 0.6 | 0 | 1.61 | | AvgPronBl_indefinite | 0.72 | 0.12 | 1.44 | 0.64 | 0 | 0.62 | 0 | 0.44 | | AvgSenAdjCoh_Path | 0.69 | 0.35 | 0.83 | 0.49 | 0.57 | 0.77 | 0 | 0.88 | | AvgDepsBl_cop | 0.68 | 0.15 | 1.2 | 0.44 | 2.55 | 0.92 | 0 | 0 | | CharEnt | 0.68 | 0.26 | 1.44 | 0.69 | 0.42 | 0.73 | 0 | 0 | | AvgConnBl_simp_subords | 0.66 | 0.11 | 1.2 | 0.45 | 0.31 | 1.05 | 0 | 0 | | LexChainAvgSpan | 0.66 | 0.49 | 1.17 | 0.7 | 0 | 0.94 | 0 | 0 | | AvgConnBl_reas_purp | 0.66 | 0.11 | 1.26 | 0.5 | 0 | 1.02 | 0 | 0 | | AvgDepsBl_advcl | 0.64 | 0.05 | 1.35 | 0.56 | 1.13 | 0.7 | 0 | 0 | | TCorefChainBigSpan | 0.63 | 0.01 | 1.16 | 0.42 | 1.31 | 0.54 | 0 | 0.44 | | AvgUnqAdjectiveBl | 0.63 | 0.48 | 1.37 | 0.58 | 0 | 0.61 | 0 | 0 | | AvgDepsBl_amod | 0.63 | 0.05 | 1.28 | 0.51 | 0.32 | 0.85 | 0 | 0 | | AvgDepsBl_aux | 0.62 | 0.57 | 0.98 | 0.3 | 0 | 1.07 | 0 | 0 | | WdPathCntHypernymTree | 0.62 | 0.99 | 0.79 | 0.44 | 3.29 | 0.7 | 0 | 0.15 | | WdSylCnt | 0.62 | 0.37 | 0.8 | 0.56 | 0 | 1.3 | 0 | 0 | | AvgUnqPronounBl | 0.61 | 0.21 | 1.3 | 0.53 | 0.62 | 0.65 | 0 | 0 | | FrqRhythmId | 0.59 | 0.73 | 0.9 | 0.44 | 0 | 0.94 | 0 | 0 | | AvgDepsBl_nsubjpass | 0.57 | 0.05 | 1.01 | 0.32 | 1.18 | 0.88 | 0 | 0 | | AvgDepsBl_ccomp | 0.55 | 0.26 | 1.13 | 0.39 | 1.76 | 0.54 | 0 | 0 | | AvgConnSen_addition | 0.54 | 0.25 | 0.59 | 0.25 | 0 | 0.95 | 0 | 0.44 | | AvgConnBl_order | 0.52 | 0.07 | 0.72 | 0.16 | 2.74 | 0.92 | 0 | 0 | | AvgBlVoiceCoOcc | 0.52 | 0 | 1.09 | 0.36 | 0 | 0.73 | 0 | 0 | | AvgInferenceDistChain | 0.51 | 0.32 | 0.74 | 0.45 | 0.3 | 0.79 | 0 | 0.15 | | AvgAOESen_InvLinRegSlo | 0.5 | 0.56 | 0.65 | 0.35 | 0.86 | 0.28 | 0 | 0.73 | | AvgRhythmUnitStreesSyll | 0.49 | 0.29 | 0.03 | 0.3 | 0 | 1.02 | 0 | 0.88 | | AvgConnBl_semi_coords | 0.48 | 0.01 | 0.98 | 0.3 | 0 | 0.68 | 0 | 0 | | LxcSoph | 0.48 | 0.25 | 0.83 | 0.32 | 0 | 0.8 | 0 | 0 | | AvgConnBl_logical_cons | 0.47 | 0.05 | 0.58 | 0.1 | 1.58 | 0.63 | 0 | 0.44 | | AvgDepsBl_neg | 0.47 | 0.04 | 0.81 | 0.21 | 0 | 0.86 | 0 | 0 | | AvgCommaBl | 0.47 | 0.09 | 0.9 | 0.25 | 0 | 0.75 | 0 | 0 | | AvgNounNmdEntBl | 0.47 | 0.08 | 0.67 | 0.14 | 0 | 0.43 | 0 | 0.73 | | AvgNounSen | 0.46 | 0.23 | 0.12 | 0.07 | 0 | 0.62 | 0 | 1.17 | | AvgAdverbSen | 0.45 | 0.17 | 0.46 | 0.55 | 0 | 0.81 | 0 | 0.29 | | AvgConnBl_oppositions | 0.44 | 0.07 | 0.9 | 0.26 | 0 | 0.62 | 0 | 0 | | AvgNmdEntSen | 0.44 | 0.62 | 0.13 | 0.5 | 0 | 0.93 | 0 | 0.44 | | AvgConnBl_contrasts | 0.44 | 0.25 | 1.02 | 0.32 | 0 | 0.41 | 0 | 0 | | AvgConnSen_semi_coords | 0.43 | 0.05 | 0.43 | 0.06 | 0.08 | 0.93 | 0 | 0.29 | | AvgAOASen_Bristol | 0.43 | 0.48 | 0.32 | 0.48 | 1.36 | 0.7 | 0 | 0.29 | | AvgDepsBl_xcomp | 0.43 | 0.11 | 1.15 | 0.41 | 0 | 0.23 | 0 | 0 | | AvgConnSen_simp_subords | 0.43 | 0.74 | 0.28 | 0.31 | 1.49 | 0.97 | 0 | 0 | | WdPolysemyCnt | 0.41 | 0.4 | 0 | 0.71 | 0.46 | 0.17 | 0 | 1.31 | | AvgPronBl_third_person | 0.41 | 0.39 | 0.88 | 0.24 | 0 | 0.44 | 0 | 0 | | AvgAOASen_Bird | 0.4 | 0.25 | 0.58 | 0.51 | 0 | 0.7 | 0 | 0 | | AvgDepsBl_mwe | 0.4 | 0.03 | 0.49 | 0.08 | 3.31 | 0.71 | 0 | 0 | | AvgPronounSen | 0.38 | 0.27 | 0.06 | 0.18 | 0 | 0.59 | 0 | 0.88 | | AvgConnBl_addition | 0.37 | 0.23 | 0.58 | 0.1 | 0 | 0.71 | 0 | 0 | | AvgAOABl_Kuperman | 0.37 | 0.6 | 0.32 | 0.48 | 0 | 0.43 | 0 | 0.44 | | AvgAOASen_Kuperman | 0.36 | 0.16 | 0.62 | 0.55 | 0 | 0.52 | 0 | 0 | | AvgAOABl_Cortese | 0.36 | 0.33 | 0.3 | 0.59 | 2.07 | 0.66 | 0 | 0 | | AvgDepsSen_advcl | 0.35 | 0.22 | 0.5 | 0.51 | 0 | 0.6 | 0 | 0 | | AvgDepsSen_det | 0.33 | 0.32 | 0.08 | 0.27 | 0 | 0.31 | 0 | 0.88 | | AvgDepsBl_acl | 0.33 | 0.01 | 0.54 | 0.1 | 0 | 0.67 | 0 | 0 | | AvgRhythmUnits | 0.33 | 0.54 | 0.6 | 0.43 | 0 | 0.22 | 0 | 0.15 | | AggPronSen_indefinite | 0.32 | 0.14 | 0.24 | 0.49 | 0.23 | 0.55 | 0 | 0.29 | | AvgConnBl_temp_cons | 0.31 | 0.16 | 0.87 | 0.24 | 0 | 0.1 | 0 | 0 | | AvgDepsSen_ccomp | 0.3 | 0.19 | 0.04 | 0.34 | 0 | 0.79 | 0 | 0.29 | | SenAsson | 0.29 | 0.02 | 0.43 | 0.06 | 1 | 0.57 | 0 | 0 | | AggPronSen_third_person | 0.29 | 0.46 | 0.3 | 0.15 | 0 | 0.52 | 0 | 0.15 | | AvgAOABl_Bird | 0.29 | 0.27 | 0.46 | 0.53 | 0 | 0.3 | 0 | 0.15 | | AvgDepsSen_aux | 0.29 | 0.08 | 0.12 | 0.49 | 0 | 0.9 | 0 | 0 | | AvgDepsSen_dobj | 0.28 | 0.1 | 0.11 | 0.06 | 0 | 0.95 | 0 | 0 | | AvgConnSen_oppositions | 0.26 | 0.03 | 0.28 | 0.03 | 0 | 0.69 | 0 | 0 | | AvgDepsSen_nmod | 0.26 | 0.18 | 0.37 | 0.37 | 0 | 0.47 | 0 | 0 | | AvgDepsSen_amod | 0.25 | 0.26 | 0.17 | 0.1 | 0 | 0.47 | 0 | 0.29 | | AvgAOASen_Cortese | 0.24 | 0.2 | 0.59 | 0.44 | 0 | 0.08 | 0 | 0 | | AvgDepsSen_compound | 0.24 | 0.16 | 0.38 | 0.18 | 0 | 0.43 | 0 | 0 | | AvgDepsSen_mark | 0.23 | 0.3 | 0.23 | 0.27 | 0 | 0.48 | 0 | 0 | | AvgDepsSen_mwe | 0.22 | 0.1 | 0.2 | 0.02 | 0 | 0.6 | 0 | 0 | | AvgAdjectiveSen | 0.22 | 0.16 | 0.04 | 0.08 | 0 | 0.78 | 0 | 0 | | AvgAOEBl_IndPolyFAT.3 | 0.21 | 0.26 | 0.05 | 0.29 | 0.04 | 0.24 | 0 | 0.44 | | AvgAOEBl_InvLinRegSlo | 0.21 | 0.44 | 0.17 | 0.25 | 0 | 0.43 | 0 | 0 | | AvgDepsBl_dep | 0.21 | 0.09 | 0.17 | 0.01 | 0.69 | 0.57 | 0 | 0 | | AvgDepsSen_xcomp | 0.2 | 0.26 | 0 | 0.39 | 0 | 0.4 | 0 | 0.29 | | AvgDepsSen_cop | 0.19 | 0.26 | 0.13 | 0.36 | 0 | 0.31 | 0 | 0.15 | | AvgDepsBl_compound | 0.18 | 0.07 | 0.17 | 0.01 | 0.45 | 0.23 | 0 | 0.29 | | LangRhythmDiameter | 0.16 | 0.18 | 0.29 | 0.03 | 0.86 | 0.14 | 0 | 0 | | AvgConnSen_temp_cons | 0.13 | 0.3 | 0.18 | 0.42 | 0 | 0.09 | 0 | 0 | | AvgAOEBl_IndAbThr.0.3. | 0.12 | 0.31 | 0.01 | 0.33 | 0.04 | 0.27 | 0 | 0 | | AvgDepsSen_acl | 0.12 | 0.04 | 0.13 | 0.01 | 0 | 0.3 | 0 | 0 | | AvgConnSen_order | 0.12 | 0.21 | 0.17 | 0.01 | 0 | 0.23 | 0 | 0 | | LangRhythmCoeff | 0.12 | 0.41 | 0 | 0.23 | 0 | 0.31 | 0 | 0 | | AvgSenBlCoh_LeackChod | 0.12 | 0.06 | 0.08 | 0.42 | 0 | 0.28 | 0 | 0 | | AvgUnqWdBl | 0.11 | 0 | 0 | 1.79 | 0 | 0 | 0 | 0 | | AvgBlLen | 0.11 | 0 | 0 | 1.8 | 0 | 0 | 0 | 0 | | AvgVerbBl | 0.1 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 | | AvgDepsSen_neg | 0.1 | 0.05 | 0.02 | 0 | 0.08 | 0.37 | 0 | 0 | | Words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | Content.words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | AvgDepsBl_case | 0.08 | 0 | 0 | 1.27 | 0 | 0 | 0 | 0 | | AvgNounBl | 0.07 | 0 | 0 | 1.15 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.07 | 0.02 | 0.23 | 0.02 | 0 | 0 | 0 | 0 | | AvgPrepositionBl | 0.07 | 0 | 0 | 1.21 | 0 | 0 | 0 | 0 | | AvgAdverbBl | 0.05 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.04 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.04 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.04 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.04 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 | | SynSoph | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | Sentences | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LSA | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.04 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.03 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 | | AvgConnSen_reas_purp | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.03 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.03 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.03 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgDepsSen_advmod | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | AvgAdjectiveBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgConnBl_sentence_link | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.03 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_Path | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.03 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.02 | 0 | 0 | 0.26 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgDepsSen_conj | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.02 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.02 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conns | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOESen_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LDA | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgDepsSen_nsubj | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgAOESen_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | AvgVoice | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.02 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | RdbltyKincaid | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.02 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conns | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.01 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | AvgDepsBl_cc | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnSen_logical_cons | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0 | 0 | 0 | 0.06 | 0 | 0 | 0 | 0 |
ReaderBench Model 1b
This model was trained on winter data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:------| | -2.0039 | 0.3112 | 0.1353 | 0.2667 | -0.0102 | 0.1234 | 0.0268 | 0.222 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:--------------------------|:------|:------|:-----|:-----|:-----|:-----|:------|:------| | WdEnt | 11.34 | 23.56 | 2.43 | 1.65 | 0.74 | 3.11 | 31.83 | 12.9 | | AvgDepsBl_det | 6.98 | 12.06 | 2.13 | 1.17 | 2.37 | 2.1 | 9.48 | 12.41 | | AvgPronounBl | 3.71 | 2.92 | 1.9 | 0.99 | 0 | 1.27 | 7.58 | 10.22 | | AvgUnqVerbBl | 3.43 | 6.53 | 2.29 | 1.36 | 1.89 | 2.17 | 0 | 3.65 | | AvgDepsBl_nsubj | 2.97 | 5.88 | 2.3 | 1.43 | 1.14 | 2.11 | 0 | 2.19 | | AvgUnqPrepositionBl | 2.78 | 4.75 | 2.01 | 1.04 | 1.59 | 1.86 | 0 | 3.65 | | LxcDiv | 2.6 | 4 | 2.33 | 1.44 | 0.22 | 1.92 | 0 | 3.16 | | WdDiffWdStem | 1.99 | 0.84 | 0.97 | 0.5 | 0.82 | 0.63 | 0 | 7.3 | | AvgBlScore | 1.56 | 2.98 | 1.74 | 1.22 | 1.18 | 1.81 | 0 | 0 | | AvgDepsBl_punct | 1.52 | 2.67 | 1.81 | 0.85 | 0.54 | 1.23 | 0 | 0.97 | | AvgUnqNoundBl | 1.48 | 0.02 | 1.81 | 0.84 | 0.31 | 1.09 | 11.62 | 2.68 | | AggPronSen_third_person | 1.37 | 0.36 | 0.55 | 0.21 | 0.28 | 0.67 | 18.65 | 2.19 | | TCorefChainDoc | 1.3 | 0.78 | 1.88 | 0.91 | 0.67 | 1.7 | 0 | 2.19 | | RdbltyFlesch | 1.3 | 2.18 | 0.2 | 0.44 | 1.65 | 0.98 | 0 | 2.19 | | AvgPronBl_first_person | 1.19 | 0 | 1.69 | 0.74 | 1.34 | 0.66 | 0 | 3.65 | | AvgSenBl | 1.16 | 2.23 | 1.8 | 0.84 | 0.27 | 0.97 | 0 | 0 | | AvgDepsSen_advcl | 1.14 | 0.06 | 0.01 | 0.49 | 0.72 | 0.76 | 0 | 4.62 | | AvgSenBlCoh_LeackChod | 1.12 | 1.42 | 1.49 | 0.61 | 0.42 | 1.35 | 0 | 1.22 | | AvgSenBlCoh_LDA | 1.07 | 0.19 | 0.94 | 0.65 | 1.67 | 0.94 | 15.45 | 0.49 | | LexChainMaxSp | 0.99 | 1.34 | 1.69 | 0.74 | 2.27 | 1.78 | 0 | 0 | | AvgDepsBl_mark | 0.94 | 0.2 | 1.21 | 0.38 | 1.95 | 0.86 | 0 | 2.68 | | CharEnt | 0.86 | 0.48 | 1.61 | 0.8 | 0.84 | 0.82 | 0 | 1.22 | | AvgWdLen | 0.85 | 0.9 | 1.12 | 0.76 | 0.69 | 0.73 | 0 | 0.97 | | AvgChainSpan | 0.79 | 0.68 | 1.73 | 1.07 | 1.32 | 1.08 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.73 | 0.22 | 0.35 | 0.23 | 0.12 | 0.39 | 0 | 2.68 | | AvgDepsSen_det | 0.72 | 0.92 | 0.1 | 0.35 | 0.83 | 0.57 | 0 | 1.46 | | AvgDepsSen_dobj | 0.69 | 0.6 | 0.53 | 0.4 | 0.94 | 0.52 | 0 | 1.46 | | AvgUnqPronounBl | 0.66 | 0.14 | 1.72 | 0.76 | 0.12 | 1.13 | 0 | 0.49 | | AvgConnBl_temp_conns | 0.65 | 1.22 | 1.12 | 0.33 | 1.18 | 0.69 | 0 | 0 | | AvgDepsSen_compound | 0.64 | 0.99 | 0.55 | 0.18 | 1.46 | 1.18 | 0 | 0.49 | | AvgSenAdjCoh_Path | 0.64 | 0.78 | 1.29 | 0.71 | 0.77 | 0.79 | 0 | 0 | | WdDiffLemmaStem | 0.63 | 1.33 | 0.19 | 0.58 | 0.14 | 0.74 | 0 | 0 | | RdbltyDaleChall | 0.63 | 1.19 | 1.14 | 0.4 | 0.66 | 0.48 | 0 | 0 | | WdMaxDpthHypernymTree | 0.59 | 1.07 | 0.69 | 0.6 | 0.39 | 0.51 | 0 | 0 | | LexChainAvgSpan | 0.57 | 0.28 | 0.88 | 0.7 | 1.27 | 0.59 | 3.91 | 0 | | SenStdDevWd | 0.54 | 0.55 | 0.97 | 0.56 | 0.03 | 0.68 | 0 | 0.24 | | AvgDepsSen_amod | 0.54 | 0.18 | 0.25 | 0.34 | 0.58 | 0.73 | 0 | 1.46 | | AvgDepsBl_dobj | 0.54 | 0.02 | 1.67 | 0.72 | 2.01 | 0.84 | 0 | 0.24 | | AvgDepsSen_dep | 0.52 | 0.87 | 0.47 | 0.4 | 0.18 | 1.08 | 0 | 0 | | AvgDepsBl_nmod | 0.52 | 0.05 | 1.74 | 0.78 | 0.19 | 0.95 | 0 | 0 | | FrqRhythmId | 0.51 | 0.56 | 1.17 | 0.42 | 0.91 | 0.9 | 0 | 0 | | AvgDepsBl_advcl | 0.5 | 0.1 | 1.08 | 0.3 | 1.09 | 0.56 | 0 | 0.97 | | AvgCorefChain | 0.5 | 0.3 | 1.13 | 0.49 | 1.28 | 0.45 | 0 | 0.49 | | AvgDepsSen_nmod | 0.49 | 0.06 | 0.34 | 0.5 | 0.22 | 0.58 | 0 | 1.22 | | AvgConnSen_addition | 0.49 | 0.1 | 0.62 | 0.49 | 0.44 | 0.68 | 0 | 0.97 | | AvgAdjectiveBl | 0.48 | 0.23 | 1.51 | 0.59 | 0.56 | 0.7 | 0 | 0 | | WdLettStdDev | 0.46 | 0.17 | 1.22 | 0.74 | 0.3 | 0.72 | 0 | 0 | | AvgPronBl_indefinite | 0.44 | 0.23 | 1.25 | 0.4 | 0.87 | 1.02 | 0 | 0 | | AvgConnBl_addition | 0.44 | 0.63 | 0.99 | 0.25 | 0.74 | 0.66 | 0 | 0 | | AvgDepsSen_mark | 0.44 | 0.16 | 0.1 | 0.39 | 1.46 | 0.73 | 0 | 0.97 | | LangRhythmCoeff | 0.44 | 0.71 | 0.54 | 0.34 | 0.04 | 0.82 | 0 | 0 | | AvgVoice | 0.43 | 0.26 | 1.31 | 0.44 | 0.83 | 0.69 | 0 | 0 | | AvgUnqAdverbBl | 0.43 | 0.03 | 1.52 | 0.6 | 0.67 | 0.76 | 0 | 0 | | AvgAOABl_Shock | 0.43 | 0.25 | 1.21 | 0.44 | 0.69 | 0.88 | 0 | 0 | | AvgBlLen | 0.41 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 | | AvgPronBl_third_person | 0.41 | 0.2 | 1.12 | 0.32 | 0.64 | 0.33 | 0 | 0.49 | | AvgUnqWdBl | 0.41 | 0 | 0 | 1.7 | 0 | 0 | 0 | 0 | | Content.words | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 | | Words | 0.39 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.39 | 0.14 | 0.34 | 0.24 | 0.81 | 0.48 | 0 | 0.97 | | TCorefChainBigSpan | 0.38 | 0 | 1.06 | 0.29 | 1.22 | 1.08 | 1.47 | 0 | | AvgDepsBl_cop | 0.37 | 0 | 1.4 | 0.51 | 1.32 | 0.58 | 0 | 0 | | AvgConnSen_semi_coords | 0.37 | 0.06 | 0.12 | 0.29 | 0.84 | 0.63 | 0 | 0.97 | | AvgVerbBl | 0.37 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 | | AvgDepsSen_xcomp | 0.35 | 0.38 | 0.05 | 0.6 | 1.2 | 0.67 | 0 | 0 | | AvgRhythmUnitStreesSyll | 0.34 | 0.4 | 0.2 | 0.33 | 1.14 | 0.93 | 0 | 0 | | AvgAOEBl_InvLinRegSlo | 0.34 | 0.7 | 0.09 | 0.23 | 0.67 | 0.59 | 0 | 0 | | AvgNmdEntSen | 0.34 | 0.3 | 0.21 | 0.4 | 0.48 | 1.1 | 0 | 0 | | AvgConnBl_coord_connects | 0.34 | 0 | 1.25 | 0.4 | 1.85 | 0.63 | 0 | 0 | | AvgAOABl_Bird | 0.34 | 0.25 | 0.44 | 0.41 | 1.53 | 0.92 | 0 | 0 | | AvgDepsBl_xcomp | 0.33 | 0.18 | 1.15 | 0.34 | 0 | 0.45 | 0 | 0 | | AvgAdverbSen | 0.33 | 0.27 | 0.2 | 0.4 | 2.52 | 0.97 | 0 | 0 | | AvgConnBl_reas_purp | 0.33 | 0 | 0.96 | 0.24 | 1.1 | 0.45 | 0 | 0.49 | | AvgAdverbBl | 0.33 | 0.04 | 1.29 | 0.43 | 0.75 | 0.5 | 0 | 0 | | TActCorefChainWd | 0.33 | 0.62 | 0.39 | 0.29 | 0.22 | 0.34 | 0 | 0 | | AvgAOABl_Bristol | 0.32 | 0.11 | 0.75 | 0.33 | 0.23 | 1 | 0 | 0 | | AvgAOEBl_IndexPolyFAT.3 | 0.32 | 0.58 | 0 | 0.44 | 0.44 | 0.43 | 0 | 0 | | AvgDepsBl_aux | 0.32 | 0 | 0.52 | 0.07 | 0.8 | 0.39 | 0 | 0.97 | | AvgConnSen_logical_conns | 0.31 | 0.13 | 0.7 | 0.47 | 0.7 | 0.55 | 0 | 0 | | AvgDepsSen_punct | 0.31 | 0.05 | 0.9 | 0.55 | 0.75 | 0.45 | 0 | 0 | | AvgCommaBl | 0.3 | 0 | 1.16 | 0.35 | 0.23 | 0.69 | 0 | 0 | | AvgDepsSen_aux | 0.29 | 0.02 | 0.45 | 0.36 | 0.43 | 1.18 | 0 | 0 | | AvgNounBl | 0.29 | 0 | 0 | 1.2 | 0 | 0 | 0 | 0 | | WdPolysemyCnt | 0.29 | 0.56 | 0.15 | 0.14 | 0.03 | 0.68 | 0 | 0 | | AvgConnSen_reas_purp | 0.29 | 0.02 | 0.16 | 0.26 | 1.13 | 0.85 | 0 | 0.49 | | AvgDepsBl_compound | 0.29 | 0.02 | 0.12 | 0 | 1.6 | 0.57 | 0 | 0.97 | | AvgDepsBl_amod | 0.28 | 0 | 1.1 | 0.32 | 0.78 | 0.53 | 0 | 0 | | AvgAOASen_Bird | 0.28 | 0.51 | 0.15 | 0.23 | 1.13 | 0.46 | 0 | 0 | | WdPathCntHypernymTree | 0.28 | 0.06 | 0.7 | 0.51 | 0.06 | 0.52 | 0 | 0 | | AvgDepsBl_acl | 0.27 | 0.03 | 1 | 0.26 | 0.7 | 0.59 | 0 | 0 | | AvgDepsBl_ccomp | 0.27 | 0 | 0.9 | 0.21 | 0.92 | 0.86 | 0 | 0 | | AvgAOASen_Shock | 0.27 | 0.02 | 0.53 | 0.5 | 0.19 | 0.66 | 0 | 0 | | AggPronSen_indefinite | 0.27 | 0.07 | 0.04 | 0.61 | 0.75 | 0.79 | 0 | 0 | | AvgDepsSen_cop | 0.27 | 0.16 | 0.03 | 0.55 | 0.5 | 0.75 | 0 | 0 | | AvgConnBl_simp_subords | 0.27 | 0.06 | 1.07 | 0.3 | 0.95 | 0.41 | 0 | 0 | | AvgConnBl_semi_coords | 0.26 | 0 | 0.68 | 0.12 | 0.22 | 0.42 | 0 | 0.49 | | AvgAOESen_IndexAbThr.0.3. | 0.26 | 0.1 | 0.23 | 0.53 | 0.02 | 0.64 | 0 | 0 | | AvgAOEBl_IndexAbThr.0.3. | 0.26 | 0.21 | 0 | 0.61 | 0.28 | 0.43 | 0 | 0 | | WdSylCnt | 0.25 | 0.15 | 0.39 | 0.48 | 0.97 | 0.3 | 0 | 0 | | AvgDepsBl_neg | 0.25 | 0.03 | 0.86 | 0.19 | 0.09 | 0.84 | 0 | 0 | | AvgConnBl_contrasts | 0.23 | 0 | 0.73 | 0.14 | 0.65 | 0.85 | 0 | 0 | | AvgNounSen | 0.23 | 0.05 | 0.42 | 0.24 | 0.23 | 0.89 | 0 | 0 | | AvgDepsSen_ccomp | 0.23 | 0.18 | 0.07 | 0.29 | 0.47 | 0.83 | 0 | 0 | | AvgConnBl_logical_conns | 0.23 | 0.16 | 0.87 | 0.19 | 0.05 | 0.27 | 0 | 0 | | AvgConnSen_simp_subords | 0.23 | 0.1 | 0.05 | 0.54 | 0 | 0.54 | 0 | 0 | | AvgAdjectiveSen | 0.23 | 0.1 | 0.21 | 0.34 | 0.85 | 0.79 | 0 | 0 | | AvgConnBl_oppositions | 0.23 | 0 | 0.79 | 0.16 | 0.75 | 0.8 | 0 | 0 | | AvgInferenceDistChain | 0.23 | 0.1 | 0.42 | 0.26 | 0.53 | 0.76 | 0 | 0 | | AvgConnBl_order | 0.22 | 0 | 0.9 | 0.21 | 2.32 | 0.29 | 0 | 0 | | AvgDepsBl_conj | 0.22 | 0 | 0.79 | 0.16 | 1.65 | 0.57 | 0 | 0 | | AvgPrepositionBl | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 | | AvgSenScore | 0.22 | 0.05 | 0.14 | 0.33 | 0.33 | 0.9 | 0 | 0 | | AvgDepsBl_case | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 | | AvgAOESen_IndexPolyFAT.3 | 0.22 | 0.07 | 0.26 | 0.42 | 0.35 | 0.55 | 0 | 0 | | AvgAOABl_Cortese | 0.22 | 0.05 | 0.35 | 0.32 | 1.05 | 0.67 | 0 | 0 | | AvgConnSen_oppositions | 0.21 | 0.03 | 0.03 | 0.35 | 1.28 | 0.84 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.21 | 0 | 0 | 0.88 | 0 | 0 | 0 | 0 | | AvgRhythmUnits | 0.21 | 0 | 0.34 | 0.42 | 1.08 | 0.53 | 0 | 0 | | AvgNounNmdEntBl | 0.2 | 0.14 | 0.52 | 0.07 | 2.7 | 0.46 | 0 | 0 | | Sentences | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgAOABl_Kuperman | 0.2 | 0.13 | 0.06 | 0.32 | 0.93 | 0.64 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgNmdEntBl | 0.2 | 0.03 | 0.82 | 0.18 | 1.51 | 0.35 | 0 | 0 | | AvgConnSen_order | 0.19 | 0.11 | 0.13 | 0.37 | 0.55 | 0.41 | 0 | 0 | | AvgConnSen_temp_conns | 0.19 | 0.27 | 0.12 | 0 | 1.47 | 0.77 | 0 | 0 | | LxcSoph | 0.19 | 0.05 | 0.18 | 0.34 | 0.45 | 0.61 | 0 | 0 | | LangRhythmDiameter | 0.19 | 0 | 0.3 | 0.02 | 1.45 | 0.37 | 0 | 0.49 | | AvgIntraBlCoh_LSA | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_Path | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgAOASen_Kuperman | 0.18 | 0.28 | 0.06 | 0.17 | 0.06 | 0.45 | 0 | 0 | | AvgDepsBl_nsubjpass | 0.18 | 0.04 | 0.66 | 0.11 | 0.38 | 0.53 | 0 | 0 | | AvgAOASen_Bristol | 0.18 | 0.13 | 0.18 | 0.25 | 0.36 | 0.55 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.17 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.17 | 0 | 0 | 0.72 | 0 | 0 | 0 | 0 | | SenAsson | 0.16 | 0 | 0.47 | 0.06 | 0.6 | 0.7 | 0 | 0 | | AvgDepsBl_nummod | 0.16 | 0 | 0.47 | 0.06 | 0.07 | 0.75 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexAbThr.0.3. | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.15 | 0 | 0 | 0.62 | 0 | 0 | 0 | 0 | | SynSoph | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgUnqAdjectiveBl | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgConnBl_sentence_link | 0.14 | 0 | 0 | 0.57 | 0 | 0 | 0 | 0 | | WdAvgDpthHypernymTree | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.14 | 0 | 0 | 0.6 | 0 | 0 | 0 | 0 | | AvgDepsSen_mwe | 0.13 | 0 | 0.31 | 0.03 | 0.32 | 0.7 | 0 | 0 | | AvgDepsBl_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgDepsSen_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0.13 | 0 | 0.14 | 0.22 | 0.06 | 0.55 | 0 | 0 | | AvgDepsSen_nsubj | 0.12 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.12 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | AvgBlVoiceCoOcc | 0.12 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.11 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.1 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgDepsSen_acl | 0.1 | 0.06 | 0.2 | 0.01 | 0.6 | 0.43 | 0 | 0 | | RdbltyKincaid | 0.1 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgAOASen_Cortese | 0.1 | 0.12 | 0.06 | 0.25 | 0.22 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | AvgPronounSen | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgDepsBl_dep | 0.08 | 0.04 | 0.25 | 0.02 | 0.15 | 0.29 | 0 | 0 | | AvgDepsBl_cc | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgDepsSen_neg | 0.08 | 0 | 0.13 | 0 | 0.14 | 0.58 | 0 | 0 | | AvgDepsSen_conj | 0.08 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgDepsBl_mwe | 0.08 | 0 | 0.43 | 0.05 | 0.03 | 0.18 | 0 | 0 | | AvgConnSen_coord_connects | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.07 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.06 | 0.03 | 0.04 | 0 | 0.33 | 0.37 | 0 | 0 | | AvgAOESen_InvLinRegSlo | 0.05 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.05 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.04 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.03 | 0 | 0 | 0.12 | 0 | 0 | 0 | 0 |
ReaderBench Model 1c
This model was trained on spring data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:--------| | -5.6692 | 0.1651 | 0.2625 | 0.1043 | -0.0146 | 0.4555 | 0.1632 | -0.0348 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:--------------------------|:-----|:------|:-----|:-----|:-----|:-----|:------|:------| | WdEnt | 7.76 | 15.63 | 2.27 | 1.53 | 2 | 3.41 | 25.87 | 9.83 | | AvgUnqVerbBl | 5.89 | 24.09 | 2.26 | 1.36 | 2.3 | 3.86 | 0 | 12.86 | | AvgBlScore | 2.6 | 7.12 | 1.53 | 1.13 | 0.58 | 2.67 | 0 | 4.69 | | AvgNounSen | 2.57 | 0.61 | 0.6 | 0.39 | 1.06 | 1.04 | 14.54 | 2.12 | | LxcDiv | 2.18 | 3.58 | 2.18 | 1.33 | 0.72 | 2.54 | 0 | 3.78 | | AvgDepsSen_compound | 2.17 | 1.45 | 1.12 | 0.56 | 0.27 | 1.54 | 7.94 | 2.12 | | WdDiffLemmaStem | 2.07 | 0.57 | 0.93 | 0.49 | 0.22 | 1 | 10.51 | 0 | | AvgDepsBl_dobj | 2.06 | 0.09 | 1.63 | 0.74 | 0.92 | 1.13 | 9.09 | 0 | | AvgUnqPrepositionBl | 1.96 | 1.98 | 2.25 | 1.27 | 1.28 | 2.47 | 0 | 5.6 | | AvgDepsBl_punct | 1.85 | 2.48 | 1.93 | 0.93 | 1.96 | 2.21 | 0 | 6.2 | | AvgDepsBl_nsubj | 1.84 | 3.1 | 1.97 | 1.26 | 0.88 | 2.06 | 0 | 1.36 | | RdbltyFlesch | 1.75 | 0.27 | 0.22 | 0.16 | 1.12 | 0.23 | 12.02 | 0.15 | | AvgWdLen | 1.74 | 3.11 | 1.54 | 0.76 | 0.21 | 2.1 | 0 | 3.03 | | AvgDepsBl_nmod | 1.65 | 2.75 | 1.94 | 0.92 | 0.42 | 1.77 | 0 | 2.42 | | AvgUnqNoundBl | 1.28 | 0.06 | 0.99 | 0.68 | 1.79 | 0.1 | 6.94 | 2.72 | | AvgPronounBl | 1.28 | 0.4 | 1.85 | 1.14 | 0.98 | 1.75 | 0 | 1.21 | | AvgUnqPronounBl | 1.23 | 0.36 | 1.69 | 0.74 | 0.27 | 0.86 | 3 | 0.76 | | WdSylCnt | 1.22 | 0.89 | 1.39 | 0.58 | 1.52 | 1.62 | 0 | 5.45 | | AvgUnqAdjectiveBl | 1.19 | 0.24 | 1.37 | 0.54 | 0.34 | 0.6 | 4.3 | 0.76 | | AvgDepsBl_ccomp | 1.18 | 0.35 | 0.61 | 0.05 | 0.35 | 0.61 | 5.78 | 0.76 | | AvgChainSpan | 1.17 | 0.51 | 1.66 | 0.99 | 0.49 | 1.6 | 0 | 0.91 | | LexChainMaxSp | 1.14 | 0.38 | 1.67 | 0.75 | 1.99 | 1.62 | 0 | 0 | | WdDiffWdStem | 1.11 | 0.76 | 1.38 | 0.78 | 0.06 | 1.6 | 0 | 0 | | AvgDepsBl_det | 1.1 | 0.29 | 1.71 | 0.72 | 0.82 | 1.52 | 0 | 0.76 | | WdLettStdDev | 1.02 | 1.03 | 1.66 | 0.85 | 1.56 | 1.04 | 0 | 0.3 | | FrqRhythmId | 1 | 0.65 | 1.44 | 0.59 | 0.95 | 1.32 | 0 | 0.76 | | AvgAOABl_Shock | 0.97 | 0.91 | 1.13 | 0.67 | 0.02 | 1.34 | 0 | 0 | | AvgSenBlCoh_LDA | 0.97 | 1.01 | 1.13 | 0.82 | 1.53 | 1.24 | 0 | 0 | | AvgDepsBl_mark | 0.96 | 0.04 | 1.52 | 0.65 | 1.46 | 1.4 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.9 | 0.39 | 1.37 | 0.67 | 0.29 | 1.18 | 0 | 1.06 | | TCorefChainDoc | 0.87 | 0.32 | 1.67 | 0.71 | 2.43 | 0.96 | 0 | 0 | | AvgDepsSen_punct | 0.81 | 0.4 | 1 | 0.68 | 0.26 | 1.2 | 0 | 0 | | LangRhythmCoeff | 0.8 | 0.47 | 0.95 | 0.46 | 0.2 | 1.23 | 0 | 0 | | AvgPronBl_first_person | 0.78 | 0.58 | 1.31 | 0.44 | 0.15 | 0.86 | 0 | 1.66 | | RdbltyDaleChall | 0.78 | 1.44 | 0.96 | 0.38 | 1.08 | 0.76 | 0 | 1.06 | | AvgConnBl_sentence_link | 0.74 | 0.09 | 1.31 | 0.45 | 3.5 | 0.92 | 0 | 0.3 | | AvgDepsBl_compound | 0.74 | 1.08 | 0.77 | 0.12 | 1.07 | 0.86 | 0 | 3.48 | | AvgConnSen_logical_conns | 0.71 | 0.31 | 0.54 | 0.51 | 0.72 | 1.24 | 0 | 0.76 | | LexChainAvgSpan | 0.7 | 0.67 | 1.02 | 0.62 | 0.5 | 0.81 | 0 | 0 | | AvgUnqAdverbBl | 0.69 | 0.01 | 1.46 | 0.57 | 0.78 | 0.75 | 0 | 0.76 | | CharEnt | 0.66 | 0.48 | 1.35 | 0.78 | 1.51 | 0.49 | 0 | 0.76 | | AvgDepsBl_amod | 0.66 | 0.23 | 1.31 | 0.48 | 1.2 | 0.73 | 0 | 0 | | AvgRhythmUnitStreesSyll | 0.65 | 0.39 | 0.43 | 0.21 | 0.86 | 1.22 | 0 | 0 | | AvgAdjectiveSen | 0.64 | 0.7 | 0.68 | 0.45 | 0.03 | 0.78 | 0 | 2.72 | | AvgDepsBl_advcl | 0.62 | 0.07 | 1.21 | 0.42 | 1.83 | 0.74 | 0 | 0 | | AvgConnBl_simp_subords | 0.61 | 0.04 | 1.25 | 0.45 | 0.28 | 0.69 | 0 | 0.76 | | AvgConnBl_reas_purp | 0.61 | 0.57 | 0.98 | 0.27 | 0.17 | 0.74 | 0 | 0 | | AvgCorefChain | 0.6 | 0.54 | 1.17 | 0.48 | 1.94 | 0.52 | 0 | 0 | | AvgPronBl_indefinite | 0.59 | 0.2 | 1.33 | 0.5 | 0.05 | 0.57 | 0 | 0 | | AvgDepsBl_xcomp | 0.58 | 0.36 | 1.14 | 0.37 | 0.09 | 0.63 | 0 | 0 | | AvgBlVoiceCoOcc | 0.57 | 0 | 1.44 | 0.56 | 0.17 | 0.51 | 0 | 0 | | AvgDepsBl_aux | 0.56 | 0.17 | 1.12 | 0.32 | 1.5 | 0.64 | 0 | 0 | | AvgDepsBl_mwe | 0.55 | 0 | 0.93 | 0.23 | 1.62 | 0.79 | 0 | 0 | | AvgPronBl_third_person | 0.55 | 0.13 | 1.23 | 0.39 | 0.24 | 0.53 | 0 | 1.06 | | AggPronSen_third_person | 0.55 | 0.37 | 0.47 | 0.21 | 2.96 | 0.87 | 0 | 0.76 | | AvgAOABl_Cortese | 0.55 | 0.56 | 0.64 | 0.7 | 0.31 | 0.68 | 0 | 0 | | AvgDepsBl_neg | 0.54 | 0.08 | 0.71 | 0.15 | 0.29 | 0.91 | 0 | 0 | | AvgConnSen_order | 0.54 | 0.06 | 0.26 | 0.52 | 0.98 | 1.07 | 0 | 0 | | AvgDepsSen_amod | 0.53 | 0.43 | 0.6 | 0.49 | 0.07 | 0.72 | 0 | 0.61 | | AvgInferenceDistChain | 0.53 | 0.52 | 0.46 | 0.31 | 1.67 | 0.81 | 0 | 0 | | TCorefChainBigSpan | 0.52 | 0.04 | 1.24 | 0.35 | 0.13 | 0.53 | 0 | 0 | | SenAsson | 0.51 | 0.33 | 0.85 | 0.21 | 1.05 | 0.63 | 0 | 0.15 | | AvgConnBl_contrasts | 0.5 | 0.06 | 0.85 | 0.21 | 0 | 0.72 | 0 | 0 | | AggPronSen_indefinite | 0.49 | 0.31 | 0.15 | 0.39 | 0.83 | 0.93 | 0 | 0.76 | | AvgDepsSen_xcomp | 0.48 | 0.76 | 0.22 | 0.37 | 0.33 | 0.72 | 0 | 0 | | AvgConnBl_temp_conns | 0.47 | 0.2 | 1.04 | 0.26 | 0.13 | 0.46 | 0 | 0.61 | | AvgDepsBl_cop | 0.46 | 0.25 | 0.92 | 0.23 | 0.13 | 0.5 | 0 | 0 | | AvgAOASen_Kuperman | 0.46 | 0.19 | 0.45 | 0.37 | 1.7 | 0.71 | 0 | 0.61 | | AvgDepsSen_nmod | 0.46 | 0.2 | 0.11 | 0.46 | 1.37 | 0.74 | 0 | 4.39 | | AvgDepsSen_dobj | 0.45 | 0.17 | 0.44 | 0.42 | 0.55 | 0.74 | 0 | 0 | | AvgDepsBl_nummod | 0.44 | 0.07 | 0.35 | 0.02 | 0.46 | 0.88 | 0 | 0 | | AvgAOEBl_IndexPolyFAT.3 | 0.44 | 0.19 | 0.42 | 0.4 | 1.29 | 0.71 | 0 | 0 | | AvgConnBl_order | 0.42 | 0.05 | 0.62 | 0.11 | 0.53 | 0.67 | 0 | 0 | | LxcSoph | 0.42 | 0.28 | 0.06 | 0.07 | 0.93 | 0.88 | 0 | 0.76 | | AvgDepsSen_neg | 0.42 | 0.06 | 0.25 | 0.02 | 0.84 | 0.91 | 0 | 0 | | AvgConnSen_simp_subords | 0.41 | 0.36 | 0.07 | 0.56 | 0.46 | 0.74 | 0 | 0 | | SenStdDevWd | 0.4 | 0.08 | 0.88 | 0.66 | 0.4 | 0.32 | 0 | 0 | | AvgConnBl_oppositions | 0.4 | 0.02 | 0.95 | 0.24 | 1.8 | 0.37 | 0 | 0 | | AvgDepsSen_ccomp | 0.4 | 0.1 | 0.57 | 0.44 | 0.58 | 0.47 | 0 | 2.12 | | AvgAOABl_Kuperman | 0.39 | 0.38 | 0.37 | 0.53 | 0.22 | 0.51 | 0 | 0 | | AvgRhythmUnits | 0.39 | 0.28 | 0.16 | 0.48 | 0.22 | 0.68 | 0 | 0 | | AvgAOEBl_InvLinRegSlo | 0.37 | 0.13 | 0.67 | 0.33 | 0.23 | 0.41 | 0 | 0.76 | | TActCorefChainWd | 0.37 | 0.29 | 0.51 | 0.26 | 0.8 | 0.49 | 0 | 0 | | AvgPronounSen | 0.36 | 0.32 | 0.6 | 0.37 | 0.49 | 0.33 | 0 | 0.76 | | AvgDepsSen_mwe | 0.36 | 0.01 | 0.33 | 0.03 | 0.68 | 0.73 | 0 | 0 | | AvgAOASen_Bristol | 0.35 | 0.6 | 0.24 | 0.13 | 0.88 | 0.42 | 0 | 1.36 | | LangRhythmDiameter | 0.34 | 0.13 | 0.23 | 0.01 | 0.31 | 0.68 | 0 | 0 | | AvgCommaBl | 0.34 | 0.07 | 0.66 | 0.12 | 0.49 | 0.43 | 0 | 0 | | AvgAOASen_Cortese | 0.34 | 0.33 | 0.69 | 0.45 | 0.27 | 0.24 | 0 | 0 | | AvgDepsSen_mark | 0.34 | 0.11 | 0.22 | 0.54 | 0.34 | 0.58 | 0 | 0 | | AvgDepsSen_acl | 0.33 | 0.04 | 0.71 | 0.13 | 0.93 | 0.38 | 0 | 0 | | AvgConnBl_logical_conns | 0.33 | 0.05 | 0.53 | 0.06 | 0.26 | 0.51 | 0 | 0 | | WdPathCntHypernymTree | 0.32 | 0.52 | 0.45 | 0.13 | 0.56 | 0.32 | 0 | 0 | | AvgConnBl_semi_coords | 0.32 | 0.14 | 0.78 | 0.16 | 0.43 | 0.27 | 0 | 0 | | AvgDepsSen_cop | 0.31 | 0.2 | 0.47 | 0.47 | 0.17 | 0.34 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.3 | 0.34 | 0.31 | 0.16 | 1.12 | 0.4 | 0 | 0 | | AvgConnSen_semi_coords | 0.3 | 0.06 | 0.01 | 0.34 | 0.68 | 0.64 | 0 | 0 | | AvgAOASen_Shock | 0.3 | 0.28 | 0.21 | 0.49 | 0.06 | 0.4 | 0 | 0.61 | | AvgConnSen_oppositions | 0.3 | 0.08 | 0.02 | 0 | 0.39 | 0.73 | 0 | 0 | | AvgDepsSen_advmod | 0.29 | 0.32 | 0.18 | 0.47 | 0.28 | 0.42 | 0 | 0 | | AvgConnBl_addition | 0.28 | 0.18 | 0.64 | 0.1 | 2.79 | 0.2 | 0 | 0 | | WdPolysemyCnt | 0.28 | 0.33 | 0.11 | 0.6 | 0.09 | 0.39 | 0 | 0 | | AvgAOABl_Bristol | 0.28 | 0.19 | 0.22 | 0.38 | 0.49 | 0.43 | 0 | 0 | | AvgNmdEntSen | 0.27 | 0.26 | 0.54 | 0.39 | 0.36 | 0.19 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.27 | 0.2 | 0.35 | 0.23 | 2.11 | 0.3 | 0 | 0.76 | | AvgConnSen_temp_conns | 0.27 | 0.35 | 0.12 | 0 | 0.93 | 0.48 | 0 | 0 | | WdAvgDpthHypernymTree | 0.27 | 0.43 | 0.4 | 0.2 | 0.15 | 0.25 | 0 | 0 | | AvgDepsSen_dep | 0.27 | 0.06 | 0.38 | 0.28 | 2.35 | 0.34 | 0 | 0.3 | | AvgAOESen_InvLinRegSlo | 0.26 | 0.27 | 0.52 | 0.25 | 0.02 | 0.19 | 0 | 0.3 | | AvgAOASen_Bird | 0.26 | 0.53 | 0.04 | 0.16 | 0.75 | 0.39 | 0 | 0.15 | | AvgNmdEntBl | 0.26 | 0.17 | 0.48 | 0.04 | 1.53 | 0.3 | 0 | 0 | | AvgConnSen_reas_purp | 0.25 | 0.12 | 0.08 | 0.48 | 1.14 | 0.4 | 0 | 0 | | AvgDepsSen_det | 0.25 | 0.11 | 0.1 | 0.25 | 0.26 | 0.44 | 0 | 0.76 | | AvgAOABl_Bird | 0.23 | 0.7 | 0.21 | 0.33 | 0.44 | 0.13 | 0 | 0 | | AvgAOESen_IndexPolyFAT.3 | 0.23 | 0.11 | 0.28 | 0.31 | 0.64 | 0.31 | 0 | 0 | | AvgDepsBl_dep | 0.22 | 0.1 | 0.44 | 0.05 | 1.31 | 0.22 | 0 | 0.61 | | AvgDepsBl_nsubjpass | 0.21 | 0.02 | 0.84 | 0.2 | 0.07 | 0 | 0 | 0 | | AvgAOESen_IndexAbThr.0.3. | 0.19 | 0.27 | 0.12 | 0.39 | 0.05 | 0.22 | 0 | 0 | | AvgDepsSen_aux | 0.18 | 0.39 | 0 | 0.49 | 2.05 | 0.17 | 0 | 0 | | AvgUnqWdBl | 0.15 | 0 | 0 | 1.66 | 0 | 0 | 0 | 0 | | AvgBlLen | 0.15 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 | | AvgDepsBl_acl | 0.14 | 0.01 | 0.42 | 0.04 | 0.2 | 0.1 | 0 | 0 | | AvgVerbBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | Content.words | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | AvgNounNmdEntBl | 0.14 | 0.29 | 0.05 | 0 | 1.08 | 0.21 | 0 | 0 | | Words | 0.13 | 0 | 0 | 1.47 | 0 | 0 | 0 | 0 | | AvgPrepositionBl | 0.11 | 0 | 0 | 1.22 | 0 | 0 | 0 | 0 | | AvgDepsSen_advcl | 0.1 | 0.14 | 0.05 | 0.52 | 0.09 | 0.05 | 0 | 0 | | AvgDepsBl_case | 0.09 | 0 | 0 | 0.96 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.09 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.09 | 0.01 | 0.19 | 0 | 0.98 | 0.1 | 0 | 0 | | AvgIntraBlCoh_Path | 0.08 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.08 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LSA | 0.08 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.07 | 0 | 0 | 0.74 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.07 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_Path | 0.07 | 0 | 0 | 0.76 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.07 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.07 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgNounBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | Sentences | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | AvgSenBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.07 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.06 | 0 | 0 | 0.64 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.06 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgVoice | 0.05 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgDepsSen_nsubj | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_connects | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgAdjectiveBl | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | 0.04 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.04 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgAdverbSen | 0.04 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgAOEBl_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgConnSen_addition | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgDepsBl_advmod | 0.04 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.03 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.03 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgSenScore | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0.03 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgDepsSen_conj | 0.03 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | SynSoph | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAdverbBl | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_connects | 0.02 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | RdbltyKincaid | 0.02 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.02 | 0 | 0 | 0.28 | 0 | 0 | 0 | 0 | | AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 | | AvgDepsBl_cc | 0.01 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.01 | 0 | 0 | 0.16 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 |
ReaderBench Model 1d
This model was trained on principal component scores for fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------| | -8.4195 | 0.0406 | 0.8127 | 0.0694 | -0.0509 | 0.1058 | 0.0038 | 0.0448 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:-----|:------|:------|:----|:------|:-----|:------|:------| | PC2 | 50.2 | 61.92 | 52.85 | 0 | 12.57 | 36.4 | 46.85 | 23.81 | | PC1 | 8.03 | 1.95 | 9.25 | 0 | 1.38 | 5.78 | 0.27 | 8.39 | | PC3 | 7.38 | 1.56 | 8.43 | 0 | 3.57 | 5.47 | 0 | 8.39 | | PC5 | 5.42 | 3.46 | 5.74 | 0 | 5.67 | 3.71 | 0 | 13.83 | | PC4 | 2 | 1.81 | 2.27 | 0 | 1.61 | 1.06 | 0 | 0 | | PC24 | 1.9 | 0.9 | 1.48 | 0 | 6.31 | 1.73 | 16.76 | 1.59 | | PC14 | 1.7 | 1.4 | 1.56 | 0 | 3.22 | 1.85 | 3.63 | 3.85 | | PC8 | 1.47 | 0.85 | 1.45 | 0 | 1.88 | 2.26 | 0 | 1.59 | | PC30 | 1.47 | 1.67 | 1.1 | 0 | 6.24 | 2.51 | 0 | 6.58 | | PC6 | 1.09 | 0.73 | 0.95 | 0 | 0.78 | 2.84 | 0 | 0 | | PC31 | 1.09 | 1.01 | 0.92 | 0 | 5.49 | 0.03 | 0 | 9.98 | | PC7 | 1.03 | 0.17 | 1.17 | 0 | 1.27 | 0.95 | 0 | 0 | | PC17 | 1.02 | 1.53 | 0.62 | 0 | 1.24 | 3.12 | 0 | 4.31 | | PC43 | 0.94 | 1.87 | 0.58 | 0 | 5.88 | 1.5 | 0 | 4.76 | | PC33 | 0.87 | 0.6 | 0.88 | 0 | 5.75 | 0.44 | 0 | 0 | | PC32 | 0.82 | 0.21 | 0.6 | 0 | 3.29 | 2.36 | 0 | 1.59 | | PC39 | 0.81 | 1.14 | 0.54 | 0 | 4.22 | 1.87 | 1.81 | 0 | | PC27 | 0.77 | 0.48 | 0.66 | 0 | 2.74 | 0.62 | 0 | 4.99 | | PC13 | 0.73 | 1.28 | 0.73 | 0 | 1.02 | 0.55 | 0 | 0 | | PC34 | 0.7 | 0.46 | 0.2 | 0 | 0.43 | 2.05 | 13.64 | 0 | | PC38 | 0.67 | 0.36 | 0.55 | 0 | 4.08 | 0.86 | 0 | 2.72 | | PC23 | 0.64 | 0.5 | 0.72 | 0 | 2.45 | 0.1 | 0 | 0 | | PC19 | 0.6 | 0.9 | 0.56 | 0 | 1.28 | 0.84 | 0 | 0 | | PC16 | 0.6 | 0.67 | 0.22 | 0 | 0 | 2.25 | 6.99 | 0 | | PC45 | 0.6 | 1.92 | 0.07 | 0 | 0 | 1.77 | 10.06 | 0.91 | | PC20 | 0.56 | 0 | 0.53 | 0 | 1.3 | 1.26 | 0 | 0 | | PC28 | 0.54 | 1.55 | 0.39 | 0 | 1.29 | 0.94 | 0 | 0.45 | | PC40 | 0.54 | 0.77 | 0.52 | 0 | 4.01 | 0.04 | 0 | 0.45 | | PC18 | 0.53 | 0.36 | 0.47 | 0 | 0.92 | 1.27 | 0 | 0 | | PC11 | 0.51 | 0.87 | 0.55 | 0 | 0.48 | 0.18 | 0 | 0 | | PC22 | 0.49 | 0.35 | 0.42 | 0 | 0.99 | 1.2 | 0 | 0 | | PC36 | 0.46 | 0.95 | 0.09 | 0 | 0 | 3.14 | 0 | 0 | | PC12 | 0.45 | 0.22 | 0.46 | 0 | 0.33 | 0.68 | 0 | 0 | | PC29 | 0.44 | 0.95 | 0.29 | 0 | 0.79 | 1.36 | 0 | 0 | | PC41 | 0.43 | 0.27 | 0.35 | 0 | 2.57 | 0.79 | 0 | 0 | | PC42 | 0.4 | 0.32 | 0.31 | 0 | 2.25 | 1.03 | 0 | 0 | | PC9 | 0.37 | 0.34 | 0.4 | 0 | 0.13 | 0.33 | 0 | 0 | | PC44 | 0.29 | 0.31 | 0.22 | 0 | 1.53 | 0.68 | 0 | 0 | | PC46 | 0.28 | 0.91 | 0.05 | 0 | 0 | 1.65 | 0 | 0.45 | | PC15 | 0.26 | 0.87 | 0.14 | 0 | 0 | 0.59 | 0 | 1.36 | | PC26 | 0.25 | 0.16 | 0.29 | 0 | 0.59 | 0 | 0 | 0 | | PC35 | 0.23 | 0.02 | 0.19 | 0 | 0.43 | 0.7 | 0 | 0 | | PC37 | 0.13 | 0.09 | 0.13 | 0 | 0.03 | 0.22 | 0 | 0 | | PC10 | 0.12 | 0.4 | 0 | 0 | 0 | 0.99 | 0 | 0 | | PC25 | 0.08 | 0.29 | 0.08 | 0 | 0 | 0.03 | 0 | 0 | | PC21 | 0.06 | 0.64 | 0.03 | 0 | 0 | 0.01 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components are displayed.
| Variable | RC1 | RC2 | RC3 | RC5 | RC4 | |:----------------------|:------|:------|:------|:------|:-----| | SS loadings | 44.56 | 31.07 | 19.29 | 10.07 | 9.38 | | Proportion Var | 0.22 | 0.15 | 0.10 | 0.05 | 0.05 | | Cumulative Var | 0.22 | 0.38 | 0.47 | 0.52 | 0.57 | | Proportion Explained | 0.39 | 0.27 | 0.17 | 0.09 | 0.08 | | Cumulative Proportion | 0.39 | 0.66 | 0.83 | 0.92 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC5 | RC4 | |:---------------------------------------|:------:|:------:|:------:|:------:|:------:| | Sentences | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 | | Words | 0.086 | 0.947 | 0.097 | 0.203 | 0.039 | | Content.words | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 | | RdbltyFlesch | -0.891 | -0.159 | -0.034 | -0.06 | 0.033 | | RdbltyFog | 0.95 | 0.117 | 0.07 | 0.127 | -0.022 | | RdbltyKincaid | 0.948 | 0.116 | 0.044 | 0.128 | -0.02 | | RdbltyDaleChall | 0.4 | -0.276 | -0.098 | 0.136 | -0.503 | | AvgBlLen | -0.041 | 0.903 | 0.095 | 0.055 | 0.156 | | AvgCommaBl | -0.1 | 0.309 | 0.05 | -0.227 | 0.006 | | AvgSenLen | 0.91 | 0.198 | 0.058 | 0.044 | 0.17 | | AvgSenBl | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 | | AvgUnqWdBl | -0.015 | 0.901 | 0.103 | 0.081 | 0.118 | | AvgUnqWdSen | 0.922 | 0.161 | 0.089 | 0.096 | 0.134 | | AvgWdLen | -0.089 | 0.407 | 0.562 | -0.271 | 0.362 | | AvgWdBl | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 | | AvgWdSen | 0.931 | 0.147 | 0.057 | 0.086 | 0.135 | | CharEnt | -0.006 | 0.489 | 0.287 | -0.198 | 0.059 | | SenStDevUnqWd | -0.429 | 0.328 | 0.099 | 0.075 | 0.453 | | SenStdDevWd | -0.354 | 0.326 | 0.076 | 0.112 | 0.472 | | WdEnt | 0.01 | 0.886 | 0.21 | 0.066 | 0.085 | | WdLettStdDev | -0.158 | 0.401 | 0.362 | -0.257 | 0.224 | | LxcDiv | 0.065 | 0.791 | 0.172 | 0.047 | 0.204 | | LxcSoph | 0.334 | 0.207 | 0.6 | -0.068 | 0.407 | | SynSoph | 0.825 | 0.348 | 0.092 | 0.118 | 0.259 | | AvgNounBl | -0.017 | 0.705 | 0.149 | 0.361 | 0.031 | | AvgPronounBl | 0.159 | 0.77 | 0.042 | 0.026 | -0.004 | | AvgVerbBl | 0.102 | 0.911 | 0.04 | 0.051 | 0.058 | | AvgAdverbBl | 0.043 | 0.697 | 0.012 | -0.126 | -0.103 | | AvgAdjectiveBl | -0.011 | 0.517 | 0.115 | 0.028 | 0.031 | | AvgPrepositionBl | 0.078 | 0.786 | 0.06 | 0.154 | -0.004 | | AvgNounSen | 0.834 | -0.018 | 0.109 | 0.357 | -0.019 | | AvgPronounSen | 0.928 | 0.03 | 0.053 | 0.017 | -0.023 | | AvgVerbSen | 0.957 | 0.1 | 0.031 | 0.013 | 0.042 | | AvgAdverbSen | 0.701 | 0.22 | -0.042 | -0.147 | -0.05 | | AvgAdjectiveSen | 0.699 | 0.007 | 0.137 | 0.004 | -0.055 | | AvgPrepositionSen | 0.803 | 0.212 | 0.018 | 0.139 | -0.013 | | AvgUnqNoundBl | -0.014 | 0.632 | 0.13 | 0.431 | -0.023 | | AvgUnqPronounBl | -0.007 | 0.533 | 0.088 | 0.074 | 0.058 | | AvgUnqVerbBl | 0.122 | 0.858 | 0.047 | 0.038 | 0.043 | | AvgUnqAdverbBl | -0.007 | 0.719 | 0.011 | -0.17 | -0.142 | | AvgUnqAdjectiveBl | -0.025 | 0.523 | 0.104 | -0.022 | 0.022 | | AvgUnqPrepositionBl | 0.054 | 0.813 | 0.061 | 0.075 | -0.009 | | AvgPronBl_first_person | 0.168 | 0.637 | 0.017 | 0.058 | -0.022 | | AvgPronBl_indefinite | 0.011 | 0.577 | 0.018 | 0.033 | 0.138 | | AggPronSen_indefinite | 0.677 | 0.159 | -0.005 | 0.004 | 0.112 | | AvgPronBl_third_person | 0.131 | 0.424 | 0.057 | -0.018 | 0.019 | | AggPronSen_third_person | 0.782 | -0.078 | 0.043 | -0.033 | 0.007 | | AvgSemDep | 0.967 | 0.07 | 0.088 | 0.181 | 0.001 | | WdDiffLemmaStem | -0.016 | 0.289 | 0.104 | 0.007 | -0.106 | | WdDiffWdStem | 0.041 | 0.483 | 0.031 | -0.313 | 0.132 | | WdMaxDpthHypernymTree | -0.08 | 0.083 | 0.356 | 0.059 | 0.349 | | WdAvgDpthHypernymTree | -0.064 | 0.073 | 0.369 | 0.063 | 0.349 | | WdPathCntHypernymTree | -0.066 | 0.123 | 0.217 | -0.019 | 0.464 | | WdPolysemyCnt | 0 | -0.045 | -0.037 | 0.192 | 0.244 | | WdSylCnt | -0.126 | 0.215 | 0.592 | -0.117 | 0.108 | | AvgAOADoc_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 | | AvgAOABl_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 | | AvgAOASen_Shock | 0.289 | 0.196 | 0.419 | 0.105 | 0.111 | | AvgAOADoc_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 | | AvgAOABl_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 | | AvgAOASen_Cortese | 0.261 | 0.168 | 0.575 | 0.045 | 0.211 | | AvgAOADoc_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 | | AvgAOABl_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 | | AvgAOASen_Kuperman | 0.1 | 0.18 | 0.742 | 0.068 | -0.023 | | AvgAOADoc_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 | | AvgAOABl_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 | | AvgAOASen_Bird | 0.262 | 0.14 | 0.554 | 0.029 | 0.173 | | AvgAOADoc_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 | | AvgAOABl_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 | | AvgAOASen_Bristol | 0.413 | 0.102 | 0.453 | 0.113 | 0.185 | | AvgAOEDoc_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 | | AvgAOEBl_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 | | AvgAOESen_IndexPolyFitAbThr.0.3. | 0.006 | 0.133 | 0.617 | 0.152 | -0.495 | | AvgAOEDoc_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 | | AvgAOEBl_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 | | AvgAOESen_InverseLinearRegressionSlope | 0.077 | 0.163 | 0.795 | 0.054 | -0.145 | | AvgAOEDoc_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 | | AvgAOEBl_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 | | AvgAOESen_InflectionPointPolynomial | 0.064 | 0.198 | 0.8 | -0.021 | -0.076 | | AvgAOEDoc_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 | | AvgAOEBl_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 | | AvgAOESen_InverseAverage | 0.06 | 0.191 | 0.808 | -0.021 | -0.097 | | AvgAOEDoc_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 | | AvgAOEBl_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 | | AvgAOESen_IndexAboveThreshold.0.3. | -0.071 | 0.107 | 0.491 | 0.174 | -0.556 | | AvgNmdEntBl | 0.057 | 0.52 | -0.052 | 0.139 | 0.032 | | AvgNounNmdEntBl | 0.088 | 0.363 | -0.01 | 0.198 | -0.015 | | AvgUnqNmdEntBl | 0.119 | 0.537 | -0.058 | 0.14 | -0.003 | | AvgNmdEntSen | 0.752 | 0.106 | -0.084 | 0.167 | 0.06 | | TCorefChainDoc | -0.03 | 0.621 | 0.189 | 0.003 | -0.062 | | AvgCorefChain | 0.143 | 0.47 | 0.111 | 0.016 | 0.079 | | AvgChainSpan | 0.088 | 0.729 | 0.038 | -0.003 | 0.128 | | AvgInferenceDistChain | 0.245 | 0.306 | 0.046 | 0.001 | -0.012 | | TActCorefChainWd | -0.092 | -0.329 | 0.268 | -0.225 | -0.102 | | TCorefChainBigSpan | 0.108 | 0.426 | 0.243 | -0.098 | 0.002 | | AvgConnBl_addition | 0.067 | 0.309 | 0.032 | 0.777 | 0.055 | | AvgConnSen_addition | 0.658 | -0.166 | 0.072 | 0.622 | -0.077 | | AvgConnBl_conjunctions | 0.168 | 0.362 | 0.061 | 0.775 | -0.017 | | AvgConnSen_conjunctions | 0.72 | -0.147 | 0.097 | 0.579 | -0.152 | | AvgConnBl_contrasts | 0.076 | 0.444 | 0.114 | 0.035 | -0.118 | | AvgConnSen_contrasts | 0.512 | 0.196 | 0.125 | -0.059 | -0.149 | | AvgConnBl_coordinating_conjuncts | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 | | AvgConnSen_coordinating_conjuncts | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 | | AvgConnBl_coordinating_connectives | 0.255 | 0.506 | 0.035 | 0.7 | 0.003 | | AvgConnSen_coordinating_connectives | 0.818 | -0.034 | 0.071 | 0.48 | -0.119 | | AvgConnBl_logical_connectors | 0.186 | 0.317 | 0.046 | 0.76 | -0.008 | | AvgConnSen_logical_connectors | 0.693 | -0.154 | 0.069 | 0.563 | -0.121 | | AvgConnBl_oppositions | 0.096 | 0.391 | 0.08 | 0.048 | -0.132 | | AvgConnSen_oppositions | 0.494 | 0.14 | 0.106 | -0.083 | -0.187 | | AvgConnBl_order | -0.115 | 0.255 | -0.013 | 0.188 | 0.176 | | AvgConnSen_order | 0.315 | 0.084 | -0.013 | 0.115 | 0.215 | | AvgConnBl_reason_and_purpose | 0.204 | 0.536 | -0.017 | 0.194 | 0.163 | | AvgConnSen_reason_and_purpose | 0.71 | 0.226 | 0.002 | 0.034 | 0.064 | | AvgConnBl_semi_coordinators | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 | | AvgConnSen_semi_coordinators | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 | | AvgConnBl_sentence_linking | 0.241 | 0.603 | 0.014 | 0.606 | 0.063 | | AvgConnSen_sentence_linking | 0.862 | 0.027 | 0.048 | 0.411 | -0.059 | | AvgConnBl_simple_subordinators | 0.081 | 0.52 | 0.062 | -0.082 | -0.029 | | AvgConnSen_simple_subordinators | 0.681 | 0.165 | -0.053 | -0.039 | 0.023 | | AvgConnBl_temporal_connectors | 0.121 | 0.382 | -0.081 | -0.07 | 0.114 | | AvgConnSen_temporal_connectors | 0.637 | 0.153 | -0.183 | -0.036 | 0.106 | | LexChainAvgSpan | 0.128 | 0.407 | 0.308 | 0.073 | 0.415 | | LexChainMaxSp | -0.007 | 0.663 | 0.02 | 0.171 | 0.207 | | AvgBlScore | 0.148 | 0.801 | 0.043 | 0.165 | 0.285 | | AvgSenScore | 0.909 | 0.12 | 0.044 | 0.055 | 0.165 | | SenScoreStDev | -0.466 | 0.332 | 0.071 | 0.09 | 0.501 | | AvgIntraBlCoh_LeackockChodorow | -0.741 | 0.302 | 0.152 | 0.008 | 0.471 | | AvgSenAdjCoh_LeackockChodorow | -0.735 | 0.263 | 0.166 | -0.002 | 0.488 | | AvgSenBlCoh_LeackockChodorow | 0.434 | -0.139 | 0.696 | 0.053 | 0.24 | | AvgIntraBlCoh_WuPalmer | -0.748 | 0.296 | 0.152 | 0.001 | 0.465 | | AvgSenAdjCoh_WuPalmer | -0.743 | 0.259 | 0.167 | -0.01 | 0.482 | | AvgSenBlCoh_WuPalmer | 0.443 | -0.145 | 0.694 | 0.049 | 0.226 | | AvgIntraBlCoh_Path | -0.749 | 0.277 | 0.154 | -0.007 | 0.46 | | AvgSenAdjCoh_Path | -0.744 | 0.234 | 0.175 | -0.021 | 0.478 | | AvgSenBlCoh_Path | 0.528 | -0.219 | 0.642 | 0.044 | 0.155 | | AvgIntraBlCoh_LSA | -0.739 | 0.338 | 0.148 | 0.005 | 0.446 | | AvgSenAdjCoh_LSA | -0.729 | 0.295 | 0.166 | -0.011 | 0.473 | | AvgSenBlCoh_LSA | 0.598 | -0.209 | 0.547 | 0.09 | 0.148 | | AvgIntraBlCoh_LDA | -0.738 | 0.345 | 0.151 | -0.004 | 0.449 | | AvgSenAdjCoh_LDA | -0.718 | 0.324 | 0.158 | -0.003 | 0.473 | | AvgSenBlCoh_LDA | 0.513 | -0.066 | 0.598 | 0.08 | 0.202 | | AvgIntraBlCoh_word2vec | -0.743 | 0.31 | 0.158 | -0.026 | 0.457 | | AvgSenAdjCoh_word2vec | -0.734 | 0.273 | 0.179 | -0.046 | 0.48 | | AvgSenBlCoh_word2vec | 0.581 | -0.259 | 0.577 | 0.048 | 0.165 | | AvgBlVoiceCoOcc | 0.083 | 0.49 | -0.117 | 0.351 | 0.258 | | AvgVoice | 0.086 | 0.506 | -0.117 | 0.355 | 0.247 | | AvgSenSyll | 0.973 | 0.081 | 0.076 | 0.138 | -0.008 | | AvgSenStressedSyll | 0.939 | 0.141 | 0.049 | 0.106 | 0.117 | | AvgRhythmUnits | 0.315 | 0.184 | -0.003 | -0.262 | 0.043 | | AvgRhythmUnitSyll | 0.807 | -0.027 | 0.082 | 0.284 | -0.045 | | AvgRhythmUnitStreesSyll | 0.809 | 0.024 | 0.053 | 0.235 | 0.073 | | LangRhythmCoeff | 0.24 | 0.048 | -0.011 | 0.129 | -0.121 | | LangRhythmId | -0.173 | -0.023 | -0.06 | 0.028 | -0.387 | | FrqRhythmId | 0.657 | -0.336 | -0.016 | 0.096 | 0.056 | | LangRhythmDiameter | 0.373 | 0.113 | 0.117 | 0.197 | -0.139 | | SenAsson | -0.051 | 0.221 | 0.079 | 0.066 | 0.065 | | AvgDepsBl_acl | 0.092 | 0.23 | 0.074 | 0.206 | 0.093 | | AvgDepsSen_acl | 0.485 | 0.081 | 0.031 | 0.343 | 0.108 | | AvgDepsBl_advcl | 0.394 | 0.519 | -0.037 | 0.042 | 0.175 | | AvgDepsSen_advcl | 0.768 | 0.219 | -0.041 | 0.009 | 0.065 | | AvgDepsBl_advmod | 0.141 | 0.71 | -0.023 | -0.136 | -0.045 | | AvgDepsSen_advmod | 0.753 | 0.193 | -0.054 | -0.132 | -0.006 | | AvgDepsBl_amod | -0.092 | 0.476 | 0.104 | 0.038 | 0.13 | | AvgDepsSen_amod | 0.563 | 0.072 | 0.117 | -0.001 | 0.112 | | AvgDepsBl_aux | -0.02 | 0.402 | 0.09 | -0.282 | 0.005 | | AvgDepsSen_aux | 0.621 | 0.072 | 0.112 | -0.18 | -0.071 | | AvgDepsBl_auxpass | -0.079 | 0.424 | 0.001 | -0.052 | -0.166 | | AvgDepsBl_case | -0.121 | 0.772 | 0.094 | 0.169 | -0.072 | | AvgDepsSen_case | 0.747 | 0.175 | 0.093 | 0.178 | -0.024 | | AvgDepsBl_cc | 0.203 | 0.37 | 0.063 | 0.746 | -0.039 | | AvgDepsSen_cc | 0.73 | -0.143 | 0.088 | 0.553 | -0.149 | | AvgDepsBl_ccomp | 0.383 | 0.496 | 0.036 | -0.022 | 0.173 | | AvgDepsSen_ccomp | 0.781 | 0.073 | 0.031 | -0.041 | 0.09 | | AvgDepsBl_compound | 0.104 | 0.149 | 0.038 | 0.366 | -0.006 | | AvgDepsSen_compound | 0.587 | -0.062 | 0.073 | 0.412 | -0.064 | | AvgDepsBl_conj | 0.31 | 0.297 | 0.122 | 0.722 | -0.045 | | AvgDepsSen_conj | 0.731 | -0.134 | 0.134 | 0.536 | -0.162 | | AvgDepsBl_cop | -0.001 | 0.419 | 0.055 | -0.12 | -0.002 | | AvgDepsSen_cop | 0.583 | 0.061 | 0.018 | -0.107 | 0.003 | | AvgDepsBl_dep | 0.234 | 0.123 | 0.104 | 0.47 | -0.059 | | AvgDepsSen_dep | 0.527 | -0.187 | 0.15 | 0.494 | -0.174 | | AvgDepsBl_det | -0.182 | 0.549 | 0.13 | 0.222 | 0.071 | | AvgDepsSen_det | 0.576 | -0.06 | 0.152 | 0.415 | -0.023 | | AvgDepsBl_dobj | 0.119 | 0.64 | 0.078 | 0.196 | 0.13 | | AvgDepsSen_dobj | 0.856 | -0.002 | 0.047 | 0.128 | 0.028 | | AvgDepsBl_mark | 0.367 | 0.535 | 0.01 | 0.087 | 0.139 | | AvgDepsSen_mark | 0.795 | 0.152 | -0.063 | 0.085 | 0.073 | | AvgDepsBl_mwe | -0.01 | 0.262 | 0.021 | 0.142 | 0.008 | | AvgDepsSen_mwe | 0.383 | 0.097 | 0.025 | 0.044 | 0.031 | | AvgDepsBl_neg | 0.197 | 0.361 | 0.053 | -0.181 | -0.132 | | AvgDepsSen_neg | 0.549 | 0.073 | -0.024 | -0.17 | -0.086 | | AvgDepsBl_nmod | -0.134 | 0.727 | 0.094 | 0.237 | -0.052 | | AvgDepsSen_nmod | 0.661 | 0.15 | 0.086 | 0.281 | -0.02 | | AvgDepsBl_nsubj | 0.141 | 0.88 | 0.062 | 0.041 | 0.08 | | AvgDepsSen_nsubj | 0.969 | 0.02 | 0.051 | 0.018 | 0.003 | | AvgDepsBl_nsubjpass | -0.015 | 0.417 | -0.005 | -0.065 | -0.158 | | AvgDepsBl_nummod | 0.132 | 0.492 | -0.071 | -0.009 | 0.13 | | AvgDepsBl_punct | -0.483 | 0.644 | -0.017 | -0.108 | -0.086 | | AvgDepsSen_punct | -0.192 | 0.389 | 0.063 | -0.232 | 0.191 | | AvgDepsBl_xcomp | 0.053 | 0.501 | 0.03 | 0.186 | 0.068 | | AvgDepsSen_xcomp | 0.666 | 0.075 | 0.061 | 0.196 | 0.024 |
ReaderBench Model 1e
This model was trained on principal component scores for winter data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------| | -8.0185 | 0.0573 | 0.5839 | 0.5269 | -0.3984 | 0.1184 | 0.1066 | 0.0459 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:-----|:-----|:------|:----|:-----|:------|:------|:------| | PC2 | 32.8 | 53 | 47.34 | 0 | 9.95 | 26.16 | 39.14 | 22.22 | | PC1 | 7.63 | 1.98 | 13.73 | 0 | 1.84 | 4.83 | 0 | 11.11 | | PC39 | 4.81 | 1.96 | 1.08 | 0 | 7.68 | 2.76 | 17.37 | 8.89 | | PC5 | 4.35 | 2.14 | 4.78 | 0 | 4.24 | 3.9 | 0 | 13.33 | | PC11 | 4.33 | 1.43 | 3.55 | 0 | 5.85 | 1.52 | 4.87 | 11.11 | | PC37 | 4.21 | 1.94 | 1.15 | 0 | 7.47 | 1.75 | 12.15 | 6.67 | | PC6 | 3.45 | 3.05 | 3.87 | 0 | 3.53 | 2.17 | 0 | 8.89 | | PC26 | 3.28 | 2.41 | 1.25 | 0 | 4.41 | 1.69 | 14.49 | 0 | | PC38 | 3.13 | 0.57 | 1.08 | 0 | 7.55 | 0.99 | 0 | 6.67 | | PC14 | 2.89 | 4.55 | 1.94 | 0 | 3.33 | 6.23 | 0 | 6.67 | | PC24 | 2.2 | 1.36 | 1.07 | 0 | 3.24 | 0.49 | 8.2 | 0 | | PC40 | 2.12 | 0.85 | 0.72 | 0 | 5.11 | 1.28 | 0 | 2.22 | | PC9 | 1.75 | 0.72 | 2.06 | 0 | 2.21 | 0.62 | 0 | 2.22 | | PC33 | 1.63 | 0.91 | 0.67 | 0 | 2.86 | 2.41 | 2.5 | 0 | | PC45 | 1.61 | 0.26 | 0.5 | 0 | 4.33 | 0.63 | 0 | 0 | | PC44 | 1.51 | 0.65 | 0.49 | 0 | 3.6 | 1.94 | 0 | 0 | | PC32 | 1.3 | 0.87 | 0.67 | 0 | 2.61 | 1.92 | 0 | 0 | | PC20 | 1.29 | 1.94 | 1 | 0 | 2.28 | 0.76 | 0 | 0 | | PC4 | 1.16 | 1.23 | 1.61 | 0 | 0.82 | 1.5 | 0 | 0 | | PC19 | 1.07 | 1.79 | 0.78 | 0 | 1.41 | 2.19 | 0 | 0 | | PC34 | 0.98 | 1.11 | 0.5 | 0 | 1.9 | 1.2 | 0.26 | 0 | | PC43 | 0.97 | 0.27 | 0.42 | 0 | 2.53 | 0 | 0 | 0 | | PC8 | 0.95 | 0.61 | 1.17 | 0 | 0.8 | 1.73 | 0 | 0 | | PC15 | 0.94 | 1.05 | 0.88 | 0 | 1.21 | 1.47 | 0 | 0 | | PC23 | 0.9 | 0.73 | 0.64 | 0 | 1.33 | 1.89 | 0 | 0 | | PC17 | 0.9 | 0.77 | 0.85 | 0 | 1.41 | 0.6 | 0 | 0 | | PC35 | 0.86 | 1.17 | 0.46 | 0 | 1.61 | 1.24 | 0 | 0 | | PC3 | 0.81 | 0.64 | 1.28 | 0 | 0.16 | 1.75 | 0 | 0 | | PC12 | 0.76 | 0.42 | 0.78 | 0 | 0.71 | 1.86 | 0 | 0 | | PC28 | 0.73 | 0.92 | 0.48 | 0 | 1.01 | 1.9 | 0 | 0 | | PC16 | 0.69 | 0.41 | 0.71 | 0 | 0.86 | 0.92 | 0 | 0 | | PC25 | 0.59 | 0.21 | 0.47 | 0 | 0.73 | 1.59 | 0 | 0 | | PC30 | 0.42 | 1.15 | 0.31 | 0 | 0.25 | 1.74 | 0 | 0 | | PC42 | 0.41 | 0.61 | 0.22 | 0 | 0.34 | 1.92 | 0 | 0 | | PC36 | 0.36 | 0.14 | 0.29 | 0 | 0.51 | 0.88 | 0 | 0 | | PC27 | 0.35 | 0.38 | 0.35 | 0 | 0.34 | 0.77 | 0 | 0 | | PC7 | 0.34 | 1.67 | 0.06 | 0 | 0 | 1.66 | 1.01 | 0 | | PC13 | 0.28 | 0.89 | 0.01 | 0 | 0 | 2.56 | 0 | 0 | | PC29 | 0.28 | 0.97 | 0.05 | 0 | 0 | 2.36 | 0 | 0 | | PC10 | 0.27 | 0.54 | 0.27 | 0 | 0 | 1.27 | 0 | 0 | | PC18 | 0.24 | 0.52 | 0.14 | 0 | 0 | 1.68 | 0 | 0 | | PC31 | 0.18 | 0.35 | 0.08 | 0 | 0 | 1.51 | 0 | 0 | | PC46 | 0.11 | 0.43 | 0.15 | 0 | 0 | 0.24 | 0 | 0 | | PC21 | 0.07 | 0.18 | 0 | 0 | 0 | 0.64 | 0 | 0 | | PC22 | 0.07 | 0.25 | 0.09 | 0 | 0 | 0.27 | 0 | 0 | | PC41 | 0.06 | 0 | 0 | 0 | 0 | 0.6 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components are displayed.
| Variable | RC1 | RC2 | RC3 | RC4 | RC5 | |:----------------------|:------|:------|:------|:-----|:-----| | SS loadings | 46.99 | 34.24 | 14.94 | 7.95 | 6.95 | | Proportion Var | 0.23 | 0.17 | 0.07 | 0.04 | 0.03 | | Cumulative Var | 0.23 | 0.40 | 0.48 | 0.52 | 0.55 | | Proportion Explained | 0.42 | 0.31 | 0.13 | 0.07 | 0.06 | | Cumulative Proportion | 0.42 | 0.73 | 0.87 | 0.94 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC4 | RC5 | |:--------------------------|:------|:------|:------|:------|:------| | Sentences | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 | | Words | 0.07 | 0.97 | 0 | -0.08 | -0.05 | | Content.words | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 | | RdbltyFlesch | -0.82 | -0.19 | -0.02 | -0.12 | 0.09 | | RdbltyFog | 0.91 | 0.17 | 0.01 | 0.04 | 0.01 | | RdbltyKincaid | 0.91 | 0.18 | 0.01 | 0.05 | 0 | | RdbltyDaleChall | 0.5 | -0.34 | 0.11 | -0.06 | -0.22 | | AvgBlLen | -0.11 | 0.91 | 0.08 | 0.13 | -0.16 | | AvgCommaBl | -0.23 | 0.38 | 0.06 | -0.1 | -0.1 | | AvgSenLen | 0.9 | 0.26 | 0.08 | 0.15 | 0.03 | | AvgSenBl | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 | | AvgUnqWdBl | -0.08 | 0.91 | 0.05 | 0.12 | -0.15 | | AvgUnqWdSen | 0.93 | 0.22 | 0.08 | 0.12 | 0.05 | | AvgWdLen | -0.21 | 0.26 | -0.17 | 0.25 | -0.46 | | AvgWdBl | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 | | AvgWdSen | 0.93 | 0.23 | 0.07 | 0.09 | 0.06 | | CharEnt | -0.14 | 0.54 | 0.11 | -0.02 | -0.03 | | SenStDevUnqWd | -0.23 | 0.5 | 0.04 | 0.09 | 0.41 | | SenStdDevWd | -0.16 | 0.48 | 0.04 | 0.07 | 0.45 | | WdEnt | 0.03 | 0.86 | 0.04 | 0.1 | -0.13 | | WdLettStdDev | -0.09 | 0.4 | 0.37 | 0.24 | 0.04 | | LxcDiv | 0 | 0.81 | 0.09 | 0.2 | -0.07 | | LxcSoph | 0.36 | 0.09 | -0.28 | 0.11 | -0.35 | | SynSoph | 0.82 | 0.44 | 0.12 | 0.14 | 0.12 | | AvgNounBl | 0.01 | 0.76 | 0.11 | 0.03 | -0.35 | | AvgPronounBl | 0.1 | 0.79 | -0.1 | -0.13 | 0.18 | | AvgVerbBl | 0.03 | 0.91 | -0.05 | -0.02 | 0.05 | | AvgAdverbBl | 0.09 | 0.63 | -0.01 | -0.22 | 0.11 | | AvgAdjectiveBl | -0.06 | 0.55 | 0.12 | -0.07 | -0.09 | | AvgPrepositionBl | 0.11 | 0.75 | -0.04 | 0.27 | -0.1 | | AvgNounSen | 0.9 | 0.06 | 0.06 | 0.06 | -0.14 | | AvgPronounSen | 0.91 | 0.1 | -0.08 | -0.02 | 0.15 | | AvgVerbSen | 0.94 | 0.16 | -0.01 | 0.04 | 0.09 | | AvgAdverbSen | 0.77 | 0.15 | -0.02 | -0.13 | 0.18 | | AvgAdjectiveSen | 0.71 | 0.11 | 0.13 | -0.13 | 0.1 | | AvgPrepositionSen | 0.85 | 0.2 | -0.07 | 0.26 | -0.04 | | AvgUnqNoundBl | 0.02 | 0.71 | 0.05 | 0.02 | -0.36 | | AvgUnqPronounBl | 0.09 | 0.66 | -0.01 | -0.09 | 0.02 | | AvgUnqVerbBl | 0.03 | 0.85 | -0.05 | 0.01 | 0.04 | | AvgUnqAdverbBl | -0.01 | 0.61 | -0.02 | -0.13 | 0.05 | | AvgUnqAdjectiveBl | -0.06 | 0.57 | 0.12 | -0.09 | -0.13 | | AvgUnqPrepositionBl | 0.09 | 0.71 | 0 | 0.3 | -0.14 | | AvgPronBl_first_person | 0.09 | 0.69 | -0.05 | -0.14 | 0.12 | | AvgPronBl_indefinite | -0.05 | 0.45 | 0.01 | 0.24 | -0.04 | | AggPronSen_indefinite | 0.62 | 0.17 | 0.05 | 0.23 | 0.02 | | AvgPronBl_third_person | 0.13 | 0.48 | -0.08 | -0.12 | 0.14 | | AggPronSen_third_person | 0.84 | -0.02 | -0.09 | -0.09 | 0.12 | | AvgSemDep | 0.97 | 0.12 | -0.01 | -0.06 | 0.06 | | WdDiffLemmaStem | -0.11 | 0.06 | -0.23 | -0.01 | -0.24 | | WdDiffWdStem | -0.28 | 0.19 | 0.03 | 0.16 | -0.28 | | WdMaxDpthHypernymTree | -0.02 | -0.26 | -0.09 | 0.14 | -0.23 | | WdAvgDpthHypernymTree | 0 | -0.27 | -0.11 | 0.13 | -0.24 | | WdPathCntHypernymTree | -0.1 | -0.26 | -0.17 | 0.16 | -0.09 | | WdPolysemyCnt | 0.06 | 0.11 | -0.09 | -0.09 | 0.35 | | WdSylCnt | -0.1 | 0.05 | -0.26 | 0.11 | -0.46 | | AvgAOADoc_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 | | AvgAOABl_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 | | AvgAOASen_Shock | 0.46 | 0.21 | 0.05 | 0.31 | -0.27 | | AvgAOADoc_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 | | AvgAOABl_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 | | AvgAOASen_Cortese | 0.14 | 0.01 | 0.55 | 0.3 | 0.17 | | AvgAOADoc_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 | | AvgAOABl_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 | | AvgAOASen_Kuperman | -0.03 | -0.04 | 0.42 | 0.41 | -0.3 | | AvgAOADoc_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 | | AvgAOABl_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 | | AvgAOASen_Bird | 0.23 | 0.12 | 0.43 | 0.36 | 0.21 | | AvgAOADoc_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 | | AvgAOABl_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 | | AvgAOASen_Bristol | 0.37 | 0.08 | 0.34 | 0.26 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 | | AvgAOEBl_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 | | AvgAOESen_IndexPolyFAT.3 | 0.02 | -0.02 | 0.74 | -0.12 | -0.17 | | AvgAOEDoc_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 | | AvgAOEBl_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 | | AvgAOESen_InvLinRegSlo | 0.17 | -0.02 | 0.67 | -0.02 | -0.11 | | AvgAOEDoc_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 | | AvgAOEBl_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 | | AvgAOESen_InfPointPoly | 0.07 | 0.03 | 0.73 | 0.02 | 0.05 | | AvgAOEDoc_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 | | AvgAOEBl_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 | | AvgAOESen_InvAverage | 0.06 | 0.02 | 0.74 | 0.03 | 0.02 | | AvgAOEDoc_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 | | AvgAOEBl_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 | | AvgAOESen_IndexAbThr.0.3. | 0.05 | 0 | 0.75 | -0.04 | -0.27 | | AvgNmdEntBl | -0.07 | 0.32 | 0.15 | -0.11 | -0.44 | | AvgNounNmdEntBl | -0.02 | 0.24 | 0.24 | -0.13 | -0.43 | | AvgUnqNmdEntBl | -0.07 | 0.34 | 0.12 | -0.09 | -0.48 | | AvgNmdEntSen | 0.59 | 0.01 | 0.19 | -0.09 | -0.26 | | TCorefChainDoc | 0.04 | 0.69 | 0.06 | -0.2 | -0.01 | | AvgCorefChain | 0.02 | 0.46 | -0.18 | 0 | 0.11 | | AvgChainSpan | 0.06 | 0.73 | -0.03 | 0.07 | -0.1 | | AvgInferenceDistChain | 0.45 | 0.31 | -0.02 | 0.31 | 0.07 | | TActCorefChainWd | -0.02 | -0.25 | -0.04 | -0.19 | 0.04 | | TCorefChainBigSpan | 0.19 | 0.52 | -0.02 | -0.09 | -0.01 | | AvgConnBl_addition | 0.1 | 0.53 | 0.1 | -0.58 | -0.08 | | AvgConnSen_addition | 0.74 | 0.04 | 0.05 | -0.4 | -0.02 | | AvgConnBl_conjunctions | 0.15 | 0.59 | 0.09 | -0.57 | 0.06 | | AvgConnSen_conjunctions | 0.83 | 0.04 | 0.06 | -0.36 | 0.06 | | AvgConnBl_contrasts | 0.19 | 0.37 | 0 | -0.11 | 0.28 | | AvgConnSen_contrasts | 0.63 | 0.11 | 0.04 | -0.11 | 0.22 | | AvgConnBl_coord_conjs | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 | | AvgConnSen_coord_conjs | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 | | AvgConnBl_coord_connects | 0.21 | 0.7 | -0.02 | -0.4 | 0.18 | | AvgConnSen_coord_connects | 0.88 | 0.1 | -0.02 | -0.23 | 0.17 | | AvgConnBl_logical_conn | 0.09 | 0.5 | 0.07 | -0.64 | 0.05 | | AvgConnSen_logical_conn | 0.67 | -0.02 | 0.03 | -0.5 | 0.1 | | AvgConnBl_oppositions | 0.19 | 0.37 | 0.05 | 0.02 | 0.22 | | AvgConnSen_oppositions | 0.6 | 0.16 | 0.07 | 0.12 | 0.11 | | AvgConnBl_order | 0.12 | 0.38 | 0.05 | -0.29 | -0.16 | | AvgConnSen_order | 0.57 | 0.14 | 0.02 | -0.07 | -0.1 | | AvgConnBl_reas_purp | 0.18 | 0.58 | -0.07 | -0.03 | 0.09 | | AvgConnSen_reas_purp | 0.73 | 0.25 | -0.06 | 0.08 | 0.09 | | AvgConnBl_semi_coords | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 | | AvgConnSen_semi_coords | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 | | AvgConnBl_sentence_link | 0.2 | 0.77 | -0.01 | -0.33 | 0.13 | | AvgConnSen_sentence_link | 0.92 | 0.14 | -0.03 | -0.13 | 0.12 | | AvgConnBl_simp_subords | -0.02 | 0.41 | -0.05 | 0.27 | 0.11 | | AvgConnSen_simp_subords | 0.52 | 0.18 | -0.09 | 0.34 | 0.16 | | AvgConnBl_temp_conn | -0.14 | 0.36 | 0.05 | -0.23 | 0.08 | | AvgConnSen_temp_conn | 0.36 | 0.06 | 0.04 | -0.25 | 0.27 | | LexChainAvgSpan | 0.07 | 0.49 | 0.03 | -0.01 | 0.16 | | LexChainMaxSp | 0 | 0.72 | 0 | -0.02 | 0.08 | | AvgBlScore | 0.14 | 0.82 | 0.01 | 0 | 0.13 | | AvgSenScore | 0.9 | 0.22 | 0.02 | 0.04 | 0.14 | | SenScoreStDev | -0.27 | 0.5 | 0.04 | 0.05 | 0.45 | | AvgIntraBlCoh_LeackChod | -0.73 | 0.44 | 0.16 | 0.2 | 0.17 | | AvgSenAdjCoh_LeackChod | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 | | AvgSenBlCoh_LeackChod | 0.77 | -0.38 | -0.13 | -0.08 | 0.09 | | AvgIntraBlCoh_WuPalmer | -0.74 | 0.44 | 0.16 | 0.2 | 0.18 | | AvgSenAdjCoh_WuPalmer | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 | | AvgSenBlCoh_WuPalmer | 0.79 | -0.39 | -0.13 | -0.09 | 0.08 | | AvgIntraBlCoh_Path | -0.73 | 0.43 | 0.16 | 0.2 | 0.19 | | AvgSenAdjCoh_Path | -0.7 | 0.42 | 0.19 | 0.22 | 0.2 | | AvgSenBlCoh_Path | 0.82 | -0.42 | -0.12 | -0.13 | 0.08 | | AvgIntraBlCoh_LSA | -0.73 | 0.46 | 0.16 | 0.18 | 0.18 | | AvgSenAdjCoh_LSA | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 | | AvgSenBlCoh_LSA | 0.8 | -0.35 | -0.11 | -0.12 | 0.11 | | AvgIntraBlCoh_LDA | -0.73 | 0.48 | 0.16 | 0.18 | 0.15 | | AvgSenAdjCoh_LDA | -0.7 | 0.47 | 0.2 | 0.2 | 0.15 | | AvgSenBlCoh_LDA | 0.75 | -0.21 | -0.13 | -0.11 | 0.08 | | AvgIntraBlCoh_word2vec | -0.73 | 0.45 | 0.16 | 0.19 | 0.18 | | AvgSenAdjCoh_word2vec | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 | | AvgSenBlCoh_word2vec | 0.8 | -0.39 | -0.1 | -0.11 | 0.11 | | AvgBlVoiceCoOcc | 0.03 | 0.63 | -0.05 | -0.04 | -0.01 | | AvgVoice | -0.01 | 0.6 | -0.08 | -0.05 | 0 | | AvgSenSyll | 0.98 | 0.12 | 0.01 | 0.02 | 0.02 | | AvgSenStressedSyll | 0.94 | 0.22 | 0.07 | 0.08 | 0.05 | | AvgRhythmUnits | 0.23 | 0.26 | 0.15 | 0.01 | 0.28 | | AvgRhythmUnitSyll | 0.84 | 0.01 | -0.09 | 0.02 | -0.09 | | AvgRhythmUnitStreesSyll | 0.81 | 0.11 | -0.01 | 0.08 | -0.05 | | LangRhythmCoeff | 0.39 | -0.1 | -0.21 | -0.17 | -0.06 | | LangRhythmId | -0.11 | -0.02 | -0.4 | -0.27 | -0.14 | | FrqRhythmId | 0.72 | -0.3 | 0.01 | -0.12 | 0.04 | | LangRhythmDiameter | 0.27 | -0.01 | -0.34 | -0.25 | -0.19 | | SenAsson | 0.13 | 0.25 | 0.12 | -0.11 | -0.17 | | AvgDepsBl_acl | -0.03 | 0.24 | 0.15 | 0.19 | -0.16 | | AvgDepsSen_acl | 0.44 | 0.03 | 0.26 | 0.09 | -0.13 | | AvgDepsBl_advcl | 0.23 | 0.52 | -0.05 | 0.1 | 0.16 | | AvgDepsSen_advcl | 0.67 | 0.21 | -0.08 | 0.2 | 0.09 | | AvgDepsBl_advmod | 0.07 | 0.67 | 0.03 | -0.25 | 0.13 | | AvgDepsSen_advmod | 0.78 | 0.15 | 0.02 | -0.17 | 0.2 | | AvgDepsBl_amod | -0.07 | 0.39 | 0.16 | 0.06 | -0.2 | | AvgDepsSen_amod | 0.62 | 0.1 | 0.17 | 0 | -0.01 | | AvgDepsBl_aux | 0.13 | 0.37 | -0.13 | 0.13 | 0.24 | | AvgDepsSen_aux | 0.56 | 0.02 | -0.14 | -0.01 | 0.27 | | AvgDepsBl_auxpass | 0.06 | 0.26 | 0.08 | 0.05 | 0.03 | | AvgDepsBl_case | 0.06 | 0.69 | 0.01 | 0.15 | -0.26 | | AvgDepsSen_case | 0.84 | 0.13 | -0.02 | 0.15 | -0.17 | | AvgDepsBl_cc | 0.17 | 0.59 | 0.08 | -0.61 | 0.09 | | AvgDepsSen_cc | 0.8 | 0.02 | 0.05 | -0.42 | 0.11 | | AvgDepsBl_ccomp | 0.37 | 0.51 | -0.03 | -0.01 | 0.32 | | AvgDepsSen_ccomp | 0.78 | 0.16 | -0.01 | -0.01 | 0.19 | | AvgDepsBl_compound | 0.17 | 0.16 | 0.19 | -0.05 | -0.36 | | AvgDepsSen_compound | 0.61 | -0.12 | 0.18 | -0.07 | -0.27 | | AvgDepsBl_conj | 0.23 | 0.52 | 0.06 | -0.58 | 0.14 | | AvgDepsSen_conj | 0.77 | 0.02 | 0.04 | -0.4 | 0.13 | | AvgDepsBl_cop | 0.04 | 0.49 | 0.09 | 0.03 | -0.1 | | AvgDepsSen_cop | 0.71 | 0.16 | 0.14 | 0.04 | -0.03 | | AvgDepsBl_dep | 0.2 | 0.22 | 0.08 | -0.34 | 0.07 | | AvgDepsSen_dep | 0.56 | -0.05 | 0.1 | -0.31 | 0.11 | | AvgDepsBl_det | -0.09 | 0.58 | -0.03 | 0.05 | -0.28 | | AvgDepsSen_det | 0.86 | 0.1 | -0.02 | 0.04 | -0.16 | | AvgDepsBl_dobj | 0 | 0.73 | 0.03 | -0.2 | 0.01 | | AvgDepsSen_dobj | 0.91 | 0.07 | -0.01 | -0.04 | 0.1 | | AvgDepsBl_mark | 0.22 | 0.63 | -0.11 | 0.28 | 0.09 | | AvgDepsSen_mark | 0.78 | 0.25 | -0.1 | 0.28 | 0.05 | | AvgDepsBl_mwe | -0.07 | 0.19 | 0.06 | 0 | -0.02 | | AvgDepsSen_mwe | 0.3 | 0 | 0.03 | -0.09 | 0.05 | | AvgDepsBl_neg | 0.03 | 0.37 | -0.08 | -0.03 | 0.25 | | AvgDepsSen_neg | 0.38 | 0.12 | -0.04 | 0.05 | 0.28 | | AvgDepsBl_nmod | 0.05 | 0.64 | -0.01 | 0.17 | -0.27 | | AvgDepsSen_nmod | 0.83 | 0.07 | -0.06 | 0.14 | -0.18 | | AvgDepsBl_nsubj | 0.03 | 0.9 | -0.1 | -0.1 | 0.08 | | AvgDepsSen_nsubj | 0.94 | 0.14 | -0.07 | 0.01 | 0.14 | | AvgDepsBl_nsubjpass | 0.05 | 0.2 | 0.03 | 0.08 | 0.16 | | AvgDepsBl_nummod | -0.09 | 0.17 | -0.05 | 0.04 | -0.2 | | AvgDepsBl_punct | -0.53 | 0.52 | -0.01 | -0.06 | -0.17 | | AvgDepsSen_punct | -0.13 | 0.35 | 0.14 | 0.12 | 0.26 | | AvgDepsBl_xcomp | 0.08 | 0.49 | -0.03 | 0.01 | -0.11 | | AvgDepsSen_xcomp | 0.63 | 0.15 | 0.02 | -0.06 | -0.1 |
ReaderBench Model 1f
This model was trained on principal component scores for spring data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:-------|:-------| | -9.2262 | 0.1219 | 0.7713 | 0.1603 | -0.3706 | -0.0217 | 0.3129 | 0.0233 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:------|:------|:------|:------| | PC2 | 31.31 | 56.99 | 35.09 | 0 | 11.67 | 23.86 | 35.8 | 21.83 | | PC1 | 11.79 | 6.72 | 17.18 | 0 | 3.69 | 5.61 | 8.89 | 21.83 | | PC4 | 10.38 | 7.69 | 9.11 | 0 | 8.29 | 5.24 | 16.85 | 12.66 | | PC44 | 4.45 | 0.7 | 0.97 | 0 | 7.79 | 1.41 | 10.78 | 3.71 | | PC9 | 4.36 | 2.62 | 4.85 | 0 | 7.31 | 1.81 | 0 | 10.92 | | PC27 | 3.4 | 0.68 | 0.79 | 0 | 1.88 | 1.75 | 12.84 | 0 | | PC14 | 3.37 | 1.1 | 3.45 | 0 | 6.67 | 1.95 | 0 | 7.21 | | PC43 | 2.78 | 1.23 | 1.19 | 0 | 9.29 | 1.77 | 0 | 0 | | PC6 | 2.51 | 0.56 | 3.24 | 0 | 3.33 | 1.2 | 0.08 | 9.17 | | PC16 | 2.38 | 0.82 | 1.17 | 0 | 1.74 | 1.2 | 6.85 | 0 | | PC15 | 2.31 | 1.41 | 2.25 | 0 | 4.31 | 1.89 | 0 | 10.92 | | PC5 | 2.22 | 0.82 | 2.97 | 0 | 2.79 | 2.34 | 0.2 | 1.75 | | PC28 | 1.47 | 0.78 | 1.17 | 0 | 3.69 | 1.94 | 0 | 0 | | PC10 | 1.46 | 1.85 | 1.73 | 0 | 2.08 | 1.98 | 0 | 0 | | PC11 | 1.33 | 1.77 | 1.55 | 0 | 1.91 | 2.26 | 0 | 0 | | PC37 | 1.29 | 0.57 | 0.8 | 0 | 3.79 | 1.57 | 0 | 0 | | PC12 | 1.19 | 0.4 | 1.5 | 0 | 1.87 | 2.16 | 0 | 0 | | PC34 | 1.08 | 0.19 | 0.77 | 0 | 3.06 | 0.68 | 0 | 0 | | PC31 | 1.05 | 0.52 | 0.79 | 0 | 2.74 | 2.09 | 0 | 0 | | PC30 | 0.9 | 0.05 | 0.21 | 0 | 0 | 0.29 | 4.02 | 0 | | PC19 | 0.84 | 1.4 | 0.92 | 0 | 1.29 | 0.05 | 0 | 0 | | PC23 | 0.8 | 0.38 | 0.83 | 0 | 1.59 | 1.37 | 0 | 0 | | PC3 | 0.77 | 0.92 | 0.36 | 0 | 0 | 2.84 | 2.68 | 0 | | PC40 | 0.75 | 0.22 | 0.5 | 0 | 2.14 | 1.16 | 0 | 0 | | PC21 | 0.74 | 0.02 | 0.7 | 0 | 0.87 | 1.03 | 1 | 0 | | PC45 | 0.73 | 0.12 | 0.43 | 0 | 2.23 | 1.46 | 0 | 0 | | PC25 | 0.65 | 0.57 | 0.67 | 0 | 1.22 | 1.65 | 0 | 0 | | PC18 | 0.43 | 0.14 | 0.62 | 0 | 0.45 | 2.08 | 0 | 0 | | PC20 | 0.38 | 0.49 | 0.55 | 0 | 0.3 | 1.36 | 0 | 0 | | PC38 | 0.37 | 0.35 | 0.35 | 0 | 0.72 | 1.69 | 0 | 0 | | PC35 | 0.37 | 0.16 | 0.4 | 0 | 0.7 | 0 | 0 | 0 | | PC32 | 0.27 | 0.8 | 0.34 | 0 | 0.17 | 1.33 | 0 | 0 | | PC17 | 0.24 | 0.08 | 0.47 | 0 | 0 | 1.41 | 0 | 0 | | PC39 | 0.23 | 0.46 | 0.29 | 0 | 0.25 | 1.31 | 0 | 0 | | PC13 | 0.23 | 1.06 | 0.32 | 0 | 0 | 2.26 | 0 | 0 | | PC22 | 0.21 | 0.42 | 0.36 | 0 | 0 | 1.86 | 0 | 0 | | PC41 | 0.2 | 0.42 | 0.27 | 0 | 0.16 | 1.77 | 0 | 0 | | PC7 | 0.18 | 0.41 | 0.3 | 0 | 0 | 1.37 | 0 | 0 | | PC26 | 0.17 | 1.46 | 0.12 | 0 | 0 | 2.05 | 0 | 0 | | PC24 | 0.12 | 1.74 | 0 | 0 | 0 | 1.36 | 0 | 0 | | PC42 | 0.1 | 0.32 | 0.15 | 0 | 0 | 1.77 | 0 | 0 | | PC8 | 0.07 | 0.36 | 0.06 | 0 | 0 | 1.67 | 0 | 0 | | PC33 | 0.06 | 0.12 | 0.08 | 0 | 0 | 1.34 | 0 | 0 | | PC36 | 0.06 | 0 | 0.11 | 0 | 0 | 0.72 | 0 | 0 | | PC29 | 0.02 | 0.13 | 0.02 | 0 | 0 | 2.08 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first five principal components are displayed.
| RC1 | RC2 | RC3 | RC4 | RC5 | | |:----------------------|:------|:------|:------|:------|:-----| | SS loadings | 49.09 | 28.90 | 15.04 | 12.78 | 8.00 | | Proportion Var | 0.24 | 0.14 | 0.07 | 0.06 | 0.04 | | Cumulative Var | 0.24 | 0.39 | 0.46 | 0.53 | 0.57 | | Proportion Explained | 0.43 | 0.25 | 0.13 | 0.11 | 0.07 | | Cumulative Proportion | 0.43 | 0.69 | 0.82 | 0.93 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC3 | RC4 | RC5 | |:--------------------------|:------|:------|:------|:------|:------| | Sentences | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 | | Words | 0.13 | 0.87 | -0.07 | 0.37 | 0.19 | | Content.words | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 | | RdbltyFlesch | -0.83 | -0.11 | -0.08 | 0.04 | 0.04 | | RdbltyFog | 0.9 | 0.08 | 0.01 | -0.01 | 0 | | RdbltyKincaid | 0.89 | 0.07 | 0.02 | 0 | -0.01 | | RdbltyDaleChall | 0.47 | -0.12 | 0.41 | -0.23 | 0.28 | | AvgBlLen | -0.01 | 0.88 | 0.03 | 0.29 | -0.02 | | AvgCommaBl | -0.18 | 0.28 | -0.06 | -0.18 | 0.05 | | AvgSenLen | 0.94 | 0.15 | 0.04 | 0 | -0.04 | | AvgSenBl | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 | | AvgUnqWdBl | 0.02 | 0.89 | 0.06 | 0.22 | 0.02 | | AvgUnqWdSen | 0.95 | 0.12 | 0.03 | 0.01 | -0.04 | | AvgWdLen | -0.12 | 0.48 | 0.04 | -0.15 | -0.47 | | AvgWdBl | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 | | AvgWdSen | 0.95 | 0.1 | 0.02 | 0.01 | 0.03 | | CharEnt | -0.18 | 0.59 | -0.07 | -0.04 | 0.21 | | SenStDevUnqWd | -0.35 | 0.23 | 0.05 | 0.67 | 0.07 | | SenStdDevWd | -0.29 | 0.21 | 0.03 | 0.67 | 0.08 | | WdEnt | -0.01 | 0.88 | 0 | 0.12 | 0.16 | | WdLettStdDev | -0.04 | 0.56 | 0.08 | 0.02 | -0.16 | | LxcDiv | 0.08 | 0.81 | 0.05 | 0.16 | -0.06 | | LxcSoph | 0.56 | 0.08 | 0.09 | -0.08 | -0.48 | | SynSoph | 0.89 | 0.25 | 0.04 | 0.2 | 0.03 | | AvgNounBl | 0.13 | 0.55 | 0.22 | 0.28 | 0.52 | | AvgPronounBl | 0.04 | 0.71 | -0.19 | 0.37 | 0.11 | | AvgVerbBl | 0.04 | 0.84 | -0.17 | 0.36 | 0.02 | | AvgAdverbBl | 0.2 | 0.61 | -0.01 | 0.25 | -0.12 | | AvgAdjectiveBl | -0.08 | 0.65 | 0.05 | 0.06 | 0.04 | | AvgPrepositionBl | 0.11 | 0.79 | -0.1 | 0.18 | 0.08 | | AvgNounSen | 0.89 | -0.04 | 0.07 | -0.01 | 0.26 | | AvgPronounSen | 0.9 | -0.01 | -0.07 | 0.03 | 0.02 | | AvgVerbSen | 0.97 | 0.08 | -0.05 | 0 | -0.04 | | AvgAdverbSen | 0.8 | 0.17 | 0.04 | 0.09 | -0.15 | | AvgAdjectiveSen | 0.81 | 0 | 0 | -0.16 | -0.05 | | AvgPrepositionSen | 0.9 | 0.17 | 0 | 0 | 0.02 | | AvgUnqNoundBl | 0.13 | 0.52 | 0.2 | 0.16 | 0.48 | | AvgUnqPronounBl | -0.01 | 0.54 | -0.15 | 0.18 | 0.09 | | AvgUnqVerbBl | 0.01 | 0.8 | -0.09 | 0.29 | 0.01 | | AvgUnqAdverbBl | 0.12 | 0.69 | -0.01 | 0.1 | -0.11 | | AvgUnqAdjectiveBl | -0.07 | 0.65 | 0.04 | -0.02 | 0.04 | | AvgUnqPrepositionBl | 0.04 | 0.77 | -0.11 | 0.17 | 0.05 | | AvgPronBl_first_person | -0.02 | 0.47 | -0.22 | 0.38 | 0.16 | | AvgPronBl_indefinite | -0.03 | 0.57 | -0.13 | -0.04 | 0.14 | | AggPronSen_indefinite | 0.74 | 0.09 | -0.09 | -0.12 | 0.15 | | AvgPronBl_third_person | 0.08 | 0.54 | -0.03 | 0.14 | -0.03 | | AggPronSen_third_person | 0.78 | 0.06 | -0.02 | -0.05 | -0.13 | | AvgSemDep | 0.98 | 0.06 | -0.01 | 0.01 | 0.08 | | WdDiffLemmaStem | -0.11 | 0.34 | 0.02 | -0.16 | -0.19 | | WdDiffWdStem | -0.11 | 0.39 | -0.03 | -0.1 | -0.42 | | WdMaxDpthHypernymTree | 0.02 | -0.21 | 0.13 | 0.1 | 0.23 | | WdAvgDpthHypernymTree | 0.02 | -0.22 | 0.09 | 0.1 | 0.19 | | WdPathCntHypernymTree | 0.01 | -0.23 | 0.09 | 0.04 | 0.27 | | WdPolysemyCnt | -0.04 | -0.04 | -0.4 | 0.25 | 0.07 | | WdSylCnt | -0.11 | 0.5 | 0.32 | -0.08 | -0.3 | | AvgAOADoc_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 | | AvgAOABl_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 | | AvgAOASen_Shock | 0.49 | 0.17 | -0.02 | 0.08 | -0.39 | | AvgAOADoc_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 | | AvgAOABl_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 | | AvgAOASen_Cortese | 0.25 | -0.19 | 0.35 | 0.03 | -0.39 | | AvgAOADoc_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 | | AvgAOABl_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 | | AvgAOASen_Kuperman | 0.11 | -0.05 | 0.62 | 0 | -0.32 | | AvgAOADoc_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 | | AvgAOABl_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 | | AvgAOASen_Bird | 0.35 | -0.02 | 0.26 | 0.18 | -0.53 | | AvgAOADoc_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 | | AvgAOABl_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 | | AvgAOASen_Bristol | 0.41 | -0.05 | 0.1 | 0.01 | -0.51 | | AvgAOEDoc_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 | | AvgAOEBl_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 | | AvgAOESen_IndexPolyFAT.3 | 0.06 | -0.03 | 0.83 | 0.04 | 0.08 | | AvgAOEDoc_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 | | AvgAOEBl_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 | | AvgAOESen_InvLinRegSlo | 0.17 | -0.15 | 0.69 | 0.07 | -0.03 | | AvgAOEDoc_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 | | AvgAOEBl_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 | | AvgAOESen_InfPointPoly | 0.21 | -0.05 | 0.7 | 0.01 | -0.14 | | AvgAOEDoc_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 | | AvgAOEBl_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 | | AvgAOESen_InvAverage | 0.2 | -0.06 | 0.73 | -0.01 | -0.13 | | AvgAOEDoc_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 | | AvgAOEBl_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 | | AvgAOESen_IndexAbThr.0.3. | -0.03 | 0.02 | 0.78 | 0.04 | 0.18 | | AvgNmdEntBl | 0.06 | 0.31 | 0.06 | 0.04 | 0.62 | | AvgNounNmdEntBl | 0.04 | 0.16 | 0.2 | 0 | 0.67 | | AvgUnqNmdEntBl | 0.03 | 0.34 | 0.06 | 0.03 | 0.63 | | AvgNmdEntSen | 0.57 | -0.06 | 0.08 | 0 | 0.43 | | TCorefChainDoc | 0.02 | 0.51 | -0.04 | 0.33 | 0.25 | | AvgCorefChain | 0 | 0.44 | -0.14 | 0.18 | 0.03 | | AvgChainSpan | 0.17 | 0.66 | -0.09 | 0.15 | -0.05 | | AvgInferenceDistChain | 0.26 | 0.29 | -0.06 | 0.06 | 0.15 | | TActCorefChainWd | -0.09 | -0.34 | -0.03 | -0.01 | 0.03 | | TCorefChainBigSpan | 0.21 | 0.34 | -0.02 | 0.16 | 0.02 | | AvgConnBl_addition | 0.45 | 0.25 | -0.03 | 0.6 | 0.11 | | AvgConnSen_addition | 0.85 | -0.02 | 0.02 | 0.17 | 0.09 | | AvgConnBl_conjunctions | 0.48 | 0.32 | -0.03 | 0.43 | 0.19 | | AvgConnSen_conjunctions | 0.91 | -0.02 | 0.02 | 0.05 | 0.11 | | AvgConnBl_contrasts | 0.16 | 0.51 | 0.12 | -0.15 | 0 | | AvgConnSen_contrasts | 0.67 | 0.22 | 0.05 | -0.24 | -0.1 | | AvgConnBl_coord_conjs | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 | | AvgConnSen_coord_conjs | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 | | AvgConnBl_coord_connects | 0.47 | 0.41 | -0.03 | 0.52 | 0.09 | | AvgConnSen_coord_connects | 0.94 | 0.01 | 0.02 | 0.08 | 0.05 | | AvgConnBl_logical_conns | 0.44 | 0.24 | -0.04 | 0.5 | 0.17 | | AvgConnSen_logical_conns | 0.86 | -0.04 | 0.03 | 0.12 | 0.1 | | AvgConnBl_oppositions | 0.17 | 0.49 | 0.07 | -0.22 | 0.04 | | AvgConnSen_oppositions | 0.66 | 0.19 | 0.03 | -0.29 | -0.05 | | AvgConnBl_order | 0.04 | 0.16 | -0.02 | 0.45 | -0.06 | | AvgConnSen_order | 0.4 | -0.09 | 0.01 | 0.28 | 0.01 | | AvgConnBl_reas_purp | 0.07 | 0.32 | -0.03 | 0.49 | -0.16 | | AvgConnSen_reas_purp | 0.58 | 0.04 | 0.03 | 0.25 | -0.17 | | AvgConnBl_semi_coords | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 | | AvgConnSen_semi_coords | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 | | AvgConnBl_sentence_link | 0.39 | 0.51 | -0.1 | 0.58 | 0.07 | | AvgConnSen_sentence_link | 0.95 | 0 | 0 | 0.11 | 0.06 | | AvgConnBl_simp_subords | -0.13 | 0.52 | -0.14 | -0.04 | 0.01 | | AvgConnSen_simp_subords | 0.52 | 0.16 | -0.03 | -0.1 | -0.02 | | AvgConnBl_temp_conns | -0.1 | 0.34 | -0.21 | 0.22 | 0.04 | | AvgConnSen_temp_conns | 0.33 | 0 | -0.12 | 0.07 | 0.06 | | LexChainAvgSpan | -0.04 | 0.29 | -0.21 | 0.47 | 0.17 | | LexChainMaxSp | 0.04 | 0.61 | -0.1 | 0.45 | 0.02 | | AvgBlScore | 0.22 | 0.56 | -0.18 | 0.53 | 0.08 | | AvgSenScore | 0.93 | 0.05 | -0.06 | 0.06 | 0.07 | | SenScoreStDev | -0.39 | 0.27 | -0.02 | 0.67 | 0.07 | | AvgIntraBlCoh_LeackChod | -0.71 | 0.35 | 0.01 | 0.54 | 0.08 | | AvgSenAdjCoh_LeackChod | -0.69 | 0.33 | 0.02 | 0.56 | 0.07 | | AvgSenBlCoh_LeackChod | 0.76 | -0.5 | -0.07 | -0.05 | -0.17 | | AvgIntraBlCoh_WuPalmer | -0.71 | 0.35 | 0.01 | 0.53 | 0.08 | | AvgSenAdjCoh_WuPalmer | -0.7 | 0.33 | 0.01 | 0.55 | 0.08 | | AvgSenBlCoh_WuPalmer | 0.76 | -0.51 | -0.08 | -0.07 | -0.17 | | AvgIntraBlCoh_Path | -0.72 | 0.34 | 0 | 0.53 | 0.09 | | AvgSenAdjCoh_Path | -0.7 | 0.32 | 0 | 0.55 | 0.08 | | AvgSenBlCoh_Path | 0.78 | -0.52 | -0.07 | -0.17 | -0.14 | | AvgIntraBlCoh_LSA | -0.7 | 0.35 | -0.01 | 0.55 | 0.09 | | AvgSenAdjCoh_LSA | -0.68 | 0.34 | -0.01 | 0.57 | 0.09 | | AvgSenBlCoh_LSA | 0.76 | -0.49 | -0.13 | -0.04 | -0.15 | | AvgIntraBlCoh_LDA | -0.7 | 0.37 | -0.02 | 0.53 | 0.08 | | AvgSenAdjCoh_LDA | -0.68 | 0.35 | -0.02 | 0.55 | 0.07 | | AvgSenBlCoh_LDA | 0.71 | -0.39 | -0.2 | 0.02 | -0.19 | | AvgIntraBlCoh_word2vec | -0.69 | 0.35 | 0.01 | 0.55 | 0.08 | | AvgSenAdjCoh_word2vec | -0.69 | 0.33 | 0 | 0.56 | 0.09 | | AvgSenBlCoh_word2vec | 0.76 | -0.53 | -0.09 | -0.07 | -0.13 | | AvgBlVoiceCoOcc | -0.05 | 0.47 | -0.14 | 0.51 | 0.08 | | AvgVoice | -0.05 | 0.46 | -0.15 | 0.54 | 0.08 | | AvgSenSyll | 0.98 | 0.06 | 0.01 | -0.01 | 0.06 | | AvgSenStressedSyll | 0.96 | 0.08 | 0.03 | 0.01 | 0.06 | | AvgRhythmUnits | 0.18 | 0 | -0.08 | -0.18 | 0.01 | | AvgRhythmUnitSyll | 0.81 | -0.04 | 0.02 | 0.09 | 0.09 | | AvgRhythmUnitStreesSyll | 0.8 | -0.03 | 0.04 | 0.11 | 0.1 | | LangRhythmCoeff | 0.27 | -0.25 | -0.03 | 0.24 | 0.22 | | LangRhythmId | -0.19 | 0.11 | 0.02 | -0.04 | 0.21 | | FrqRhythmId | 0.68 | -0.44 | -0.04 | -0.07 | -0.11 | | LangRhythmDiameter | 0.34 | 0.07 | 0 | 0.11 | 0.35 | | SenAsson | -0.05 | 0.28 | -0.05 | 0.17 | -0.09 | | AvgDepsBl_acl | -0.14 | 0.18 | -0.12 | 0.02 | -0.13 | | AvgDepsSen_acl | 0.12 | -0.23 | -0.17 | -0.14 | -0.17 | | AvgDepsBl_advcl | 0.14 | 0.49 | -0.08 | 0.25 | -0.2 | | AvgDepsSen_advcl | 0.66 | 0.18 | 0.02 | 0.03 | -0.23 | | AvgDepsBl_advmod | 0.14 | 0.61 | -0.03 | 0.33 | -0.12 | | AvgDepsSen_advmod | 0.79 | 0.15 | 0.01 | 0.15 | -0.12 | | AvgDepsBl_amod | -0.1 | 0.58 | 0.09 | 0.11 | 0.15 | | AvgDepsSen_amod | 0.7 | 0 | 0.02 | -0.07 | 0.09 | | AvgDepsBl_aux | 0.16 | 0.54 | -0.15 | 0.06 | -0.03 | | AvgDepsSen_aux | 0.76 | 0.21 | -0.09 | -0.06 | -0.08 | | AvgDepsBl_auxpass | 0.04 | 0.4 | -0.05 | 0.2 | -0.08 | | AvgDepsBl_case | 0.08 | 0.67 | -0.08 | 0.16 | 0.28 | | AvgDepsSen_case | 0.84 | 0.1 | -0.03 | -0.01 | 0.17 | | AvgDepsBl_cc | 0.49 | 0.35 | 0 | 0.43 | 0.19 | | AvgDepsSen_cc | 0.92 | 0 | 0.03 | 0.05 | 0.09 | | AvgDepsBl_ccomp | 0.2 | 0.28 | -0.25 | 0.31 | 0.11 | | AvgDepsSen_ccomp | 0.76 | -0.05 | -0.15 | -0.02 | 0.11 | | AvgDepsBl_compound | -0.02 | -0.06 | 0.5 | 0.1 | 0.4 | | AvgDepsSen_compound | 0.37 | -0.22 | 0.24 | 0.02 | 0.28 | | AvgDepsBl_conj | 0.6 | 0.31 | 0.02 | 0.37 | 0.15 | | AvgDepsSen_conj | 0.88 | 0.02 | 0.02 | 0.03 | 0.08 | | AvgDepsBl_cop | -0.05 | 0.49 | -0.09 | -0.11 | -0.03 | | AvgDepsSen_cop | 0.64 | 0.02 | -0.13 | -0.16 | -0.15 | | AvgDepsBl_dep | 0.38 | 0.28 | 0.03 | 0.25 | 0.25 | | AvgDepsSen_dep | 0.73 | 0.01 | 0.05 | 0.02 | 0.18 | | AvgDepsBl_det | 0.14 | 0.58 | 0.03 | 0.31 | 0.22 | | AvgDepsSen_det | 0.76 | 0.11 | -0.01 | 0.03 | 0.19 | | AvgDepsBl_dobj | 0.14 | 0.56 | -0.02 | 0.47 | 0.08 | | AvgDepsSen_dobj | 0.9 | -0.03 | 0 | 0.06 | 0.06 | | AvgDepsBl_mark | 0.09 | 0.6 | -0.1 | 0.25 | -0.23 | | AvgDepsSen_mark | 0.75 | 0.1 | -0.03 | 0 | -0.24 | | AvgDepsBl_mwe | 0.14 | 0.34 | 0.04 | 0.06 | 0.15 | | AvgDepsSen_mwe | 0.45 | 0.15 | 0.08 | 0.04 | 0.14 | | AvgDepsBl_neg | 0 | 0.38 | 0.01 | -0.11 | 0.02 | | AvgDepsSen_neg | 0.39 | 0.05 | 0.05 | -0.21 | -0.07 | | AvgDepsBl_nmod | 0.09 | 0.62 | -0.1 | 0.16 | 0.3 | | AvgDepsSen_nmod | 0.83 | 0.08 | -0.03 | -0.03 | 0.2 | | AvgDepsBl_nsubj | 0.07 | 0.79 | -0.16 | 0.34 | 0.15 | | AvgDepsSen_nsubj | 0.97 | 0.03 | -0.04 | 0 | 0.04 | | AvgDepsBl_nsubjpass | 0.05 | 0.34 | -0.04 | 0.2 | -0.04 | | AvgDepsBl_nummod | 0.02 | 0.2 | -0.15 | 0.06 | 0.24 | | AvgDepsBl_punct | -0.53 | 0.62 | -0.01 | -0.15 | 0.13 | | AvgDepsSen_punct | -0.11 | 0.31 | 0 | -0.06 | -0.15 | | AvgDepsBl_xcomp | 0.03 | 0.41 | 0.03 | 0.34 | -0.22 | | AvgDepsSen_xcomp | 0.71 | 0.07 | 0.07 | 0.06 | -0.24 |
ReaderBench Model 2 {#readerbench-model-2}
General Description
ReaderBench Model 2 is a simplified version of Model 1 that better handles multi-paragraph compositions, and Model 2 is recommended over Model 1.
Model 2 is an ensemble (formed by averaging predicted quality scores) of three sub-models that are described below. All of these models used ReaderBench scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
ReaderBench Model 2a
This model was trained with fall data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model pls = partial least squares regression rf = random forest regression mars = bagged multivariate adaptive regression splines svm = support vector machines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | -4.338 | 0.2371 | 0.1755 | 0.1780 | 0.2234 | 0.2532 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | svm | cube | |:------------------------------------------------|:--------|:-----|:------|:------|:-----|:------| | WdEnt | 20.53 | 4.67 | 10.12 | 73.84 | 5.16 | 18.67 | | AvgDepsSen_dep | 4.65 | 1.23 | 0.88 | 16.82 | 0.88 | 5.25 | | Content.words | 4.59 | 4.32 | 4.77 | 0 | 4.68 | 7.87 | | Words | 3.72 | 4.44 | 4.67 | 0 | 4.67 | 4.17 | | LxcDiv | 3.1 | 4.08 | 3.29 | 0 | 4.06 | 3.4 | | AvgAOASen_Shock | 2.77 | 1.45 | 1.16 | 9.34 | 1.39 | 1.7 | | TCorefChainDoc | 2.62 | 2.98 | 0.81 | 0 | 2.06 | 5.86 | | AvgChainSpan | 2.59 | 3.27 | 3.83 | 0 | 3.1 | 2.47 | | WdDiffWdStem | 2.46 | 2.73 | 3.07 | 0 | 2.24 | 3.7 | | SynSoph | 2.12 | 1.71 | 0.92 | 0 | 1.82 | 5.09 | | AvgDepsSen_punct | 2.03 | 2.48 | 1.68 | 0 | 1.74 | 3.55 | | TActCorefChainWd | 1.93 | 1.6 | 1.91 | 0 | 1.47 | 4.01 | | WdDiffLemmaStem | 1.66 | 1.52 | 0.72 | 0 | 2.44 | 2.93 | | RdbltyFlesch | 1.55 | 0.77 | 1.22 | 0 | 1.09 | 4.01 | | WdLettStdDev | 1.44 | 2.35 | 1.51 | 0 | 2.13 | 0.93 | | AvgAOESen_InverseAverage | 1.37 | 1.44 | 1.22 | 0 | 1.09 | 2.62 | | Sentences | 1.3 | 2.84 | 1.77 | 0 | 1.82 | 0 | | AvgWdLen | 1.27 | 2.65 | 1.57 | 0 | 2.02 | 0 | | LexChainMaxSp | 1.26 | 2.89 | 1.19 | 0 | 2.02 | 0 | | AvgAOADoc_Shock | 1.26 | 2.36 | 1.89 | 0 | 1.68 | 0.31 | | AvgAOADoc_Kuperman | 1.25 | 0.72 | 1.01 | 0 | 1.3 | 2.78 | | WdSylCnt | 1.15 | 1.57 | 1.83 | 0 | 1.51 | 0.77 | | CharEnt | 1.14 | 2.65 | 0.96 | 0 | 1.85 | 0 | | LexChainAvgSpan | 1.12 | 2.18 | 1.5 | 0 | 1.86 | 0 | | AvgDepsSen_advcl | 1.07 | 0.93 | 0.85 | 0 | 1.35 | 1.85 | | AvgAOASen_Kuperman | 1.04 | 1.23 | 1.48 | 0 | 1.46 | 0.93 | | AvgCorefChain | 1 | 1.86 | 0.85 | 0 | 0.9 | 1.08 | | WdAvgDpthHypernymTree | 1 | 1.14 | 0.87 | 0 | 0.97 | 1.7 | | SenStdDevWd | 0.98 | 1.96 | 1.43 | 0 | 1.49 | 0 | | TCorefChainBigSpan | 0.95 | 2.16 | 1.44 | 0 | 1.13 | 0 | | AvgAOADoc_Bristol | 0.94 | 1.75 | 1.03 | 0 | 1.1 | 0.62 | | LxcSoph | 0.92 | 1.64 | 1.2 | 0 | 0.85 | 0.77 | | AvgAdverbSen | 0.88 | 0.89 | 1.38 | 0 | 1.46 | 0.62 | | RdbltyDaleChall | 0.87 | 1.75 | 1.63 | 0 | 1 | 0 | | AvgSenAdjCoh_LDA | 0.82 | 1.97 | 0.64 | 0 | 1.33 | 0 | | AvgRhythmUnits | 0.82 | 1.12 | 1.13 | 0 | 1.15 | 0.62 | | FrqRhythmId | 0.8 | 1.69 | 1.07 | 0 | 1.18 | 0 | | AvgAOADoc_Bird | 0.78 | 0.95 | 0.3 | 0 | 1.43 | 0.93 | | AvgVoice | 0.78 | 2.01 | 0.76 | 0 | 0.99 | 0 | | AvgAOADoc_Cortese | 0.77 | 0.69 | 1.3 | 0 | 1.57 | 0.31 | | WdPathCntHypernymTree | 0.71 | 1.45 | 0.84 | 0 | 1.17 | 0 | | AvgConnSen_simple_subordinators | 0.7 | 0.51 | 2.49 | 0 | 0.82 | 0 | | AvgAOASen_Bristol | 0.68 | 0.66 | 0.71 | 0 | 1.29 | 0.62 | | AvgRhythmUnitStreesSyll | 0.63 | 0.08 | 0.91 | 0 | 0.81 | 1.23 | | AvgInferenceDistChain | 0.62 | 1.39 | 0.34 | 0 | 1.2 | 0 | | AggPronSen_indefinite | 0.62 | 0.45 | 0.63 | 0 | 1.31 | 0.62 | | AvgAOASen_Bird | 0.6 | 1.13 | 0.37 | 0 | 1.37 | 0 | | AvgDepsSen_compound | 0.6 | 0.72 | 0.5 | 0 | 0.48 | 1.08 | | WdPolysemyCnt | 0.58 | 0 | 1.09 | 0 | 1.9 | 0 | | AvgDepsSen_ccomp | 0.57 | 0.09 | 1.32 | 0 | 0.9 | 0.62 | | AvgAOASen_Cortese | 0.55 | 1.15 | 0.3 | 0 | 1.17 | 0 | | AvgDepsSen_cop | 0.54 | 0.24 | 0.58 | 0 | 0.97 | 0.77 | | AvgPronounSen | 0.54 | 0.12 | 0.93 | 0 | 0.48 | 1.08 | | AvgNmdEntSen | 0.52 | 0.24 | 1.12 | 0 | 1.33 | 0 | | AvgNounSen | 0.52 | 0.24 | 0.15 | 0 | 0.18 | 1.7 | | AvgDepsSen_nmod | 0.48 | 0.7 | 0.69 | 0 | 1 | 0 | | AvgDepsSen_aux | 0.48 | 0.24 | 0.92 | 0 | 1.31 | 0 | | AvgConnSen_addition | 0.48 | 1.1 | 0.6 | 0 | 0.66 | 0 | | AvgDepsSen_dobj | 0.48 | 0.23 | 1.51 | 0 | 0.16 | 0.62 | | AvgAOEDoc_InverseLinearRegressionSlope | 0.44 | 0.4 | 0.8 | 0 | 0.68 | 0.31 | | AvgDepsSen_mark | 0.41 | 0.43 | 0.95 | 0 | 0.73 | 0 | | AvgConnSen_temporal_connectors | 0.41 | 0.32 | 0.64 | 0 | 1.11 | 0 | | AvgDepsSen_det | 0.4 | 0.18 | 0.4 | 0 | 0.72 | 0.62 | | AvgConnSen_semi_coordinators | 0.38 | 0.8 | 0.15 | 0 | 0.16 | 0.62 | | AvgConnSen_order | 0.36 | 0.31 | 1.74 | 0 | 0.03 | 0 | | AggPronSen_third_person | 0.36 | 0.57 | 0.91 | 0 | 0.41 | 0 | | LangRhythmDiameter | 0.35 | 0.57 | 0.79 | 0 | 0.08 | 0.31 | | SenAsson | 0.35 | 0.8 | 0.83 | 0 | 0.16 | 0 | | AvgAOEDoc_IndexAboveThreshold.0.3. | 0.33 | 0.03 | 0.43 | 0 | 0.87 | 0.31 | | AvgDepsSen_amod | 0.29 | 0.33 | 0.98 | 0 | 0.27 | 0 | | AvgAdjectiveSen | 0.28 | 0.1 | 1.28 | 0 | 0.21 | 0 | | AvgConnSen_oppositions | 0.27 | 0.54 | 0.82 | 0 | 0.07 | 0 | | AvgDepsSen_xcomp | 0.24 | 0.01 | 0.13 | 0 | 1.04 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.21 | 0.12 | 0.1 | 0 | 0.78 | 0 | | LangRhythmId | 0.19 | 0.47 | 0.45 | 0 | 0.05 | 0 | | AvgDepsSen_neg | 0.18 | 0.03 | 1.05 | 0 | 0 | 0 | | AvgDepsSen_mwe | 0.17 | 0.38 | 0.47 | 0 | 0.04 | 0 | | LangRhythmCoeff | 0.16 | 0 | 0.22 | 0 | 0.61 | 0 | | AvgDepsSen_acl | 0.06 | 0.25 | 0 | 0 | 0.02 | 0 |
ReaderBench Model 2b
This model was trained with winter data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model pls = partial least squares regression rf = random forest regression mars = bagged multivariate adaptive regression splines svm = support vector machines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | -5.4658 | 0.2205 | 0.5768 | 0.2047 | 0.0528 | 0.0400 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | svm | cube | |:------------------------------------------------|:--------|:-----|:-----|:------|:-----|:------| | Content.words | 11.94 | 5.23 | 4.51 | 41.33 | 4.49 | 15.7 | | WdEnt | 8.27 | 5.15 | 5.03 | 19.27 | 4.44 | 21.01 | | SynSoph | 4.17 | 1.03 | 2.06 | 14.97 | 1.69 | 0 | | LxcDiv | 3.24 | 4.93 | 3.5 | 0 | 3.86 | 5.8 | | AvgDepsSen_det | 3.18 | 0.25 | 1.12 | 12.65 | 0.94 | 3.38 | | TCorefChainDoc | 2.63 | 4 | 2.76 | 0 | 2.45 | 6.76 | | AvgChainSpan | 2.28 | 3.64 | 2.57 | 0 | 2.88 | 1.45 | | LexChainMaxSp | 2.25 | 3.56 | 2.72 | 0 | 1.98 | 0 | | TActCorefChainWd | 2.22 | 0.83 | 0.71 | 7.46 | 0.78 | 6.76 | | Sentences | 2.18 | 3.8 | 2.48 | 0 | 2.25 | 0 | | AvgNounSen | 2.07 | 0.89 | 1.54 | 4.33 | 0.64 | 6.52 | | CharEnt | 1.7 | 3.41 | 1.66 | 0 | 2.16 | 0.97 | | RdbltyFlesch | 1.36 | 0.46 | 1.91 | 0 | 1.18 | 5.56 | | WdLettStdDev | 1.31 | 2.58 | 1.31 | 0 | 2 | 0 | | AvgSenAdjCoh_LeackockChodorow | 1.3 | 2.75 | 1.24 | 0 | 1.87 | 0 | | FrqRhythmId | 1.28 | 2.48 | 1.39 | 0 | 1.13 | 0 | | AvgDepsSen_aux | 1.25 | 0.94 | 1.69 | 0 | 0.97 | 3.38 | | AvgWdLen | 1.24 | 2.36 | 1.26 | 0 | 2.04 | 0 | | AvgAOADoc_Bristol | 1.21 | 1.59 | 1.61 | 0 | 0.88 | 0 | | AvgDepsSen_compound | 1.2 | 1.17 | 1.78 | 0 | 0.48 | 0 | | AvgVoice | 1.2 | 2.75 | 1.13 | 0 | 1.19 | 0 | | AvgAOADoc_Shock | 1.16 | 2.54 | 1.13 | 0 | 1.17 | 0 | | TCorefChainBigSpan | 1.13 | 2.24 | 1.22 | 0 | 0.78 | 0 | | AvgConnSen_addition | 1.07 | 1.31 | 1.29 | 0 | 1.31 | 1.69 | | WdDiffWdStem | 1.04 | 2.05 | 1.06 | 0 | 1.35 | 0 | | AvgConnSen_logical_connectors | 1.03 | 1.49 | 1.11 | 0 | 1.27 | 2.17 | | AvgCorefChain | 1.01 | 2.38 | 0.9 | 0 | 1.32 | 0 | | AggPronSen_third_person | 0.98 | 1.18 | 1.23 | 0 | 0.56 | 1.93 | | AvgDepsSen_punct | 0.98 | 1.89 | 1.01 | 0 | 1.48 | 0 | | AvgDepsSen_dep | 0.95 | 1.01 | 1.31 | 0 | 1.07 | 0 | | AvgRhythmUnitStreesSyll | 0.95 | 0.44 | 1.46 | 0 | 0.9 | 1.21 | | AvgDepsSen_dobj | 0.95 | 1.12 | 1.04 | 0 | 1.07 | 3.38 | | AvgAdjectiveSen | 0.91 | 0.44 | 1.47 | 0 | 0.9 | 0 | | SenStdDevWd | 0.9 | 2.04 | 0.69 | 0 | 1.52 | 1.21 | | LexChainAvgSpan | 0.87 | 1.85 | 0.77 | 0 | 1.88 | 0 | | WdPathCntHypernymTree | 0.86 | 1.46 | 0.94 | 0 | 1.38 | 0 | | AvgAOESen_InverseAverage | 0.85 | 0.71 | 1.27 | 0 | 0.81 | 0 | | AvgDepsSen_mark | 0.83 | 0.22 | 1.4 | 0 | 1.06 | 0 | | WdPolysemyCnt | 0.83 | 0.32 | 1.43 | 0 | 0.39 | 0 | | AvgConnSen_reason_and_purpose | 0.82 | 0.35 | 1.3 | 0 | 0.7 | 0.97 | | LangRhythmCoeff | 0.8 | 1.15 | 1 | 0 | 0.92 | 0 | | AvgConnSen_simple_subordinators | 0.78 | 0.11 | 1.3 | 0 | 1.47 | 0 | | AvgDepsSen_xcomp | 0.76 | 0.11 | 1.25 | 0 | 1.6 | 0 | | AvgAOASen_Bird | 0.76 | 0.33 | 1.19 | 0 | 0.62 | 0.97 | | AvgDepsSen_ccomp | 0.75 | 0.16 | 1.29 | 0 | 0.79 | 0 | | RdbltyDaleChall | 0.75 | 2.41 | 0.41 | 0 | 1.07 | 0 | | AvgAOEDoc_InflectionPointPolynomial | 0.73 | 0.7 | 1.06 | 0 | 0.65 | 0 | | AvgAOESen_IndexAboveThreshold.0.3. | 0.7 | 0.47 | 1.01 | 0 | 1.43 | 0 | | AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.7 | 0.55 | 1.01 | 0 | 1.13 | 0 | | AggPronSen_indefinite | 0.7 | 0.09 | 1.07 | 0 | 1.64 | 1.21 | | AvgDepsSen_cop | 0.7 | 0.09 | 1.16 | 0 | 1.49 | 0 | | AvgNmdEntSen | 0.68 | 0.45 | 1.02 | 0 | 1.07 | 0 | | AvgConnSen_contrasts | 0.68 | 0.32 | 1.03 | 0 | 0.59 | 1.21 | | AvgConnSen_oppositions | 0.68 | 0.07 | 1.18 | 0 | 0.94 | 0 | | AvgDepsSen_advcl | 0.67 | 0.03 | 1.13 | 0 | 1.31 | 0 | | AvgAdverbSen | 0.67 | 0.43 | 1.01 | 0 | 1.08 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.66 | 0 | 1.14 | 0 | 1.18 | 0 | | AvgDepsSen_nmod | 0.66 | 0.74 | 0.76 | 0 | 1.35 | 1.21 | | AvgAOADoc_Bird | 0.65 | 0.95 | 0.77 | 0 | 1.11 | 0 | | AvgDepsSen_amod | 0.65 | 0.53 | 0.69 | 0 | 0.9 | 3.86 | | AvgConnSen_semi_coordinators | 0.64 | 0.24 | 1.04 | 0 | 0.78 | 0 | | WdMaxDpthHypernymTree | 0.62 | 1.46 | 0.46 | 0 | 1.61 | 0 | | AvgAOASen_Shock | 0.62 | 1.13 | 0.62 | 0 | 1.34 | 0 | | AvgAOASen_Kuperman | 0.6 | 0.15 | 1.01 | 0 | 0.45 | 0.48 | | AvgConnSen_temporal_connectors | 0.58 | 0.27 | 0.99 | 0 | 0.01 | 0 | | AvgAOASen_Bristol | 0.57 | 0.38 | 0.87 | 0 | 0.67 | 0 | | LangRhythmDiameter | 0.56 | 0.65 | 0.81 | 0 | 0.06 | 0 | | AvgConnSen_order | 0.52 | 0.29 | 0.7 | 0 | 1 | 1.21 | | AvgAOEDoc_IndexAboveThreshold.0.3. | 0.5 | 0.01 | 0.79 | 0 | 1.65 | 0 | | AvgRhythmUnits | 0.5 | 0.73 | 0.57 | 0 | 1.13 | 0 | | AvgAOADoc_Kuperman | 0.5 | 0.14 | 0.82 | 0 | 0.86 | 0 | | AvgAOASen_Cortese | 0.49 | 0.13 | 0.83 | 0 | 0.66 | 0 | | AvgInferenceDistChain | 0.48 | 0.87 | 0.51 | 0 | 0.71 | 0 | | WdDiffLemmaStem | 0.48 | 0.4 | 0.62 | 0 | 1.55 | 0 | | SenAsson | 0.42 | 0.99 | 0.4 | 0 | 0.15 | 0 | | AvgDepsSen_mwe | 0.41 | 0.66 | 0.52 | 0 | 0.07 | 0 | | AvgDepsSen_neg | 0.39 | 0.28 | 0.64 | 0 | 0 | 0 | | AvgDepsSen_acl | 0.33 | 0.45 | 0.46 | 0 | 0.03 | 0 | | LxcSoph | 0.31 | 0.39 | 0.35 | 0 | 0.92 | 0 | | AvgAOEDoc_InverseLinearRegressionSlope | 0.27 | 0.19 | 0.39 | 0 | 0.61 | 0 | | AvgAOADoc_Cortese | 0.24 | 0.76 | 0.09 | 0 | 0.85 | 0 | | WdSylCnt | 0.23 | 0.83 | 0 | 0 | 1.29 | 0 | | LangRhythmId | 0.03 | 0.09 | 0.03 | 0 | 0 | 0 |
ReaderBench Model 2c
This model was trained on spring data from [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model pls = partial least squares regression rf = random forest regression mars = bagged multivariate adaptive regression splines gbm = stochastic gradient boosted trees svm = support vector machines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -7.3027 | 0.2354 | 0.1868 | 0.1595 | 0.1816 | 0.2191 | 0.0704 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | gbm | svm | cube | |:------------------------------------------------|:--------|:-----|:-----|:------|:------|:-----|:------| | Content.words | 11.99 | 4.55 | 5.81 | 30.16 | 21.71 | 4.24 | 11.11 | | WdEnt | 7.28 | 4.3 | 5.74 | 0 | 21.09 | 4.12 | 12.09 | | AvgDepsSen_compound | 3.97 | 2.07 | 1.98 | 13.22 | 2.22 | 1.52 | 6.82 | | AvgWdLen | 3.87 | 2.64 | 2.65 | 7.11 | 4.85 | 2.04 | 7.02 | | LxcDiv | 3.77 | 4.06 | 4.13 | 0 | 7.72 | 3.59 | 0.78 | | AvgChainSpan | 3.36 | 3.09 | 2.64 | 5.1 | 4.15 | 2.66 | 2.34 | | TCorefChainBigSpan | 2.64 | 2.23 | 1.48 | 10.59 | 0.43 | 0.93 | 0 | | Sentences | 2.37 | 3.33 | 2.15 | 0 | 2.63 | 2.24 | 4.87 | | AvgDepsSen_mark | 2.21 | 0.38 | 1.17 | 10.59 | 0.08 | 1.45 | 0 | | AvgDepsSen_dobj | 2 | 0.81 | 0.96 | 8.72 | 0.14 | 1.13 | 0.97 | | AvgSenAdjCoh_LSA | 1.95 | 2.68 | 1.87 | 0 | 3.17 | 2.26 | 0 | | AvgCorefChain | 1.94 | 2.2 | 1 | 5.1 | 0.28 | 1.28 | 2.73 | | WdDiffWdStem | 1.92 | 2.4 | 1.86 | 0 | 2.95 | 2.09 | 1.56 | | LexChainMaxSp | 1.82 | 3.13 | 2.35 | 0 | 1.28 | 2.01 | 0.97 | | WdLettStdDev | 1.79 | 3 | 1.66 | 0 | 1.64 | 2.28 | 0.97 | | TCorefChainDoc | 1.62 | 3.23 | 1.85 | 0 | 0.17 | 1.92 | 2.14 | | CharEnt | 1.59 | 2.56 | 0.9 | 0 | 0.29 | 2.1 | 5.46 | | WdSylCnt | 1.53 | 2.45 | 1.7 | 0 | 1.52 | 1.55 | 1.36 | | FrqRhythmId | 1.47 | 2.67 | 1.7 | 0 | 1.03 | 1.59 | 0.97 | | AvgDepsSen_punct | 1.36 | 1.82 | 1.57 | 0 | 0.72 | 1.83 | 2.53 | | AvgAOEDoc_InverseLinearRegressionSlope | 1.32 | 1.31 | 0.73 | 4.26 | 0.24 | 0.89 | 0.39 | | RdbltyDaleChall | 1.25 | 1.81 | 1.27 | 0 | 1.04 | 1.02 | 3.51 | | AvgAOADoc_Shock | 1.2 | 2.2 | 1.24 | 0 | 0.69 | 1.8 | 0 | | LangRhythmCoeff | 1.06 | 1.61 | 1.41 | 0 | 1.03 | 1.24 | 0.19 | | LexChainAvgSpan | 1.05 | 1.94 | 1.36 | 0 | 0.16 | 1.66 | 0 | | SenAsson | 1.05 | 1.63 | 0.58 | 3.07 | 0.02 | 0.56 | 0 | | AvgVoice | 1 | 2.62 | 0.58 | 0 | 0 | 1.36 | 0.39 | | AvgNounSen | 0.97 | 1.09 | 1.59 | 0 | 0.47 | 1.06 | 2.14 | | WdDiffLemmaStem | 0.94 | 1.65 | 1.01 | 0 | 0.36 | 1.32 | 0.78 | | TActCorefChainWd | 0.94 | 0.93 | 0.74 | 0 | 1.05 | 0.71 | 4.09 | | AvgAOADoc_Cortese | 0.93 | 1.16 | 0.9 | 0 | 0.56 | 1.89 | 0.39 | | AvgAOASen_Bristol | 0.92 | 0.35 | 1.28 | 2.08 | 1.15 | 0.34 | 0.39 | | SenStdDevWd | 0.92 | 1.6 | 0.97 | 0 | 0.09 | 1.78 | 0 | | AvgDepsSen_xcomp | 0.83 | 0.41 | 1.61 | 0 | 1.42 | 0.99 | 0 | | AvgAdjectiveSen | 0.83 | 1.24 | 1 | 0 | 0.18 | 1.21 | 1.36 | | AvgDepsSen_nmod | 0.81 | 0.16 | 1.23 | 0 | 0.38 | 1.25 | 3.51 | | AvgAOADoc_Kuperman | 0.8 | 0.7 | 1.18 | 0 | 0.65 | 1.44 | 0.39 | | AvgDepsSen_amod | 0.79 | 1.11 | 0.91 | 0 | 0.1 | 1.33 | 1.36 | | AvgDepsSen_ccomp | 0.78 | 1.06 | 1.41 | 0 | 0.27 | 1.18 | 0 | | AvgAOASen_Kuperman | 0.78 | 0.78 | 0.83 | 0 | 0.41 | 0.99 | 2.73 | | AvgNmdEntSen | 0.78 | 0.93 | 1 | 0 | 1.05 | 1.05 | 0 | | AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.76 | 0.58 | 1.12 | 0 | 0.4 | 0.84 | 2.73 | | AvgConnSen_simple_subordinators | 0.74 | 0.25 | 0.86 | 0 | 1.12 | 1.52 | 0.39 | | AvgPronounSen | 0.72 | 1.09 | 1.13 | 0 | 0.02 | 0.99 | 0.97 | | AvgAOASen_Shock | 0.69 | 0.48 | 1.51 | 0 | 0.21 | 1.32 | 0 | | AvgConnSen_reason_and_purpose | 0.68 | 0.16 | 1.31 | 0 | 0.82 | 1.29 | 0 | | AvgAOASen_Cortese | 0.66 | 1.25 | 0.45 | 0 | 0.24 | 1.22 | 0 | | AvgAOESen_InverseLinearRegressionSlope | 0.66 | 0.99 | 1.02 | 0 | 0.31 | 0.68 | 0.97 | | AvgAOEDoc_InflectionPointPolynomial | 0.65 | 0.64 | 0.36 | 0 | 0.37 | 0.61 | 3.7 | | AvgConnSen_addition | 0.65 | 0.88 | 0.88 | 0 | 0.12 | 1.21 | 0.39 | | AvgConnSen_order | 0.64 | 0.44 | 0.48 | 0 | 0.92 | 1.41 | 0 | | AvgInferenceDistChain | 0.64 | 0.8 | 0.91 | 0 | 0.7 | 0.83 | 0 | | WdPolysemyCnt | 0.62 | 0.27 | 0.93 | 0 | 0.37 | 1.61 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.61 | 0.83 | 0.71 | 0 | 0.07 | 1.07 | 0.97 | | AvgRhythmUnits | 0.61 | 0.3 | 1.24 | 0 | 0.32 | 1.3 | 0 | | AvgDepsSen_aux | 0.57 | 0.03 | 1.26 | 0 | 0.38 | 1.32 | 0 | | SynSoph | 0.57 | 0.59 | 0.85 | 0 | 0.08 | 1.02 | 0.97 | | AvgDepsSen_cop | 0.55 | 0.87 | 0.48 | 0 | 0.05 | 1.25 | 0 | | AvgRhythmUnitStreesSyll | 0.52 | 0.76 | 1.17 | 0 | 0.14 | 0.56 | 0 | | AvgDepsSen_advmod | 0.48 | 0.33 | 0.6 | 0 | 0.2 | 1.26 | 0 | | AvgDepsSen_det | 0.48 | 0.22 | 1.04 | 0 | 0.45 | 0.68 | 0.39 | | AggPronSen_third_person | 0.47 | 0.86 | 0.8 | 0 | 0.08 | 0.58 | 0 | | AvgAOADoc_Bristol | 0.45 | 0.36 | 0.71 | 0 | 0.12 | 1.02 | 0.19 | | AvgDepsSen_acl | 0.44 | 1.28 | 0.29 | 0 | 0.16 | 0.36 | 0 | | AvgAOADoc_Bird | 0.44 | 0.38 | 0.84 | 0 | 0.13 | 0.89 | 0 | | WdAvgDpthHypernymTree | 0.43 | 0.79 | 0.71 | 0 | 0.06 | 0.54 | 0 | | RdbltyFlesch | 0.43 | 0.42 | 1.35 | 0 | 0.03 | 0.44 | 0 | | AvgDepsSen_dep | 0.42 | 0.68 | 0.6 | 0 | 0.02 | 0.75 | 0 | | AggPronSen_indefinite | 0.41 | 0.34 | 0.51 | 0 | 0.14 | 1.05 | 0 | | AvgConnSen_semi_coordinators | 0.39 | 0 | 1.01 | 0 | 0.13 | 0.92 | 0 | | AvgDepsSen_mwe | 0.38 | 0.6 | 1.17 | 0 | 0.1 | 0.07 | 0 | | AvgDepsSen_advcl | 0.38 | 0.06 | 0.44 | 0 | 0.01 | 1.4 | 0 | | AvgDepsSen_neg | 0.37 | 0.51 | 0.97 | 0 | 0.4 | 0.05 | 0 | | WdPathCntHypernymTree | 0.36 | 0.89 | 0.33 | 0 | 0.19 | 0.35 | 0 | | AvgAOESen_IndexAboveThreshold.0.3. | 0.35 | 0.27 | 0 | 0 | 0.41 | 1.05 | 0 | | AvgAOASen_Bird | 0.33 | 0.04 | 0.75 | 0 | 0.59 | 0.42 | 0 | | LxcSoph | 0.31 | 0.02 | 0.61 | 0 | 0.16 | 0.18 | 1.95 | | AvgConnSen_oppositions | 0.26 | 0.11 | 0.98 | 0 | 0.36 | 0 | 0 | | LangRhythmDiameter | 0.24 | 0.3 | 0.84 | 0 | 0.12 | 0.03 | 0 | | AvgConnSen_temporal_connectors | 0.17 | 0.23 | 0.58 | 0 | 0.09 | 0.01 | 0 | | LangRhythmId | 0.09 | 0.22 | 0.23 | 0 | 0.02 | 0.01 | 0 |
ReaderBench Model 3 {#readerbench-model-3}
General Description
ReaderBench Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below.
The models were trained on ReaderBench scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons).
Highly correlated ReaderBench metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
More details on the sample will be provided once peer review is complete on the main study using this model.
ReaderBench Model 3narr
This model was trained on 15-minute narrative writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression gbm = stochastic gradient boosted trees svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------|:-------| | 0.0000 | 0.1419 | 0.0945 | 0.3143 | 0.0729 | 0.0816 | 0.1792 | 0.1538 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |rf |mars |gbm |svm |enet|cube| |-------------------------------------------------|-------|----|----|-----|-----|----|----|----| |Content.words |13.76 |2 |2.22|32.41|16.24|2.38|4.95|8.73| |RB.AvgWdLen |7.5 |0.81|1.06|21.05|1.14 |1.01|1.38|3.55| |RB.AvgDepsBl_compound |4.66 |0.89|1.01|11.15|2.17 |0.45|2.45|3.09| |RB.WdEnt |4.59 |1.7 |2.06|7.03 |4.52 |1.75|4.27|5.73| |RB.LangRhythmId |3.17 |0.69|0.35|8.26 |0.06 |0.21|2.85|0.18| |RB.RdbltyDaleChall |3.15 |1.27|1.24|4.63 |2.86 |0.95|4.05|3.27| |RB.AvgUnqWdBl |2.75 |1.63|1.92|4.02 |0.46 |1.82|0 |6.45| |RB.LxcDiv |2.67 |1.87|1.89|0 |11.82|2.04|3.58|4.27| |Sentences |2.62 |1.71|2.13|0 |5.6 |1.52|4.67|5.91| |RB.TCorefChainDoc |2.35 |1.99|1.91|0 |5.33 |1.97|5.29|3.09| |RB.AvgAOADoc_Cortese |1.8 |0.21|0.37|3.53 |0.76 |0.8 |2.06|1.36| |RB.CAF |1.78 |1.83|1.52|0 |1.84 |1.87|3.65|3.27| |RB.AvgNounNmdEntBl |1.6 |0.53|0.45|4.63 |0.21 |0.19|0 |0.36| |RB.AvgDepsBl_nsubjpass |1.42 |0.81|0.27|3.29 |0.2 |0.41|1.23|0.18| |RB.AvgDepsBl_aux |1.21 |1.48|1.75|0 |2.12 |1.53|1.3 |2.36| |RB.TActCorefChainWd |1.06 |0.33|0.95|0 |1.19 |1.17|2.65|2 | |RB.AvgDepsBl_nsubj |1.05 |1.67|1.78|0 |2.94 |1.85|0 |2.09| |RB.AvgPronounBl |0.99 |1.72|1.84|0 |3.59 |2.06|0 |1.18| |RB.AvgUnqNoundBl |0.93 |0.71|0.59|0 |0.37 |0.65|2.75|1.55| |RB.TCorefChainBigSpan |0.89 |1.61|1.19|0 |0.16 |1.33|2.58|0 | |RB.AvgAOESen_InflectionPointPolynomial |0.87 |0.97|1.18|0 |0.57 |1.09|2.33|0.73| |RB.AvgBlScore |0.83 |1.46|1.28|0 |0.92 |1.62|0 |2.18| |RB.AvgConnBl_addition |0.81 |0.99|0.83|0 |0.77 |0.66|1.36|1.73| |RB.AvgChainSpan |0.81 |1.52|1.28|0 |2.39 |1.7 |0 |1.27| |RB.AvgPrepositionBl |0.79 |1.51|1.03|0 |0.76 |1.6 |0.95|1 | |RB.AvgUnqPrepositionBl |0.76 |1.48|1.09|0 |0.48 |1.55|0.8 |1.09| |RB.SenStdDevWd |0.74 |0.86|1.25|0 |1.41 |1.28|1.45|0.36| |RB.AvgAOADoc_Shock |0.72 |0.86|0.94|0 |1.36 |1.15|0.8 |1.27| |RB.AvgDepsBl_punct |0.68 |1.26|1.34|0 |1 |1 |0.35|1.18| |RB.AvgCorefChain |0.68 |1.19|1.03|0 |0.34 |1.28|1.1 |0.73| |RB.AvgNmdEntSen |0.67 |0.27|0.42|0 |0.17 |0.72|2.16|1 | |RB.AvgPronBl_indefinite |0.65 |1.4 |1.66|0 |1.98 |1.31|0.13|0.27| |RB.AvgDepsBl_det |0.65 |1.18|0.96|0 |0.43 |0.97|0.92|0.91| |RB.AvgDepsBl_dobj |0.65 |1.4 |0.87|0 |0.37 |1.26|0 |1.73| |RB.SynDiv |0.6 |0.64|0.7 |0 |0.42 |0.71|1.55|0.64| |RB.AvgAOEBl_InflectionPointPolynomial |0.6 |0.88|0.95|0 |2.01 |1.2 |0 |1.09| |RB.FrqRhythmId |0.59 |1.12|1.27|0 |0.11 |0.77|1.31|0.18| |RB.LangRhythmDiameter |0.58 |0.18|0.35|0 |0.17 |0.01|2.28|0.82| |RB.AvgDepsBl_expl |0.58 |0.64|0.28|0 |0.35 |0.2 |1.79|0.82| |RB.CharEnt |0.57 |1.37|1.01|0 |0.53 |1.35|0.86|0 | |RB.AvgNounSen |0.55 |0.74|0.91|0 |0.31 |0.46|1.46|0.36| |RB.AvgDepsBl_amod |0.55 |0.92|0.33|0 |0.1 |0.55|1.08|1.09| |RB.AvgUnqVerbBl |0.54 |1.56|1.01|0 |0.48 |1.48|0.16|0.36| |RB.AvgUnqPronounBl |0.54 |1.6 |0.94|0 |1.5 |1.7 |0 |0 | |RB.AvgPronBl_first_person |0.53 |1.35|0.7 |0 |0.34 |1.23|0.61|0.36| |RB.AvgConnBl_sentence_linking |0.53 |1.45|1.07|0 |0.41 |1.41|0 |0.64| |RB.LxcSoph |0.52 |0.32|0.77|0 |0.46 |0.6 |0.1 |2.09| |RB.AvgAOEBl_IndexPolynomialFitAboveThreshold.0.3.|0.5 |0.94|1.01|0 |0.41 |1.09|0.73|0.27| |RB.AvgRhythmUnitStreesSyll |0.49 |0.65|0.67|0 |0.45 |0.43|0.75|1 | |RB.AvgDepsBl_mark |0.47 |1.36|0.95|0 |0.08 |1.19|0 |0.64| |RB.AvgDepsBl_nmod |0.47 |1.27|0.79|0 |0.43 |1.1 |0 |0.73| |RB.WdDiffLemmaStem |0.45 |0.64|0.88|0 |0.65 |1.02|0.74|0.18| |RB.AvgDepsBl_conj |0.45 |0.95|0.52|0 |0.28 |0.64|0.41|0.91| |RB.AvgAOABl_Bird |0.44 |0.56|0.39|0 |0.58 |0.57|0.96|0.55| |RB.AvgPronBl_third_person |0.43 |1.36|1.14|0 |0.33 |1.26|0 |0.09| |RB.AvgDepsBl_ccomp |0.43 |0.93|0.86|0 |0.04 |0.47|1.06|0 | |RB.AggPronSen_third_person |0.43 |0.52|0.51|0 |0.11 |1.01|1.15|0.18| |RB.AvgDepsSen_punct |0.43 |0.38|0.62|0 |0.19 |0.94|0.81|0.64| |RB.AvgConnBl_simple_subordinators |0.42 |1.31|1.02|0 |0.73 |1.08|0 |0.09| |RB.AvgConnSen_simple_subordinators |0.41 |0.15|0.52|0 |0.09 |0.52|1.54|0.18| |RB.AvgSenBlCoh_LDA |0.4 |0.74|0.95|0 |0.2 |1.21|0 |0.73| |RB.AvgDepsBl_xcomp |0.4 |1.15|0.82|0 |0.37 |0.88|0.45|0 | |RB.AvgCommaBl |0.4 |0.72|0.45|0 |0.05 |0.39|0.96|0.45| |RB.AvgAOASen_Shock |0.39 |0.4 |0.74|0 |0.45 |0.9 |0.79|0.18| |RB.AvgSenBlCoh_word2vec |0.36 |1.11|0.78|0 |0.18 |1.03|0 |0.27| |RB.WdLettStdDev |0.34 |0.72|0.63|0 |0.39 |0.7 |0.46|0.18| |RB.AvgConnBl_temporal_connectors |0.34 |1.03|0.92|0 |0.02 |0.77|0.1 |0.27| |RB.AvgDepsBl_acl |0.34 |0.58|0.44|0 |0.07 |0.2 |1.19|0 | |RB.LangRhythmCoeff |0.33 |0.7 |0.59|0 |0.25 |0.66|0.61|0 | |RB.WdSylCnt |0.33 |0.38|0.9 |0 |0.27 |0.79|0.26|0.45| |RB.AvgDepsBl_auxpass |0.33 |0.86|0.55|0 |0.01 |0.5 |0.7 |0 | |RB.AvgConnBl_oppositions |0.33 |0.85|0.49|0 |0 |0.42|0.63|0.18| |RB.AvgAdverbBl |0.32 |1.1 |0.72|0 |0.11 |0.83|0 |0.18| |RB.AvgConnBl_order |0.32 |0.73|0.27|0 |0.01 |0.29|1 |0 | |RB.AvgAOABl_Bristol |0.31 |0.75|0.29|0 |0.7 |0.77|0.39|0 | |RB.AvgDepsSen_nmod |0.31 |0.06|0.43|0 |0.11 |0.34|0.45|1 | |RB.AvgPronounSen |0.31 |0.36|0.7 |0 |0.05 |0.55|0 |1 | |RB.AvgIntraBlCoh_Path |0.3 |1.14|0.3 |0 |0.16 |0.97|0 |0.18| |RB.AvgAOABl_Kuperman |0.3 |0.51|0.51|0 |0.71 |0.59|0.1 |0.45| |RB.AvgDepsSen_nsubj |0.3 |0.05|0.83|0 |0.04 |0.49|0 |1.18| |RB.AvgDepsSen_aux |0.3 |0.17|0.62|0 |0.21 |0.49|0.68|0.36| |RB.AvgInferenceDistChain |0.29 |0.8 |0.71|0 |0.19 |0.74|0.25|0 | |RB.AvgConnBl_conditions |0.29 |0.9 |0.45|0 |0.15 |0.49|0.44|0 | |RB.AvgDepsBl_cop |0.28 |1.07|0.53|0 |0.08 |0.7 |0 |0.18| |RB.RdbltyFlesch |0.28 |0.49|0.88|0 |0.38 |0.54|0 |0.45| |RB.AvgConnSen_temporal_connectors |0.28 |0.28|0.82|0 |0.17 |0.05|0.75|0.18| |RB.AvgUnqAdjectiveBl |0.27 |1.2 |0.24|0 |0.01 |0.97|0.03|0 | |RB.AvgDepsBl_advcl |0.27 |1.18|0.37|0 |0.08 |0.9 |0.01|0 | |RB.AvgDepsSen_advcl |0.27 |0.16|0.53|0 |0.12 |0.69|0.66|0.18| |RB.AggPronSen_indefinite |0.25 |0.42|0.84|0 |0.26 |1.17|0.03|0 | |RB.WdDiffWdStem |0.25 |0.65|0.56|0 |0.42 |0.81|0.13|0 | |RB.AvgDepsBl_neg |0.24 |0.45|0.08|0 |0.02 |0.12|0.9 |0 | |RB.AvgDepsBl_nummod |0.23 |0.45|0.11|0 |0 |0.12|0.89|0 | |RB.AvgDepsBl_mwe |0.22 |0.29|0.46|0 |0 |0.06|0.76|0 | |RB.AvgDepsSen_amod |0.22 |0.3 |0.64|0 |0.32 |0.72|0.22|0 | |RB.AvgAOASen_Bird |0.21 |0.32|0.74|0 |0.27 |0.42|0.25|0 | |RB.AvgPrepositionSen |0.21 |0.07|0.66|0 |0.05 |0.35|0 |0.73| |RB.AvgConnBl_contrasts |0.21 |1.04|0.19|0 |0.01 |0.64|0 |0 | |RB.AvgAOASen_Kuperman |0.21 |0.5 |0.2 |0 |0.88 |0.48|0 |0.18| |RB.AvgDepsSen_xcomp |0.21 |0.19|0.71|0 |0.05 |1.01|0.23|0 | |RB.AvgDepsBl_root |0.2 |0.04|0.29|0 |0.02 |0 |0.99|0 | |RB.AvgDepsSen_cop |0.2 |0.06|0.49|0 |0.28 |0.36|0.6 |0 | |RB.AvgConnSen_reason_and_purpose |0.19 |0.14|0.21|0 |0.11 |0.61|0.53|0 | |RB.AvgDepsSen_conj |0.19 |0.14|0.56|0 |0.02 |0.42|0 |0.55| |RB.AvgDepsSen_dobj |0.19 |0.06|0.6 |0 |0.23 |0.38|0 |0.55| |RB.AvgDepsSen_dep |0.19 |0.49|0.57|0 |0.29 |0.65|0 |0 | |RB.AvgAdverbSen |0.19 |0 |0.72|0 |0.05 |0.87|0 |0.36| |RB.AvgSenLen |0.18 |0.06|0.76|0 |0.12 |0.29|0 |0.45| |RB.AvgPronBl_second_person |0.18 |0.7 |0.62|0 |0.01 |0.3 |0 |0 | |RB.AvgConnBl_disjunctions |0.18 |0.73|0.25|0 |0.01 |0.35|0.16|0 | |RB.AvgConnBl_reason_and_purpose |0.18 |0.64|0.26|0 |0.03 |0.26|0.28|0 | |RB.AggPronSen_second_person |0.18 |0.32|0.47|0 |0.01 |0.08|0.53|0 | |RB.AvgConnBl_semi_coordinators |0.16 |0.35|0.31|0 |0.01 |0.09|0.43|0 | |RB.AvgPronBl_interrogative |0.16 |0.7 |0.35|0 |0.01 |0.28|0.07|0 | |RB.AvgAOASen_Bristol |0.15 |0.41|0.39|0 |0.14 |0.58|0 |0 | |RB.AvgDepsSen_ccomp |0.15 |0.18|0.59|0 |0.11 |0.47|0 |0.18| |RB.AvgDepsBl_iobj |0.15 |0.62|0.33|0 |0.01 |0.29|0 |0.09| |RB.AvgDepsSen_det |0.15 |0.21|0.32|0 |0.24 |0.07|0.12|0.36| |RB.AvgConnSen_addition |0.13 |0.11|0.27|0 |0.75 |0.46|0 |0 | |RB.AvgDepsSen_acl |0.13 |0.38|0.58|0 |0.11 |0.08|0 |0.09| |RB.AvgDepsSen_mark |0.12 |0.09|0.35|0 |0.04 |0.36|0 |0.27| |RB.AvgConnSen_oppositions |0.12 |0.25|0.65|0 |0.08 |0.05|0.13|0 | |RB.AvgDepsBl_dep |0.11 |0.58|0.04|0 |0.06 |0.19|0.04|0 | |RB.AvgConnSen_semi_coordinators |0.11 |0.22|0.65|0 |0.23 |0.04|0 |0 | |RB.AvgConnBl_complex_subordinators |0.11 |0.39|0.19|0 |0 |0.12|0.19|0 | |RB.AvgAOASen_Cortese |0.11 |0.12|0.25|0 |0.35 |0.64|0 |0 | |RB.AvgAdjectiveSen |0.1 |0.09|0.4 |0 |0.05 |0.56|0 |0 | |RB.AvgDepsSen_iobj |0.07 |0.16|0.45|0 |0.02 |0.02|0 |0 | |RB.AggPronSen_interrogative |0.07 |0.11|0.4 |0 |0.21 |0.01|0 |0 | |RB.AvgConnSen_order |0.07 |0.02|0.48|0 |0.13 |0 |0 |0.09| |RB.SenAsson |0.07 |0.24|0.41|0 |0 |0.02|0 |0 | |RB.AvgConnSen_conditions |0.06 |0.07|0.49|0 |0.08 |0 |0 |0 | |RB.AvgDepsBl_csubj |0.06 |0.02|0.31|0 |0.06 |0 |0.17|0 | |RB.AvgDepsSen_neg |0.06 |0.17|0.4 |0 |0.03 |0.03|0 |0 | |RB.AvgDepsBl_parataxis |0.05 |0.24|0.12|0 |0 |0.04|0 |0 | |RB.AvgDepsBl_appos |0.04 |0.2 |0.08|0 |0 |0.04|0 |0 | |RB.AvgDepsSen_nummod |0.04 |0.04|0.35|0 |0.02 |0 |0 |0 | |RB.AggPronSen_first_person |0.04 |0.02|0.2 |0 |0.14 |0.1 |0 |0 | |RB.AvgConnSen_disjunctions |0.03 |0.11|0.13|0 |0.04 |0.01|0 |0 | |RB.SenAllit |0.03 |0.03|0.3 |0 |0 |0 |0 |0 | |RB.AvgDepsSen_mwe |0.01 |0.07|0 |0 |0 |0.01|0 |0 |
ReaderBench Model 3exp
This model was trained on 15 min expository writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression gbm = stochastic gradient boosted trees svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | rf | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -0.0156 | 0.0826 | 0.3112 | 0.0319 | 0.1360 | 0.3306 | 0.1259 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|rf |mars |gbm |svm |enet |cube | |--------------------------------------|-------|----|-----|-----|----|-----|-----| |Content.words |20.83 |5.13|35.84|48.17|3.43|17.49|14.66| |RB.AvgWdLen |4.5 |1.29|10.64|1.06 |0.42|1.6 |4.32 | |RB.AvgDepsBl_compound |4.11 |0.71|8.06 |0.36 |0.06|3.36 |3.92 | |RB.AvgConnBl_order |3.63 |0.6 |6.17 |0.15 |0.21|4.43 |1.81 | |RB.SenStdDevWd |3.56 |1.02|5.2 |0.29 |1.18|4.38 |2.41 | |RB.LangRhythmId |3.46 |0.96|10.64|0.29 |0.26|0 |0.7 | |RB.TCorefChainDoc |3.34 |1.93|0 |4.43 |2.48|7.01 |3.51 | |RB.WdEnt |3.21 |2.23|0 |1.79 |2.35|6.08 |5.52 | |RB.AggPronSen_first_person |2.93 |0.99|8.76 |0.77 |0.12|0 |1.1 | |Sentences |2.9 |1.37|0 |2.38 |1.47|6.99 |2.01 | |RB.AvgSenAdjCoh_Path |2.68 |1.05|0 |1.28 |1.09|5.93 |3.92 | |RB.CAF |2.51 |1.33|0 |1.24 |1.75|5.87 |1.81 | |RB.AvgPronBl_third_person |2.49 |0.76|7.52 |0.04 |0.79|0 |0.2 | |RB.AvgBlScore |2.27 |2.1 |4.33 |1.67 |2.38|0 |3.31 | |RB.AvgPronBl_second_person |2.03 |1.17|0 |0.64 |0.95|4.73 |2.01 | |RB.LangRhythmDiameter |1.92 |0.73|2.84 |0.28 |0.15|2.24 |1.91 | |RB.TActCorefChainWd |1.4 |0.76|0 |0.47 |1.04|2.97 |1.81 | |RB.TCorefChainBigSpan |1.31 |0.44|0 |1.17 |1.63|2.75 |1 | |RB.AvgUnqAdjectiveBl |1.06 |1.01|0 |0.23 |1.7 |1.95 |0.9 | |RB.WdDiffWdStem |0.99 |1.06|0 |1.8 |0.52|2.17 |0.6 | |RB.AvgDepsSen_nmod |0.94 |0.95|0 |0.52 |1.14|1.67 |1.2 | |RB.AvgDepsBl_expl |0.89 |0.79|0 |0.42 |0.42|1.86 |1.2 | |RB.RdbltyDaleChall |0.86 |1.25|0 |0.91 |0.78|1.01 |2.41 | |RB.AvgAOEBl_InflectionPointPolynomial |0.77 |0.72|0 |0.27 |0.7 |1.85 |0.1 | |RB.AvgConnBl_temporal_connectors |0.76 |0.91|0 |0.04 |0.5 |1.37 |1.41 | |RB.AvgPronBl_indefinite |0.75 |2.03|0 |5.56 |1.56|0 |1.61 | |RB.SynDiv |0.71 |0.61|0 |0.28 |1.04|1.39 |0.5 | |RB.LxcDiv |0.69 |1.51|0 |1.57 |2.14|0 |1.91 | |RB.AvgAOASen_Bristol |0.66 |0.56|0 |0.14 |0.31|1.56 |0.5 | |RB.AvgDepsBl_root |0.65 |0.09|0 |0.04 |0.04|1.95 |0 | |RB.AvgDepsBl_nsubj |0.62 |1.9 |0 |0.88 |2.19|0 |1.2 | |RB.AvgPronounBl |0.59 |1.54|0 |0.25 |1.94|0 |1.61 | |RB.AvgPrepositionBl |0.59 |1.37|0 |1.41 |2.07|0 |1.31 | |RB.AvgUnqNoundBl |0.49 |0.83|0 |0.41 |1.02|0 |2.21 | |RB.AvgDepsBl_parataxis |0.47 |0.53|0 |0.01 |0.15|1.26 |0 | |RB.LangRhythmCoeff |0.44 |0.58|0 |0.33 |0.4 |0.93 |0.2 | |RB.AvgUnqPrepositionBl |0.43 |0.94|0 |0.2 |2.05|0 |0.6 | |RB.AvgAOASen_Bird |0.43 |0.63|0 |0.4 |0.79|0.63 |0.5 | |RB.WdSylCnt |0.42 |0.96|0 |0.54 |0.18|0.5 |1.1 | |RB.AvgDepsBl_nmod |0.42 |1.01|0 |0.52 |1.59|0 |0.9 | |RB.AvgChainSpan |0.41 |1.04|0 |0.4 |1.58|0 |0.8 | |RB.AvgDepsBl_nummod |0.41 |0.7 |0 |0.01 |0.21|1 |0 | |RB.AvgDepsSen_expl |0.4 |0.41|0 |0.23 |0.06|1.1 |0 | |RB.AvgPronBl_first_person |0.39 |0.71|0 |0.51 |0.5 |0.25 |1.41 | |RB.AvgUnqVerbBl |0.38 |0.91|0 |0.06 |1.71|0 |0.6 | |RB.AvgDepsBl_aux |0.37 |0.59|0 |0.19 |0.93|0.4 |0.5 | |RB.AvgAdverbBl |0.33 |0.6 |0 |0.11 |1.31|0 |0.8 | |RB.AvgDepsBl_punct |0.33 |1.26|0 |0.3 |1.17|0 |0.5 | |RB.AvgNounSen |0.33 |0.99|0 |0.05 |0.22|0 |1.81 | |RB.LxcSoph |0.32 |0.79|0 |0.3 |0.75|0 |1.2 | |RB.CharEnt |0.31 |0.49|0 |1.05 |1.09|0.13 |0.4 | |RB.AvgDepsSen_cop |0.31 |0.86|0 |0.55 |0.55|0 |1.2 | |RB.AvgDepsBl_mark |0.31 |1.04|0 |0.56 |1.59|0 |0 | |RB.AvgSenBlCoh_LDA |0.3 |0.82|0 |0.16 |1.15|0 |0.6 | |RB.RdbltyFlesch |0.29 |0.47|0 |0.19 |0.17|0 |1.81 | |RB.AvgCorefChain |0.28 |0.76|0 |0.2 |1.05|0 |0.6 | |RB.AvgDepsBl_dobj |0.28 |0.92|0 |0.09 |1.36|0 |0.2 | |RB.AvgDepsBl_cop |0.27 |0.59|0 |0.07 |0.97|0 |0.7 | |RB.AvgDepsBl_det |0.27 |0.92|0 |0.09 |1.36|0 |0.1 | |RB.AvgDepsSen_mark |0.27 |0.68|0 |0.19 |1.12|0 |0.5 | |RB.AvgDepsBl_amod |0.26 |0.58|0 |0.27 |1.23|0 |0.3 | |RB.AvgDepsBl_mwe |0.25 |0.8 |0 |0.09 |0.61|0.3 |0 | |RB.AvgUnqAdverbBl |0.25 |0.6 |0 |0.03 |1.39|0 |0.1 | |RB.AvgPrepositionSen |0.24 |0.44|0 |0.16 |0.91|0 |0.6 | |RB.AvgConnBl_simple_subordinators |0.23 |0.76|0 |0.05 |1.22|0 |0 | |RB.AvgAOASen_Kuperman |0.23 |0.53|0 |0.51 |0.39|0.2 |0.4 | |RB.AvgDepsSen_compound |0.23 |1.22|0 |0.33 |0.33|0 |0.6 | |RB.AvgDepsBl_ccomp |0.22 |0.51|0 |0.05 |0.54|0.2 |0.3 | |RB.AvgUnqPronounBl |0.22 |0.46|0 |0 |1.33|0 |0 | |RB.FrqRhythmId |0.22 |0.94|0 |0.3 |0.68|0.06 |0.2 | |RB.AggPronSen_indefinite |0.22 |0.76|0 |0.37 |0.93|0 |0.2 | |RB.AvgDepsSen_dobj |0.21 |0.98|0 |0.1 |0.49|0 |0.5 | |RB.AggPronSen_second_person |0.2 |0.81|0 |0.23 |0.64|0 |0.3 | |RB.AvgAOADoc_Shock |0.2 |0.98|0 |0.42 |0.82|0 |0 | |RB.AvgConnSen_semi_coordinators |0.19 |0.59|0 |0.29 |0 |0.38 |0.1 | |RB.AvgConnBl_addition |0.18 |0.7 |0 |0.23 |0.65|0 |0.2 | |RB.AvgRhythmUnitStreesSyll |0.18 |0.89|0 |0.17 |0.47|0 |0.3 | |RB.AvgDepsSen_ccomp |0.18 |0.31|0 |0.22 |0.94|0 |0.2 | |RB.AvgAdverbSen |0.17 |0.38|0 |0.06 |0.99|0 |0 | |RB.AvgCommaSen |0.17 |0.62|0 |0.25 |0.8 |0 |0 | |RB.AvgAOEDoc_IndexAboveThreshold.0.3. |0.17 |0.72|0 |0.12 |0.36|0 |0.5 | |RB.AvgConnBl_contrasts |0.17 |0.46|0 |0.08 |0.82|0 |0.2 | |RB.AvgConnSen_simple_subordinators |0.16 |0.44|0 |0.13 |0.88|0 |0 | |RB.AvgConnBl_reason_and_purpose |0.16 |0.73|0 |0.14 |0.62|0 |0.1 | |RB.AvgAOADoc_Bird |0.16 |0.79|0 |0.14 |0.68|0 |0 | |RB.AvgDepsSen_amod |0.16 |0.29|0 |0.25 |0.5 |0 |0.5 | |RB.AvgConnBl_oppositions |0.16 |0.65|0 |0.05 |0.6 |0.02 |0.2 | |RB.AvgAOABl_Kuperman |0.15 |0.11|0 |0.18 |0.45|0 |0.6 | |RB.AvgDepsSen_xcomp |0.15 |0.63|0 |0.06 |0.73|0 |0 | |RB.AvgPronounSen |0.14 |0.62|0 |0.03 |0.26|0 |0.4 | |RB.AvgDepsBl_advcl |0.14 |0.21|0 |0.02 |0.89|0 |0 | |RB.AvgInferenceDistChain |0.14 |0.56|0 |0.2 |0.45|0 |0.2 | |RB.AvgNounNmdEntBl |0.14 |0.49|0 |0.87 |0.55|0 |0 | |RB.AggPronSen_third_person |0.14 |0.65|0 |0.14 |0.64|0 |0 | |RB.WdLettStdDev |0.14 |0.65|0 |0.18 |0.63|0 |0 | |RB.AvgConnSen_addition |0.13 |0.47|0 |0.23 |0.63|0 |0 | |RB.AvgNmdEntSen |0.13 |0.18|0 |0.36 |0.81|0 |0 | |RB.WdDiffLemmaStem |0.12 |0.71|0 |0.26 |0.29|0 |0.1 | |RB.AvgDepsSen_aux |0.12 |0.4 |0 |0.03 |0.64|0 |0 | |RB.AvgCommaBl |0.12 |0.66|0 |0.04 |0.4 |0 |0.1 | |RB.AvgAOASen_Shock |0.12 |0.28|0 |0.05 |0.73|0 |0 | |RB.AvgDepsBl_acl |0.12 |0.47|0 |0.13 |0.6 |0 |0 | |RB.AvgAOABl_Cortese |0.12 |0.28|0 |0.1 |0.64|0 |0.1 | |RB.AvgDepsSen_advcl |0.12 |0.46|0 |0.25 |0.59|0 |0 | |RB.AvgDepsBl_xcomp |0.12 |0.23|0 |0.09 |0.78|0 |0 | |RB.AvgConnSen_temporal_connectors |0.11 |0.73|0 |0.06 |0.09|0.06 |0.1 | |RB.AvgAOESen_InflectionPointPolynomial|0.11 |0.28|0 |0.11 |0.52|0 |0.1 | |RB.AvgDepsSen_dep |0.11 |0.49|0 |0.17 |0.38|0 |0.1 | |RB.AvgAOASen_Cortese |0.11 |0.22|0 |0.17 |0.66|0 |0 | |RB.AvgDepsSen_det |0.11 |0.14|0 |0.12 |0.54|0 |0.2 | |RB.AvgConnSen_reason_and_purpose |0.11 |0.39|0 |0.12 |0.58|0 |0 | |RB.AvgAOABl_Bristol |0.1 |0.45|0 |0.15 |0.37|0 |0.1 | |RB.AvgDepsBl_iobj |0.09 |0.74|0 |0.21 |0.17|0 |0 | |RB.AvgDepsSen_mwe |0.09 |0.64|0 |0.44 |0.21|0 |0 | |RB.AvgConnSen_order |0.08 |0.69|0 |0.67 |0.01|0 |0 | |RB.AvgConnSen_oppositions |0.08 |0.63|0 |0.11 |0.09|0 |0.1 | |RB.AvgConnBl_disjunctions |0.08 |0.5 |0 |0 |0.32|0 |0 | |RB.AvgConnSen_contrasts |0.07 |0.6 |0 |0.11 |0.11|0 |0 | |RB.AvgDepsBl_auxpass |0.07 |0.56|0 |0.01 |0.17|0 |0 | |RB.AvgDepsSen_neg |0.07 |0.47|0 |0.31 |0 |0 |0.2 | |RB.AvgConnBl_conditions |0.07 |0.49|0 |0.11 |0.23|0 |0 | |RB.AvgDepsBl_neg |0.06 |0.19|0 |0.03 |0.23|0 |0.1 | |RB.AvgPronBl_interrogative |0.06 |0.54|0 |0.04 |0.14|0 |0 | |RB.SenAsson |0.05 |0.25|0 |0 |0.23|0 |0 | |RB.AvgConnSen_disjunctions |0.05 |0.55|0 |0.03 |0.05|0 |0 | |RB.AvgConnBl_semi_coordinators |0.04 |0.16|0 |0.1 |0.16|0 |0 | |RB.AvgDepsSen_nummod |0.04 |0.46|0 |0.03 |0 |0 |0 | |RB.AvgDepsSen_acl |0.04 |0.46|0 |0.01 |0.01|0 |0 | |RB.AvgDepsBl_csubj |0.04 |0.4 |0 |0.01 |0.05|0 |0 | |RB.AvgDepsBl_nsubjpass |0.04 |0.25|0 |0 |0.16|0 |0 | |RB.AvgDepsBl_appos |0.04 |0.51|0 |0 |0.02|0 |0 | |RB.AvgDepsBl_dep |0.02 |0 |0 |0.08 |0.15|0 |0 | |RB.SenAllit |0.02 |0.3 |0 |0 |0 |0 |0 |
ReaderBench Model 3per
This modelwas trained on 15 min persuasive writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -0.0141 | 0.0326 | 0.2043 | 0.2331 | 0.1507 | 0.3202 | 0.0801 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |mars |gbm |svm |enet |cube| |--------------------------------------------------|-------|----|-----|-----|----|-----|----| |RB.WdEnt |9.44 |1.97|0 |16.45|2.58|14.62|8.38| |RB.AvgPrepositionBl |8.44 |1.96|20.57|11.55|2.48|2.8 |4.83| |Sentences |6.71 |1.67|19.13|4.09 |1.41|4.14 |5.01| |RB.AvgBlScore |5.39 |2 |0 |15.75|2.73|2.76 |5.92| |RB.CAF |4.59 |1.59|19.13|1.27 |1.43|0 |2.73| |RB.AvgSenScore |3.72 |0.51|8.2 |0.15 |0.47|5.74 |2 | |RB.TCorefChainDoc |3.36 |1.97|0 |5.74 |2.16|4.69 |2.46| |RB.AvgWdLen |2.49 |1.32|0 |4.98 |1.13|2.84 |3.28| |RB.AvgAOADoc_Shock |2.39 |1.4 |8.2 |1.65 |1.21|0.2 |1.18| |RB.AvgPronBl_indefinite |2.34 |1.76|0 |3.91 |2.01|2.57 |3.73| |RB.RdbltyDaleChall |2.32 |0.79|0 |1.32 |0.68|5.11 |3.73| |RB.AvgDepsBl_compound |2.28 |0.23|7.3 |0.29 |0.02|1.88 |2 | |RB.AvgUnqNoundBl |2.11 |0.75|1.47 |0.3 |1.41|4.17 |2.64| |RB.AvgConnBl_simple_subordinators |1.75 |1.76|0 |3.08 |1.98|2.11 |0.46| |RB.AvgAOESen_InflectionPointPolynomial |1.61 |0.53|5.45 |0.22 |0.77|0.99 |0.36| |RB.AvgPronBl_interrogative |1.23 |0.61|0 |0.53 |0.13|2.92 |2 | |RB.AvgDepsBl_nsubj |1.12 |1.87|0 |1.87 |2.47|0 |3.37| |RB.AvgDepsBl_mark |1.1 |1.63|0 |0.89 |1.7 |1.41 |1.91| |RB.AvgNmdEntSen |1.07 |0.07|0 |0.25 |0.41|2.79 |0.91| |RB.AvgDepsBl_amod |1.06 |0.82|0 |0.33 |0.41|2.72 |0.64| |RB.AvgCorefChain |1.04 |0.95|0 |0.08 |1.1 |2.38 |1 | |RB.AvgDepsSen_advmod |1.01 |0.09|0 |0.22 |0.22|2.68 |1 | |RB.AvgPronBl_first_person |0.96 |0.7 |2.97 |0.06 |0.27|0.75 |0.64| |RB.LangRhythmCoeff |0.95 |0.77|0 |1.43 |0.81|1.35 |0.73| |RB.AvgAOABl_Bird |0.93 |0.47|3.59 |0.46 |0.63|0 |0 | |RB.AvgDepsSen_aux |0.92 |0 |4 |0.21 |0.18|0 |0.55| |RB.AvgSenAdjCoh_Path |0.87 |1.19|0 |2.13 |1.26|0.33 |0.73| |RB.AvgDepsBl_det |0.83 |1.51|0 |0.54 |1.48|1.21 |0.73| |RB.AvgConnSen_oppositions |0.8 |0.24|0 |0.41 |0.01|2.1 |0.46| |RB.AvgAOASen_Shock |0.8 |0.62|0 |0.13 |1.02|1.67 |0.91| |RB.LxcDiv |0.8 |1.45|0 |1.85 |1.4 |0 |1.55| |RB.AvgUnqPronounBl |0.77 |1.68|0 |0.64 |1.73|0.63 |1.46| |RB.AvgAOADoc_Cortese |0.69 |0.02|0 |0.3 |0.59|1.5 |0.73| |RB.AvgUnqAdjectiveBl |0.69 |1.13|0 |0.03 |0.81|1.58 |0.36| |RB.AvgDepsBl_nsubjpass |0.69 |0.48|0 |0.02 |0.16|2.05 |0.09| |RB.AvgAOASen_Bird |0.64 |0.52|0 |0.31 |0.39|1.46 |0.46| |RB.AvgDepsBl_cop |0.63 |1.09|0 |0.1 |0.8 |1.3 |0.55| |RB.TCorefChainBigSpan |0.6 |1.53|0 |0.34 |1.38|0.62 |1 | |RB.AvgChainSpan |0.59 |1.42|0 |1 |1.73|0 |0.73| |RB.AvgDepsBl_aux |0.59 |1.42|0 |0.59 |1.36|0.5 |0.64| |RB.AvgUnqPrepositionBl |0.58 |1.83|0 |0.66 |2.15|0 |0.64| |RB.AggPronSen_second_person |0.54 |0.32|0 |0.04 |0.55|1.28 |0.46| |RB.SynDiv |0.51 |1.15|0 |0.48 |1.14|0.52 |0.46| |RB.CharEnt |0.49 |1.19|0 |1.21 |1.21|0 |0 | |RB.AvgAOASen_Bristol |0.48 |0.35|0 |0.13 |0.36|1.1 |0.46| |RB.AvgDepsBl_punct |0.47 |1.51|0 |0.72 |1.39|0 |0.64| |RB.AvgDepsBl_nmod |0.46 |1.6 |0 |0.48 |1.72|0 |0.55| |RB.AvgUnqVerbBl |0.43 |1.51|0 |0.41 |1.47|0 |0.91| |RB.WdDiffLemmaStem |0.42 |0.86|0 |0.43 |0.9 |0.48 |0.18| |RB.AvgDepsSen_mark |0.42 |0.28|0 |0.1 |0.29|0.68 |1.73| |RB.WdDiffWdStem |0.42 |0.67|0 |0.31 |0.66|0.71 |0.18| |RB.AvgPronounBl |0.41 |1.67|0 |0.06 |1.67|0 |1.18| |RB.AvgAOASen_Cortese |0.41 |0.08|0 |0.15 |0.3 |0.88 |0.73| |RB.AvgConnBl_temporal_connectors |0.41 |0.71|0 |0.02 |0.38|1.05 |0 | |RB.AvgRhythmUnitStreesSyll |0.38 |0.09|0 |0.12 |0.17|0.87 |0.73| |RB.LxcSoph |0.37 |0.75|0 |0.68 |0.65|0 |1.18| |RB.AvgDepsBl_ccomp |0.34 |1.38|0 |0.08 |1.26|0.12 |0.64| |RB.AvgDepsSen_neg |0.34 |0.23|0 |0.09 |0.52|0.74 |0 | |RB.AvgPronBl_third_person |0.34 |1.34|0 |0.39 |1.16|0 |0.46| |RB.AvgDepsBl_root |0.33 |0.09|0 |0.06 |0 |1 |0 | |RB.TActCorefChainWd |0.33 |0.36|0 |0.36 |0.81|0.14 |0.91| |RB.WdSylCnt |0.3 |0.76|0 |0.54 |0.7 |0 |0.64| |RB.AvgUnqAdverbBl |0.29 |1.34|0 |0.09 |1.2 |0 |0.64| |RB.AvgDepsSen_punct |0.27 |0.44|0 |0.09 |0.16|0.68 |0 | |RB.AvgNmdEntBl |0.26 |1.25|0 |0.11 |1.05|0 |0.55| |RB.AvgConnBl_addition |0.25 |1.13|0 |0.16 |0.9 |0 |0.55| |RB.AvgDepsSen_compound |0.25 |0.5 |0 |0.19 |0.56|0 |1.37| |RB.AggPronSen_indefinite |0.25 |0.42|0 |0.24 |0.9 |0 |0.64| |RB.AvgDepsBl_dobj |0.25 |1.37|0 |0.02 |1.15|0 |0.46| |RB.AvgConnBl_order |0.24 |0.56|0 |0.03 |0.21|0.59 |0 | |RB.AvgAOADoc_Bristol |0.24 |0.8 |0 |0.28 |0.89|0 |0.27| |RB.SenStdDevWd |0.24 |0.98|0 |0.18 |1.06|0 |0.18| |RB.FrqRhythmId |0.23 |1.1 |0 |0.02 |0.72|0.21 |0.18| |RB.AvgDepsBl_advmod |0.23 |1.21|0 |0.12 |0.96|0 |0.27| |RB.AvgDepsBl_advcl |0.23 |1.36|0 |0.01 |1.26|0 |0 | |RB.AvgAdverbBl |0.23 |1.25|0 |0.1 |1.01|0 |0.27| |RB.AvgConnBl_logical_connectors |0.22 |1.14|0 |0.2 |0.89|0 |0.09| |RB.AvgConnBl_semi_coordinators |0.21 |0.2 |0 |0.02 |0.03|0.56 |0.18| |RB.AvgPronounSen |0.21 |0.33|0 |0.15 |0.72|0 |0.73| |RB.AvgUnqNmdEntBl |0.21 |1 |0 |0.18 |0.65|0 |0.55| |RB.AvgConnSen_simple_subordinators |0.2 |0.46|0 |0.17 |0.74|0 |0.46| |RB.AvgSenBlCoh_LDA |0.2 |0.59|0 |0.06 |0.91|0.01 |0.36| |RB.AvgConnBl_reason_and_purpose |0.2 |1.2 |0 |0.09 |0.96|0 |0 | |RB.AvgDepsSen_amod |0.2 |0.18|0 |0.18 |0.62|0 |0.82| |RB.AvgInferenceDistChain |0.19 |0.33|0 |0.23 |0.81|0 |0.09| |RB.AvgAOESen_IndexPolynomialFitAboveThreshold.0.3.|0.19 |0.68|0 |0.32 |0.65|0 |0 | |RB.AvgSenBlCoh_LSA |0.19 |0.97|0 |0.09 |0.86|0 |0.18| |RB.AvgAOEDoc_InverseAverage |0.18 |0.62|0 |0.17 |0.82|0 |0 | |RB.AvgAOEBl_IndexAboveThreshold.0.3. |0.18 |0.59|0 |0.17 |0.71|0.06 |0 | |RB.SenAllit |0.18 |0.53|0 |0.01 |0.19|0.43 |0 | |RB.AvgDepsSen_dep |0.18 |0.19|0 |0.2 |0.42|0 |0.91| |RB.AvgDepsBl_nummod |0.17 |0.66|0 |0.11 |0.32|0.24 |0 | |RB.AvgDepsSen_det |0.16 |0.21|0 |0.08 |0.89|0 |0 | |RB.AvgDepsBl_conj |0.16 |1.02|0 |0.11 |0.63|0 |0.09| |RB.AvgDepsSen_ccomp |0.16 |0.3 |0 |0.15 |0.55|0 |0.46| |RB.AvgConnSen_addition |0.15 |0.01|0 |0.11 |0.52|0 |0.55| |RB.AvgDepsSen_acl |0.15 |0.15|0 |0.13 |0.01|0.36 |0 | |RB.AvgDepsBl_xcomp |0.14 |0.89|0 |0.04 |0.56|0 |0.18| |RB.AvgPronBl_second_person |0.14 |0.91|0 |0.05 |0.55|0 |0.18| |RB.AvgAOABl_Kuperman |0.14 |0.08|0 |0.23 |0.45|0 |0.18| |RB.AvgNounSen |0.14 |0.2 |0 |0.03 |0.22|0 |1.18| |RB.AvgConnBl_contrasts |0.14 |1.03|0 |0.05 |0.67|0 |0 | |RB.WdLettStdDev |0.13 |0.6 |0 |0.19 |0.39|0 |0.09| |RB.AvgDepsBl_neg |0.13 |0.3 |0 |0.03 |0.05|0.33 |0 | |RB.AvgDepsSen_xcomp |0.13 |0.06|0 |0.05 |0.58|0 |0.36| |RB.AvgDepsSen_advcl |0.13 |0.14|0 |0.13 |0.66|0 |0 | |RB.AvgConnBl_oppositions |0.13 |0.98|0 |0.03 |0.54|0 |0.18| |RB.AggPronSen_first_person |0.13 |0.06|0 |0.22 |0.56|0 |0 | |RB.AggPronSen_third_person |0.12 |0.38|0 |0.02 |0.69|0 |0 | |RB.AvgDepsSen_dobj |0.1 |0.07|0 |0.07 |0.31|0 |0.46| |RB.AvgAdjectiveSen |0.1 |0.04|0 |0.09 |0.4 |0 |0.27| |RB.AvgDepsSen_cop |0.1 |0.14|0 |0.04 |0.59|0 |0 | |RB.AvgConnSen_reason_and_purpose |0.09 |0.05|0 |0.06 |0.25|0 |0.46| |RB.AvgConnBl_conditions |0.09 |0.72|0 |0.04 |0.39|0 |0 | |RB.LangRhythmDiameter |0.09 |0.29|0 |0.13 |0.06|0.15 |0 | |RB.AvgDepsBl_acl |0.09 |0.74|0 |0.03 |0.39|0 |0.09| |RB.AvgAOASen_Kuperman |0.08 |0.09|0 |0.11 |0.26|0 |0.18| |RB.AvgConnBl_disjunctions |0.08 |0.52|0 |0.03 |0.21|0.08 |0 | |RB.AvgDepsSen_nmod |0.08 |0.02|0 |0.15 |0.17|0 |0.27| |RB.AvgCommaBl |0.08 |0.78|0 |0.02 |0.36|0 |0 | |RB.AvgDepsBl_mwe |0.07 |0.67|0 |0.01 |0.33|0 |0 | |RB.AvgDepsBl_dep |0.07 |0.64|0 |0.07 |0.22|0 |0.09| |RB.AvgConnSen_semi_coordinators |0.06 |0.13|0 |0.01 |0.01|0.11 |0.27| |RB.AvgConnSen_conditions |0.05 |0.01|0 |0.2 |0 |0 |0 | |RB.AvgConnBl_conjuncts |0.04 |0.42|0 |0.01 |0.14|0 |0 | |RB.LangRhythmId |0.04 |0.39|0 |0.04 |0.1 |0 |0 | |RB.AvgDepsBl_csubj |0.03 |0.39|0 |0 |0.09|0 |0 | |RB.AvgDepsBl_iobj |0.03 |0.29|0 |0.03 |0.09|0 |0 | |RB.AvgDepsSen_nummod |0.03 |0.13|0 |0.09 |0.01|0 |0 | |RB.AvgDepsBl_auxpass |0.03 |0.36|0 |0 |0.11|0 |0 | |RB.AvgDepsBl_expl |0.03 |0.29|0 |0 |0.09|0.03 |0 | |RB.SenAsson |0.03 |0.37|0 |0.02 |0.1 |0 |0 | |RB.AvgCommaSen |0.03 |0.14|0 |0.05 |0.02|0 |0.18| |RB.AvgDepsSen_csubj |0.01 |0.04|0 |0.02 |0 |0 |0 | |RB.AvgConnSen_disjunctions |0.01 |0.07|0 |0.03 |0.01|0 |0 | |RB.AvgDepsBl_parataxis |0.01 |0.2 |0 |0 |0.03|0 |0 | |RB.AvgConnBl_complex_subordinators |0 |0.06|0 |0 |0.01|0 |0 | |RB.AvgConnSen_temporal_connectors |0 |0.01|0 |0.01 |0 |0 |0 |
Coh-Metrix Model 1 {#cohmetrix-model-1}
General Description
Model 1 has been replaced by the greatly simplified Model 2. Model 2 is recommended for current use.
Coh-Metrix Model 1 is an ensemble (formed by averaging predicted quality scores) of six sub-models that are detailed below.
All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
This scoring model was evaluated in the following publications: [@Matta2022; @Keller-Margulis2021]
Coh-Metrix Model 1a
This model was trained on fall Coh-Metrix scores from data described in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:------|:-------|:------| | -10.8465 | 0.0266 | 0.1506 | 0.2663 | -0.0302 | 0.296 | 0.2609 | 0.136 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 19.42 | 42.06 | 5.86 | 4.76 | 26.22 | 7.84 | 45.19 | 26.59 | | DESWLlt | 6.35 | 2.43 | 2.98 | 1.91 | 5.72 | 2.26 | 13.18 | 13.19 | | LDMTLD | 4.88 | 8.17 | 4.01 | 3.11 | 6.4 | 4.13 | 4.19 | 10.99 | | PCCONNp | 4.75 | 0.03 | 0 | 0.53 | 1.42 | 0.59 | 18.28 | 0 | | PCNARp | 3.13 | 0 | 1.76 | 0.8 | 0 | 1.17 | 10.1 | 0 | | WRDHYPn | 2.82 | 3.15 | 2.9 | 1.83 | 7.62 | 2.16 | 0 | 10.99 | | PCVERBp | 2.35 | 0 | 0.99 | 0.6 | 0 | 1.23 | 7.35 | 0 | | DESPL | 1.57 | 0.67 | 3.22 | 1.72 | 2.63 | 1.65 | 1.53 | 0 | | SYNSTRUTa | 1.39 | 1.21 | 0.85 | 1.12 | 3.49 | 2.74 | 0 | 2.42 | | PCDCp | 1.28 | 1.12 | 2 | 1.08 | 0 | 2.72 | 0 | 0.88 | | DESWLsy | 1.26 | 0.48 | 2.09 | 1.23 | 1.3 | 1.34 | 0 | 3.08 | | CNCTempx | 1.25 | 1.24 | 0.89 | 1.99 | 1.97 | 1.73 | 0 | 1.54 | | LDTTRa | 1.25 | 0.48 | 2.01 | 0.89 | 2.69 | 1.65 | 0 | 3.08 | | WRDFRQa | 1.23 | 1.38 | 1.86 | 1.58 | 1.86 | 0.82 | 0 | 3.08 | | WRDVERB | 1.16 | 1.48 | 1.04 | 0.69 | 3.13 | 1.79 | 0 | 3.08 | | LSASSpd | 1.11 | 0.1 | 1.95 | 1.43 | 3.46 | 1.39 | 0 | 1.54 | | CNCTemp | 1.07 | 1.06 | 0.93 | 1.51 | 1.89 | 1.48 | 0 | 1.54 | | CNCADC | 1 | 0.88 | 1.12 | 1.89 | 0 | 1.61 | 0 | 0 | | SMINTEp | 0.99 | 1.86 | 0.75 | 1.33 | 3.22 | 1.3 | 0 | 1.54 | | SMCAUSwn | 0.99 | 0.82 | 1.4 | 1.72 | 0 | 1.64 | 0 | 0 | | DESWLsyd | 0.98 | 2.02 | 1.97 | 1.03 | 2.25 | 1.54 | 0 | 0.66 | | CRFCWO1d | 0.97 | 1.03 | 1.67 | 1.5 | 0 | 1.64 | 0 | 0 | | PCNARz | 0.95 | 0.74 | 1.29 | 0.88 | 3.68 | 1.5 | 0 | 1.54 | | WRDHYPnv | 0.94 | 0.1 | 1.9 | 0.97 | 0 | 1.23 | 0 | 1.54 | | WRDPRO | 0.94 | 1.26 | 1.77 | 1.28 | 0 | 1.68 | 0 | 0 | | DESSLd | 0.93 | 0.15 | 1.67 | 1.09 | 0 | 1.79 | 0.18 | 0 | | DESWLltd | 0.93 | 1.34 | 2.3 | 1.28 | 1.04 | 1.35 | 0 | 0 | | DRPP | 0.93 | 0.99 | 2.18 | 1.12 | 0.47 | 1.62 | 0 | 0 | | CNCLogic | 0.91 | 1.16 | 1.42 | 1.36 | 1.86 | 0.99 | 0 | 1.1 | | LSAGN | 0.86 | 0.52 | 2.55 | 1.44 | 0 | 0.94 | 0 | 0 | | PCCONNz | 0.83 | 1.35 | 1.09 | 1.09 | 0 | 1.21 | 0 | 0.88 | | CRFCWOad | 0.82 | 0.32 | 1.51 | 1.44 | 0 | 1.24 | 0 | 0 | | WRDADV | 0.81 | 0.07 | 1.3 | 1.24 | 0 | 1.51 | 0 | 0 | | RDFRE | 0.81 | 0.86 | 1.29 | 0.98 | 2.81 | 1.65 | 0 | 0 | | LSASS1d | 0.8 | 0.4 | 1.55 | 1.38 | 0.22 | 1.17 | 0 | 0 | | WRDFRQmc | 0.8 | 1.22 | 1.35 | 0.32 | 0 | 1.92 | 0 | 0.66 | | LDTTRc | 0.8 | 0.36 | 1.82 | 0.98 | 0.02 | 1.48 | 0 | 0 | | PCVERBz | 0.8 | 0.11 | 1.55 | 0.87 | 0 | 0.93 | 0 | 1.54 | | DRNP | 0.77 | 0.25 | 1.73 | 0.66 | 0 | 1.71 | 0 | 0 | | WRDCNCc | 0.72 | 1.35 | 1.07 | 0.64 | 3.32 | 0 | 0 | 3.08 | | CRFNOa | 0.72 | 0.4 | 0.74 | 1.33 | 0.63 | 1.25 | 0 | 0 | | CNCPos | 0.71 | 0.33 | 0.88 | 1.02 | 0.09 | 0.64 | 0 | 1.54 | | SYNMEDpos | 0.71 | 1.89 | 1.75 | 1.08 | 0 | 0.86 | 0 | 0 | | LSAGNd | 0.68 | 0.78 | 1.5 | 1.1 | 0.83 | 0.94 | 0 | 0 | | CNCCaus | 0.63 | 0.09 | 1.11 | 1.17 | 0 | 0.91 | 0 | 0 | | DRVP | 0.63 | 1.02 | 0.82 | 0.6 | 0.03 | 0.7 | 0 | 1.54 | | DRNEG | 0.63 | 0.4 | 0.76 | 1.13 | 0.03 | 1.09 | 0 | 0 | | CRFCWOa | 0.61 | 0.32 | 0.46 | 1.25 | 0 | 0.98 | 0 | 0 | | RDL2 | 0.61 | 0.3 | 0.31 | 0.87 | 0 | 1.45 | 0 | 0 | | WRDPOLc | 0.61 | 0.56 | 0.85 | 1.57 | 0 | 0.5 | 0 | 0 | | WRDIMGc | 0.61 | 0.19 | 0.14 | 0.74 | 0 | 0.87 | 0 | 1.54 | | SMCAUSr | 0.6 | 0.32 | 1.58 | 0.23 | 0.02 | 1.5 | 0 | 0 | | WRDFAMc | 0.6 | 1.32 | 1.09 | 0.86 | 1.25 | 0.96 | 0 | 0 | | LSASSp | 0.59 | 0.14 | 0.79 | 1.05 | 0 | 0.99 | 0 | 0 | | SMINTEr | 0.59 | 0.58 | 1.83 | 0.36 | 0 | 1.22 | 0 | 0 | | SMCAUSlsa | 0.59 | 0.48 | 0.24 | 1.01 | 0.69 | 1.25 | 0 | 0 | | WRDAOAc | 0.59 | 0.24 | 1.66 | 0.94 | 0.16 | 0.74 | 0 | 0 | | SMCAUSvp | 0.54 | 0.02 | 0.93 | 1.33 | 0 | 0.46 | 0 | 0 | | DRAP | 0.54 | 0.23 | 0.46 | 0.94 | 0 | 1.05 | 0 | 0 | | WRDHYPv | 0.54 | 1.15 | 0.28 | 1.17 | 1.97 | 0.75 | 0 | 0 | | PCTEMPp | 0.53 | 0.38 | 1.05 | 0.38 | 0 | 0.82 | 0 | 0.88 | | SYNLE | 0.49 | 0.34 | 1.13 | 0.62 | 1.27 | 0.83 | 0 | 0 | | PCDCz | 0.48 | 0 | 0 | 1.93 | 0 | 0 | 0 | 0 | | CRFANPa | 0.46 | 0.18 | 0.76 | 0.77 | 0 | 0.78 | 0 | 0 | | WRDNOUN | 0.44 | 0.54 | 0.56 | 0.53 | 0 | 0.97 | 0 | 0 | | DESSC | 0.43 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 | | CNCNeg | 0.43 | 0 | 0 | 1.74 | 0 | 0 | 0 | 0 | | WRDPRP3s | 0.41 | 0.28 | 0.76 | 0.45 | 1.91 | 0.85 | 0 | 0 | | PCREFp | 0.4 | 0 | 0.06 | 0.46 | 0 | 1.13 | 0 | 0 | | PCREFz | 0.39 | 0.33 | 0.47 | 0.43 | 0 | 0.94 | 0 | 0 | | WRDADJ | 0.38 | 0.08 | 0.99 | 0.59 | 0 | 0.51 | 0 | 0 | | PCCNCz | 0.38 | 0.26 | 1.18 | 0.32 | 0 | 0.73 | 0 | 0 | | CRFCWO1 | 0.37 | 0 | 0 | 1.5 | 0 | 0 | 0 | 0 | | SMCAUSv | 0.35 | 0.38 | 0.39 | 0.53 | 0 | 0.68 | 0 | 0 | | WRDMEAc | 0.34 | 1.04 | 0.21 | 0.57 | 2.34 | 0.56 | 0 | 0 | | CNCAdd | 0.34 | 0 | 0 | 1.38 | 0 | 0 | 0 | 0 | | WRDFRQc | 0.34 | 0.53 | 0.54 | 0.83 | 0 | 0.28 | 0 | 0 | | PCCNCp | 0.32 | 0 | 0.57 | 0.13 | 0 | 0.93 | 0 | 0 | | SYNSTRUTt | 0.32 | 0 | 0 | 1.28 | 0 | 0 | 0 | 0 | | SYNNP | 0.32 | 0.14 | 0.03 | 0.26 | 0 | 1.02 | 0 | 0 | | LSASS1 | 0.32 | 0 | 0 | 1.3 | 0 | 0 | 0 | 0 | | CRFSOa | 0.31 | 0 | 0 | 1.25 | 0 | 0 | 0 | 0 | | PCSYNp | 0.29 | 0.55 | 0.68 | 0 | 0.13 | 0.85 | 0 | 0 | | SYNMEDlem | 0.28 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 | | SYNMEDwrd | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 | | RDFKGL | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.25 | 0.01 | 0.84 | 0.02 | 0 | 0.66 | 0 | 0 | | CNCAll | 0.22 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | CRFAOa | 0.21 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | PCTEMPz | 0.21 | 0 | 0 | 0.85 | 0 | 0 | 0 | 0 | | DESSL | 0.15 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 | | SMTEMP | 0.13 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | CRFANP1 | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | PCSYNz | 0.04 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | CRFSO1 | 0.03 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1b
This model used Coh-Metrix scores from 7 min narrative writing samples in winter [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:--------|:-------|:-------|:-------|:-------|:-------| | -9.468 | 0.2532 | -0.0876 | 0.2097 | 0.0554 | 0.2458 | 0.2979 | 0.0974 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 20.79 | 30 | 7.75 | 3.44 | 27.38 | 7.68 | 37.87 | 27.6 | | LDMTLD | 6.27 | 12.21 | 3.55 | 1.94 | 3.85 | 4.41 | 8.17 | 3.39 | | DESWLlt | 6.11 | 3.24 | 1.99 | 1.08 | 6.03 | 2.23 | 14.98 | 13.8 | | CRFCWO1d | 5.2 | 0.54 | 1.57 | 1.71 | 0.35 | 1.98 | 19.51 | 0 | | LSAGN | 4.28 | 7.08 | 2.73 | 2.24 | 1.41 | 3.97 | 0 | 17.19 | | RDL2 | 3.21 | 0.71 | 1.62 | 1.3 | 0 | 1.12 | 11.39 | 0 | | DESPL | 2.56 | 0.19 | 3.82 | 1.84 | 6.29 | 2.53 | 5.18 | 0 | | SYNLE | 1.48 | 3.13 | 0.87 | 1.41 | 1.45 | 2.12 | 0 | 0.48 | | LSAGNd | 1.29 | 0.53 | 1.67 | 1.39 | 0 | 1.34 | 0 | 7.02 | | WRDVERB | 1.28 | 1.66 | 1.91 | 1.32 | 3.02 | 2.19 | 0 | 0 | | WRDPRP3s | 1.2 | 1.06 | 2.64 | 1.2 | 3.48 | 2.15 | 0 | 0 | | RDFRE | 1.17 | 1.26 | 2.36 | 0.2 | 0 | 2.28 | 0 | 3.39 | | WRDIMGc | 1.1 | 1.23 | 0.57 | 0.96 | 3.24 | 1.06 | 0 | 3.39 | | DESSLd | 1.07 | 0.9 | 0.91 | 1.18 | 0 | 0.93 | 1.95 | 0 | | LDTTRa | 1.06 | 0.96 | 3.04 | 0.83 | 0 | 1.12 | 0 | 3.39 | | SYNMEDpos | 1.05 | 0.29 | 2.03 | 1.57 | 0 | 1.54 | 0 | 3.39 | | WRDNOUN | 1.02 | 0.83 | 1.48 | 1.39 | 5.64 | 1.14 | 0 | 0 | | PCCNCz | 1.01 | 0.78 | 1.56 | 1.26 | 0 | 1.22 | 0 | 3.39 | | WRDHYPnv | 1.01 | 1.06 | 0.45 | 1.07 | 1.76 | 1.1 | 0 | 3.39 | | DESWLltd | 0.95 | 1.21 | 0.88 | 1.2 | 2.76 | 1.52 | 0 | 0 | | LSASSp | 0.91 | 0.91 | 1.57 | 1.34 | 0 | 1.92 | 0 | 0 | | WRDHYPv | 0.88 | 0.72 | 1.87 | 1.14 | 2.79 | 1.32 | 0 | 0 | | PCCNCp | 0.88 | 0 | 1.25 | 1.28 | 1.4 | 1.19 | 0 | 3.39 | | SMCAUSwn | 0.87 | 1.08 | 1.35 | 1.6 | 2.18 | 0.78 | 0 | 0 | | LSASS1d | 0.86 | 0.25 | 1.96 | 1.47 | 3.26 | 1.3 | 0 | 0 | | CNCAdd | 0.84 | 0.74 | 1.13 | 0.8 | 1.85 | 0.53 | 0 | 3.39 | | SYNNP | 0.82 | 1.25 | 1.32 | 0.72 | 4.14 | 0.68 | 0 | 0 | | DESWLsy | 0.82 | 0.76 | 0.59 | 1.01 | 0 | 1.33 | 0.84 | 0 | | DESWLsyd | 0.81 | 1.52 | 0.69 | 1.28 | 0 | 0.96 | 0.13 | 0 | | WRDFRQmc | 0.75 | 0.84 | 0.92 | 0.99 | 2.76 | 1.03 | 0 | 0 | | PCVERBz | 0.74 | 0.37 | 1.53 | 1.4 | 0 | 1.59 | 0 | 0 | | PCREFp | 0.73 | 0 | 0.41 | 0.55 | 5.81 | 0.26 | 0 | 3.39 | | CRFAOa | 0.72 | 0.22 | 1.88 | 1.32 | 0 | 1.6 | 0 | 0 | | DRVP | 0.71 | 1.3 | 0.17 | 0.73 | 2.1 | 1.01 | 0 | 0 | | CRFCWOad | 0.71 | 0.1 | 1.93 | 1.51 | 0 | 1.49 | 0 | 0 | | SMCAUSv | 0.68 | 1.02 | 1.48 | 0.65 | 0 | 1.26 | 0 | 0 | | PCSYNz | 0.68 | 0.41 | 1.89 | 0.69 | 0 | 1.78 | 0 | 0 | | PCNARz | 0.67 | 0.59 | 1.31 | 1.29 | 0 | 1.15 | 0 | 0 | | CRFCWO1 | 0.67 | 0.42 | 1.72 | 1.38 | 0 | 1.12 | 0 | 0 | | CRFANPa | 0.66 | 0.22 | 1.25 | 1.25 | 0 | 1.57 | 0 | 0 | | CRFNOa | 0.66 | 0.63 | 0.71 | 1.5 | 0 | 1.08 | 0 | 0 | | DRPP | 0.66 | 0.69 | 1.38 | 0.81 | 2.9 | 0.69 | 0 | 0 | | CNCTemp | 0.65 | 0.55 | 0.28 | 1.47 | 0.68 | 1.12 | 0 | 0 | | SMINTEr | 0.63 | 0.72 | 1.23 | 0.55 | 0 | 1.56 | 0 | 0 | | CNCNeg | 0.62 | 1.15 | 0.39 | 0.92 | 0 | 0.95 | 0 | 0 | | LDTTRc | 0.62 | 0.65 | 1.34 | 1.03 | 0.22 | 0.99 | 0 | 0 | | WRDADV | 0.62 | 0.62 | 0.79 | 0.93 | 0 | 1.4 | 0 | 0 | | SMINTEp | 0.61 | 0.13 | 1.26 | 1.31 | 0 | 1.38 | 0 | 0 | | WRDFRQa | 0.6 | 1.08 | 1.08 | 0.8 | 0 | 0.8 | 0 | 0 | | SMCAUSlsa | 0.58 | 0.83 | 0.64 | 0.6 | 0 | 1.33 | 0 | 0 | | SMCAUSvp | 0.58 | 0.33 | 1.47 | 1.12 | 0 | 1.08 | 0 | 0 | | CNCAll | 0.58 | 0.92 | 1.12 | 0.75 | 1.02 | 0.64 | 0 | 0 | | SYNSTRUTa | 0.56 | 0.4 | 1.51 | 0.94 | 0 | 1.03 | 0 | 0 | | WRDADJ | 0.55 | 1.43 | 0.3 | 0.4 | 0.51 | 0.72 | 0 | 0 | | WRDMEAc | 0.54 | 0.65 | 0.09 | 1.1 | 0 | 1.05 | 0 | 0 | | DRNP | 0.53 | 0.53 | 1.42 | 0.45 | 0 | 1.25 | 0 | 0 | | PCTEMPp | 0.52 | 0.51 | 1.21 | 0.8 | 0 | 0.95 | 0 | 0 | | PCVERBp | 0.52 | 0 | 0.91 | 1.17 | 0 | 1.3 | 0 | 0 | | SMCAUSr | 0.52 | 0.25 | 1.49 | 0.08 | 0 | 1.87 | 0 | 0 | | PCSYNp | 0.51 | 0.01 | 1.27 | 0.44 | 0 | 1.82 | 0 | 0 | | CNCCaus | 0.5 | 0.62 | 0.42 | 0.72 | 0 | 1.12 | 0 | 0 | | WRDPRO | 0.49 | 0.65 | 0.75 | 0.93 | 0.14 | 0.62 | 0 | 0 | | WRDAOAc | 0.48 | 0.84 | 0.97 | 0.55 | 0 | 0.7 | 0 | 0 | | PCNARp | 0.46 | 0 | 1.14 | 1.07 | 0 | 0.96 | 0 | 0 | | CNCTempx | 0.45 | 0.46 | 0.67 | 0.68 | 0 | 0.99 | 0 | 0 | | WRDFRQc | 0.42 | 0.49 | 0.25 | 0.78 | 0 | 0.85 | 0 | 0 | | PCCONNp | 0.39 | 0.8 | 0.87 | 0.29 | 0 | 0.57 | 0 | 0 | | DRNEG | 0.37 | 0.17 | 0.47 | 0.85 | 0.01 | 0.75 | 0 | 0 | | WRDPOLc | 0.36 | 0.19 | 0.72 | 0.73 | 0 | 0.72 | 0 | 0 | | CNCLogic | 0.35 | 0.26 | 0.34 | 0.7 | 1.58 | 0.35 | 0 | 0 | | DESSC | 0.34 | 0 | 0 | 1.84 | 0 | 0 | 0 | 0 | | PCREFz | 0.33 | 0.51 | 0.84 | 0.55 | 0 | 0.3 | 0 | 0 | | SYNMEDwrd | 0.32 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 | | WRDFAMc | 0.31 | 0.46 | 0 | 0.71 | 0 | 0.42 | 0 | 0 | | LSASSpd | 0.3 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 | | PCDCp | 0.29 | 0.25 | 0.78 | 0.71 | 0 | 0.25 | 0 | 0 | | SYNMEDlem | 0.29 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | WRDHYPn | 0.29 | 0.36 | 1.3 | 0.66 | 0 | 0 | 0 | 0 | | CRFSOa | 0.28 | 0 | 0 | 1.53 | 0 | 0 | 0 | 0 | | DRAP | 0.26 | 0.3 | 0.38 | 0.69 | 0 | 0.21 | 0 | 0 | | LSASS1 | 0.26 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 | | CRFCWOa | 0.25 | 0 | 0 | 1.37 | 0 | 0 | 0 | 0 | | WRDCNCc | 0.21 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 | | SYNSTRUTt | 0.21 | 0 | 0 | 1.13 | 0 | 0 | 0 | 0 | | PCTEMPz | 0.21 | 0 | 0 | 1.14 | 0 | 0 | 0 | 0 | | SMTEMP | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 | | CRFANP1 | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 | | PCDCz | 0.18 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.17 | 0 | 0.55 | 0 | 0 | 0.71 | 0 | 0 | | CNCADC | 0.15 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | CNCPos | 0.14 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 | | PCCONNz | 0.12 | 0 | 0 | 0.65 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.1 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | DESSL | 0.08 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | RDFKGL | 0.06 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | CRFSO1 | 0.04 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1c
This model used Coh-Metrix scores from 7 min narrative writing samples in spring [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:-------|:--------| | -4.8423 | 0.5169 | 0.1348 | 0.6009 | -0.2375 | -0.4134 | 0.4001 | -0.0098 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 20.66 | 36.45 | 5.76 | 2.78 | 21.32 | 6.39 | 47.19 | 16.14 | | WRDVERB | 5.51 | 3.32 | 1.88 | 1.07 | 0.58 | 2.06 | 22.99 | 2.79 | | WRDHYPn | 4.27 | 2.98 | 2.38 | 1.13 | 3.13 | 1.75 | 14.74 | 1.28 | | DESSLd | 3.23 | 1.17 | 0.87 | 1.74 | 3.05 | 2.13 | 10.33 | 0 | | PCNARp | 2.65 | 2.58 | 2.72 | 1.77 | 10.29 | 2.17 | 0 | 4.99 | | DESPL | 2.51 | 2.16 | 3.29 | 1.6 | 2.76 | 2.2 | 4.27 | 2.56 | | DESWLltd | 1.8 | 4.36 | 1.75 | 1.09 | 1.39 | 1.51 | 0 | 6.97 | | WRDNOUN | 1.59 | 1.69 | 2.13 | 0.8 | 5.92 | 1.42 | 0 | 5.81 | | WRDFRQmc | 1.59 | 2.27 | 2.04 | 1.46 | 2.86 | 1.51 | 0 | 5.34 | | DESWLlt | 1.55 | 0.7 | 2.05 | 0.77 | 6.89 | 1.97 | 0 | 3.72 | | CRFANPa | 1.45 | 1.93 | 1.23 | 1.64 | 0.05 | 2.84 | 0 | 0 | | LSASS1d | 1.43 | 2.3 | 1.5 | 1.82 | 0.08 | 1.92 | 0 | 0 | | LDMTLD | 1.43 | 2.6 | 1.79 | 1.39 | 0 | 2.14 | 0 | 0 | | CRFCWOa | 1.4 | 2.06 | 1.9 | 1.74 | 0 | 2.09 | 0 | 0 | | WRDHYPv | 1.31 | 1.51 | 2.37 | 0.97 | 2.75 | 1.62 | 0 | 1.63 | | PCDCz | 1.31 | 2.21 | 1.05 | 1.47 | 0 | 2 | 0 | 1.97 | | WRDPRP3s | 1.28 | 1.96 | 1.36 | 0.66 | 3.27 | 1.43 | 0 | 0.81 | | SMCAUSwn | 1.26 | 1.16 | 2.26 | 1.16 | 3.45 | 1.13 | 0 | 2.9 | | SMCAUSvp | 1.25 | 2.1 | 0.97 | 1.55 | 0 | 1.8 | 0 | 0 | | SYNSTRUTa | 1.17 | 1.47 | 1.74 | 1.81 | 0 | 1.38 | 0 | 3.72 | | PCDCp | 1.16 | 0 | 1.81 | 1.28 | 3.29 | 2.05 | 0 | 4.53 | | LSAGN | 1.12 | 0.71 | 1.72 | 1.81 | 0 | 2.08 | 0 | 2.09 | | RDL2 | 1.04 | 0.86 | 2.17 | 0.93 | 2.07 | 1.45 | 0 | 1.28 | | SMCAUSlsa | 1.01 | 0.63 | 1.59 | 1.07 | 3.13 | 0.94 | 0 | 3.72 | | DRPP | 0.98 | 1.61 | 1.78 | 0.67 | 0.69 | 1.47 | 0 | 1.28 | | LSAGNd | 0.97 | 0.08 | 2.29 | 2 | 0 | 1.63 | 0 | 0 | | SYNMEDpos | 0.94 | 0.3 | 1.78 | 1.48 | 0 | 2.09 | 0 | 0.93 | | CNCTemp | 0.9 | 1.06 | 0.85 | 0.91 | 1.61 | 1.18 | 0 | 0 | | WRDADV | 0.89 | 1.56 | 1.9 | 0.67 | 0 | 1.4 | 0 | 0 | | CNCPos | 0.87 | 0.61 | 0.82 | 0.74 | 1.97 | 1.58 | 0 | 2.44 | | SMCAUSv | 0.86 | 1.16 | 0.91 | 1.27 | 0 | 1.22 | 0 | 0 | | PCTEMPp | 0.85 | 0.19 | 1.68 | 1.09 | 2.36 | 0.96 | 0 | 2.09 | | PCVERBz | 0.85 | 0.24 | 1.73 | 1.52 | 0 | 1.6 | 0 | 3.37 | | LDTTRc | 0.84 | 1.39 | 1.26 | 0.82 | 0 | 1.37 | 0 | 0 | | PCREFz | 0.78 | 0.25 | 1.2 | 0.56 | 2.5 | 1.41 | 0 | 0 | | RDFKGL | 0.77 | 0.36 | 2.06 | 0.91 | 0 | 1.86 | 0 | 0 | | LSASSp | 0.76 | 0.04 | 1.97 | 1.62 | 0 | 1.16 | 0 | 0.93 | | PCVERBp | 0.75 | 0.3 | 1.09 | 1.28 | 0 | 1.55 | 0 | 0 | | CRFCWO1d | 0.73 | 0.13 | 1.54 | 1.52 | 0 | 1.18 | 0 | 0 | | DRNP | 0.7 | 0.54 | 1.62 | 0.83 | 0 | 1.51 | 0 | 0 | | WRDPRO | 0.69 | 0.48 | 0.73 | 0.6 | 1.85 | 0.96 | 0 | 4.18 | | SMCAUSr | 0.69 | 0.9 | 0.05 | 0.72 | 0.63 | 1.33 | 0 | 0 | | WRDAOAc | 0.69 | 0.92 | 0.69 | 0.84 | 0.51 | 0.98 | 0 | 0 | | LDTTRa | 0.68 | 0.05 | 2.31 | 0.95 | 0 | 1.57 | 0 | 0 | | WRDCNCc | 0.67 | 0.53 | 1.3 | 0.38 | 3.63 | 0 | 0 | 1.28 | | PCSYNz | 0.62 | 0.24 | 1.76 | 0.77 | 0 | 1.45 | 0 | 1.28 | | CNCCaus | 0.61 | 0.72 | 0.73 | 0.73 | 0.12 | 1.1 | 0 | 0 | | WRDMEAc | 0.57 | 0.51 | 0.68 | 0.45 | 2.11 | 0.46 | 0 | 0 | | CNCTempx | 0.57 | 0.45 | 0.26 | 1.36 | 0.06 | 0.53 | 0 | 0 | | DESWLsy | 0.56 | 0.34 | 0.85 | 0.55 | 0.35 | 0.89 | 0.49 | 1.28 | | WRDPOLc | 0.56 | 0.48 | 0.89 | 0.63 | 0 | 1.31 | 0 | 0 | | SYNLE | 0.55 | 0.35 | 0.09 | 0.62 | 0 | 1.67 | 0 | 0 | | PCCNCz | 0.54 | 0.14 | 1.87 | 0.96 | 0 | 0.72 | 0 | 3.25 | | DRNEG | 0.54 | 0.05 | 0.73 | 0.81 | 1.44 | 0.74 | 0 | 0 | | WRDFRQa | 0.52 | 0.4 | 0.36 | 0.71 | 0 | 1.2 | 0 | 1.28 | | WRDFRQc | 0.52 | 0.97 | 0.3 | 0.33 | 0 | 1.09 | 0 | 1.28 | | DRVP | 0.5 | 0.11 | 0.79 | 0.77 | 0.61 | 0.88 | 0 | 0.81 | | SYNNP | 0.5 | 0.21 | 0.65 | 0.88 | 0 | 1.04 | 0 | 0 | | LSASSpd | 0.5 | 0 | 0 | 1.94 | 0 | 0 | 0 | 0 | | CNCLogic | 0.49 | 0.55 | 0.41 | 0.68 | 0 | 0.9 | 0 | 0 | | CNCADC | 0.49 | 0.38 | 1.61 | 0.46 | 0.24 | 0.93 | 0 | 0 | | SYNSTRUTt | 0.48 | 0 | 0 | 1.83 | 0 | 0 | 0 | 0 | | PCNARz | 0.46 | 0 | 0 | 1.77 | 0 | 0 | 0 | 0 | | PCCNCp | 0.46 | 0 | 1.15 | 0.87 | 0 | 0.93 | 0 | 0 | | WRDADJ | 0.45 | 0.29 | 1.12 | 0.72 | 0 | 0.76 | 0 | 0 | | CRFCWOad | 0.44 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 | | CRFAOa | 0.43 | 0 | 0 | 1.64 | 0 | 0 | 0 | 0 | | PCSYNp | 0.43 | 0 | 1.45 | 0.62 | 0 | 1.01 | 0 | 0 | | DESSC | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | CRFCWO1 | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | SYNMEDwrd | 0.42 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.41 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 | | LSASS1 | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 | | SYNMEDlem | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 | | DRAP | 0.4 | 0.23 | 1.23 | 0.64 | 0 | 0.58 | 0 | 0 | | SMINTEp | 0.38 | 0.36 | 1.06 | 0.68 | 0 | 0.29 | 0 | 0.81 | | PCTEMPz | 0.38 | 0 | 0 | 1.45 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.38 | 0.08 | 0.44 | 0.01 | 1.83 | 0.83 | 0 | 0 | | SMTEMP | 0.37 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 | | WRDHYPnv | 0.36 | 0.3 | 0.02 | 0.78 | 0 | 0.5 | 0 | 0 | | CRFANP1 | 0.35 | 0 | 0 | 1.35 | 0 | 0 | 0 | 0 | | CRFNOa | 0.34 | 0 | 0 | 1.32 | 0 | 0 | 0 | 0 | | CRFSOa | 0.3 | 0 | 0 | 1.17 | 0 | 0 | 0 | 0 | | PCREFp | 0.28 | 0 | 0.32 | 0.32 | 0 | 0.97 | 0 | 0 | | DESWLsyd | 0.27 | 0.32 | 0.32 | 0.45 | 0 | 0.34 | 0 | 1.28 | | PCCONNp | 0.26 | 0.07 | 1.18 | 0.05 | 0 | 0.87 | 0 | 0 | | PCCONNz | 0.26 | 0.18 | 0.53 | 0.23 | 0 | 0.69 | 0 | 0 | | DESSL | 0.24 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.22 | 0 | 0.98 | 0.06 | 0 | 0.81 | 0 | 0 | | RDFRE | 0.21 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 | | WRDFAMc | 0.21 | 0.29 | 0 | 0.07 | 1.19 | 0.02 | 0 | 0 | | SMINTEr | 0.21 | 0.07 | 0.37 | 0.32 | 0 | 0.5 | 0 | 0 | | CNCNeg | 0.15 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | CNCAll | 0.12 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | CNCAdd | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | WRDIMGc | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | CRFSO1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 1d
This model used principal components scores from 7 min narrative writing samples in fall [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:------|:-------| | -20.0773 | 0.0971 | 0.7558 | 0.5784 | -0.4401 | -4e-04 | 0.002 | 0.0227 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:-----|:------|:------|:------| | PC3 | 16.71 | 31.88 | 19.04 | 0 | 8.86 | 20.39 | 26.66 | 17.12 | | PC5 | 12.28 | 18.78 | 13.53 | 0 | 8.6 | 14.58 | 19.92 | 8.98 | | PC1 | 11.48 | 6.86 | 17.08 | 0 | 3.49 | 5.37 | 14.52 | 8.98 | | PC8 | 7.3 | 4.81 | 7.77 | 0 | 7.07 | 4.77 | 9.88 | 8.47 | | PC9 | 5 | 7.06 | 4.69 | 0 | 4.66 | 7.51 | 11.76 | 8.47 | | PC4 | 4.73 | 2.76 | 5.98 | 0 | 3.09 | 2.6 | 0 | 7.97 | | PC11 | 4.57 | 2.64 | 4.52 | 0 | 5.05 | 1.91 | 0 | 8.47 | | PC7 | 2.91 | 4.92 | 2.9 | 0 | 2.16 | 5.93 | 0 | 8.47 | | PC34 | 2.6 | 0.6 | 0.87 | 0 | 5.78 | 0.91 | 0 | 7.97 | | PC14 | 2.37 | 0.66 | 2.18 | 0 | 3.18 | 0.56 | 0 | 2.2 | | PC16 | 2.3 | 0.66 | 1.87 | 0 | 3.52 | 1.18 | 0 | 1.69 | | PC21 | 2.27 | 0.65 | 1.5 | 0 | 3.75 | 1.41 | 0 | 6.78 | | PC10 | 2.26 | 1.45 | 2.34 | 0 | 2.39 | 1.39 | 0 | 1.69 | | PC30 | 2.07 | 0.38 | 0.88 | 0 | 4.61 | 1.45 | 0 | 0.51 | | PC15 | 1.81 | 0 | 1.61 | 0 | 2.7 | 1.24 | 0 | 0.51 | | PC31 | 1.8 | 1.2 | 0.69 | 0 | 3.86 | 1.42 | 0 | 1.19 | | PC6 | 1.7 | 0.76 | 2.11 | 0 | 1.37 | 1.29 | 0 | 0 | | PC35 | 1.7 | 0.15 | 0.51 | 0 | 4.07 | 1.11 | 5.51 | 0 | | PC17 | 1.5 | 0.35 | 1.24 | 0 | 2.35 | 1.14 | 0 | 0 | | PC12 | 1.4 | 0.88 | 1.45 | 0 | 1.59 | 0.23 | 0 | 0 | | PC24 | 1.39 | 0.83 | 0.85 | 0 | 2.51 | 1.02 | 0 | 0 | | PC13 | 1.3 | 0.62 | 1.24 | 0 | 1.66 | 0.98 | 0 | 0 | | PC22 | 1.27 | 1.34 | 0.81 | 0 | 2.08 | 2.74 | 0 | 0 | | PC19 | 1.11 | 2.25 | 0.72 | 0 | 1.51 | 4.51 | 0 | 0.51 | | PC32 | 1.09 | 1.42 | 0.37 | 0 | 2.27 | 0.42 | 0 | 0 | | PC2 | 0.89 | 1.16 | 1.19 | 0 | 0.33 | 2.11 | 0.87 | 0 | | PC26 | 0.67 | 0.85 | 0.34 | 0 | 1.2 | 2.9 | 0 | 0 | | PC33 | 0.64 | 0.61 | 0.17 | 0 | 1.32 | 0.89 | 7.71 | 0 | | PC18 | 0.63 | 0.17 | 0.5 | 0 | 0.93 | 0.58 | 3.18 | 0 | | PC29 | 0.6 | 0.71 | 0.24 | 0 | 1.18 | 2.26 | 0 | 0 | | PC28 | 0.51 | 0.26 | 0.23 | 0 | 1.08 | 0.92 | 0 | 0 | | PC20 | 0.43 | 0.87 | 0.27 | 0 | 0.63 | 0 | 0 | 0 | | PC25 | 0.34 | 0.22 | 0.19 | 0 | 0.64 | 0.93 | 0 | 0 | | PC27 | 0.33 | 0.86 | 0.12 | 0 | 0.52 | 2.07 | 0 | 0 | | PC23 | 0.04 | 0.37 | 0 | 0 | 0 | 1.27 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components are displayed.
| Variable | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 14.67 | 14.32 | 5.99 | 5.95 | 5.14 | 4.89 | 4.76 | 4.01 | 4.00 | 3.01 | | Proportion Var | 0.15 | 0.15 | 0.06 | 0.06 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.15 | 0.30 | 0.36 | 0.42 | 0.47 | 0.53 | 0.57 | 0.62 | 0.66 | 0.69 | | Proportion Explained | 0.22 | 0.21 | 0.09 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 | | Cumulative Proportion | 0.22 | 0.43 | 0.52 | 0.61 | 0.69 | 0.76 | 0.83 | 0.90 | 0.95 | 1.00 |
Varimax Rotated Loadings
| Metric | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 | | DESWC | 0.06 | 0.11 | 0.34 | 0.23 | -0.09 | -0.06 | -0.07 | 0.1 | 0.78 | -0.03 | | DESPL | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 | | DESSL | -0.37 | -0.76 | -0.03 | 0.23 | 0.06 | 0.01 | -0.06 | 0.01 | 0.3 | 0.14 | | DESSLd | 0.52 | -0.1 | 0.04 | 0.32 | 0.1 | -0.04 | -0.04 | -0.01 | 0.25 | -0.45 | | DESWLsy | 0.08 | 0.09 | 0.71 | -0.01 | -0.06 | 0.24 | 0.25 | -0.07 | 0.02 | -0.02 | | DESWLsyd | 0.08 | 0.07 | 0.66 | 0.01 | 0.01 | 0.22 | 0.18 | -0.13 | 0 | 0.13 | | DESWLlt | 0.07 | 0.29 | 0.71 | 0.12 | 0.11 | 0.05 | 0.19 | 0.11 | 0.01 | -0.08 | | DESWLltd | 0.22 | 0.24 | 0.69 | 0.08 | 0.06 | 0.21 | -0.09 | -0.06 | 0.02 | 0.2 | | PCNARz | 0.74 | 0.23 | -0.02 | 0.1 | -0.22 | 0.02 | -0.51 | -0.1 | 0.01 | 0.07 | | PCNARp | 0.61 | 0.35 | 0.13 | 0.06 | -0.2 | 0.07 | -0.48 | -0.03 | 0.06 | -0.01 | | PCSYNz | -0.09 | 0.88 | 0.13 | -0.06 | -0.04 | -0.1 | 0.01 | 0.14 | -0.33 | 0.08 | | PCSYNp | -0.19 | 0.84 | 0.16 | 0.01 | -0.01 | -0.04 | -0.03 | 0.15 | -0.22 | 0.12 | | PCCNCz | -0.35 | -0.48 | -0.08 | 0.04 | 0.61 | -0.35 | 0.08 | 0.08 | -0.02 | 0.27 | | PCCNCp | -0.12 | -0.35 | -0.02 | 0.03 | 0.57 | -0.38 | 0.1 | 0.14 | -0.08 | 0.25 | | PCREFz | 0.71 | -0.34 | -0.25 | -0.08 | 0.02 | 0.04 | -0.03 | 0.14 | 0.07 | 0.48 | | PCREFp | 0.45 | -0.36 | -0.26 | -0.03 | -0.03 | 0.04 | -0.1 | 0.13 | 0.16 | 0.52 | | PCDCz | 0.12 | 0.17 | 0.12 | 0.9 | -0.09 | -0.15 | -0.02 | 0.22 | 0.04 | 0.04 | | PCDCp | 0.12 | 0.13 | 0.14 | 0.85 | -0.14 | -0.14 | 0.02 | 0.16 | -0.01 | 0.08 | | PCVERBz | -0.63 | -0.44 | -0.46 | 0.05 | -0.06 | 0.12 | 0.2 | 0.21 | 0.12 | 0.05 | | PCVERBp | -0.47 | -0.21 | -0.5 | 0.06 | -0.01 | 0.13 | 0.21 | 0.3 | 0.11 | -0.08 | | PCCONNz | 0.04 | 0.09 | 0.24 | 0.09 | -0.02 | 0.87 | -0.06 | -0.06 | 0.05 | 0.05 | | PCCONNp | -0.08 | 0.01 | 0.02 | 0.01 | 0.01 | 0.81 | 0 | -0.1 | -0.12 | 0.01 | | PCTEMPz | 0.67 | 0.64 | 0.17 | 0.06 | 0.06 | 0.01 | 0.01 | 0 | 0.01 | -0.28 | | PCTEMPp | 0.4 | 0.37 | 0.04 | 0.04 | 0.17 | -0.16 | 0.01 | 0 | 0.14 | -0.35 | | CRFNO1 | 0.61 | -0.16 | 0.11 | -0.12 | 0.06 | 0.06 | 0.49 | 0.2 | 0.17 | 0.09 | | CRFAO1 | 0.88 | 0.21 | 0.13 | 0.02 | -0.01 | 0.08 | -0.06 | 0.09 | 0 | 0.08 | | CRFSO1 | 0.64 | -0.13 | 0.12 | -0.06 | 0.04 | 0.01 | 0.51 | 0.18 | 0.19 | 0.01 | | CRFNOa | 0.64 | -0.21 | 0.11 | -0.12 | 0.1 | 0.06 | 0.48 | 0.16 | 0.13 | 0.08 | | CRFAOa | 0.91 | 0.21 | 0.07 | 0.06 | 0.06 | 0.06 | -0.09 | 0.01 | -0.07 | 0.06 | | CRFSOa | 0.65 | -0.16 | 0.12 | -0.07 | 0.09 | 0.01 | 0.53 | 0.15 | 0.13 | 0.02 | | CRFCWO1 | 0.88 | 0.18 | -0.08 | -0.03 | 0 | 0.07 | -0.06 | 0.14 | -0.01 | 0.2 | | CRFCWO1d | 0.2 | 0.62 | 0.03 | 0.16 | 0.01 | 0.06 | -0.13 | -0.03 | 0.29 | 0.02 | | CRFCWOa | 0.9 | 0.14 | -0.11 | -0.03 | 0.04 | 0.04 | -0.08 | 0.07 | -0.1 | 0.14 | | CRFCWOad | 0.2 | 0.76 | -0.03 | 0.09 | 0.03 | 0.04 | 0.04 | -0.02 | 0.24 | -0.06 | | CRFANP1 | 0.82 | 0.29 | 0.09 | 0.06 | 0 | 0.02 | -0.22 | -0.02 | -0.02 | 0 | | CRFANPa | 0.85 | 0.09 | 0.01 | 0.12 | 0.05 | -0.03 | -0.25 | -0.06 | -0.13 | 0.04 | | LSASS1 | 0.83 | 0.06 | 0.02 | -0.02 | 0.05 | -0.01 | 0.11 | -0.01 | 0.14 | -0.01 | | LSASS1d | 0.16 | 0.7 | 0.09 | 0.05 | -0.01 | 0.05 | 0.08 | -0.05 | 0.26 | 0.06 | | LSASSp | 0.85 | 0.01 | 0.03 | 0 | 0.08 | -0.01 | 0.11 | -0.04 | 0.08 | -0.04 | | LSASSpd | 0.24 | 0.76 | 0.12 | 0.05 | 0 | 0.08 | 0.13 | 0.02 | 0.28 | 0.05 | | LSAGN | 0.56 | 0.66 | 0.2 | -0.01 | -0.03 | 0 | 0.08 | 0.02 | 0.3 | -0.02 | | LSAGNd | 0.78 | 0.41 | 0.12 | 0 | 0.08 | 0.03 | 0.07 | -0.02 | 0.08 | -0.15 | | LDTTRc | -0.06 | -0.01 | 0.07 | -0.11 | 0.02 | 0.03 | -0.35 | -0.41 | -0.59 | 0.03 | | LDTTRa | -0.16 | 0.12 | 0.12 | -0.07 | -0.11 | 0.11 | -0.04 | -0.17 | -0.76 | -0.16 | | LDMTLD | -0.08 | 0.2 | 0.52 | 0.21 | -0.08 | 0.15 | -0.17 | 0.08 | 0.09 | -0.19 | | CNCAll | -0.12 | -0.12 | -0.21 | 0.51 | 0.01 | -0.77 | 0.05 | -0.03 | -0.03 | 0.07 | | CNCCaus | 0 | 0.01 | 0.08 | 0.86 | -0.02 | 0.08 | 0 | 0.06 | -0.09 | 0.01 | | CNCLogic | 0.02 | 0 | 0.05 | 0.64 | -0.3 | -0.25 | -0.04 | 0.3 | 0.01 | 0.16 | | CNCADC | 0.07 | 0.02 | 0.16 | 0.01 | -0.37 | -0.25 | -0.16 | 0.43 | 0.06 | 0.04 | | CNCTemp | 0.01 | 0.11 | 0.09 | 0.31 | -0.1 | -0.28 | -0.01 | 0.11 | 0.03 | 0.22 | | CNCTempx | 0.11 | 0.11 | -0.06 | 0.29 | 0.05 | 0.1 | 0.06 | 0 | 0.06 | 0.28 | | CNCAdd | -0.11 | -0.19 | -0.33 | -0.04 | 0.09 | -0.83 | 0.05 | -0.08 | -0.06 | -0.04 | | CNCPos | -0.13 | -0.1 | -0.22 | 0.53 | 0.12 | -0.67 | 0.09 | -0.14 | -0.06 | 0.09 | | CNCNeg | 0.03 | 0.01 | 0.13 | 0.02 | -0.35 | -0.26 | -0.19 | 0.44 | 0.03 | 0.01 | | SMCAUSv | -0.02 | 0.62 | 0.09 | 0.06 | 0.06 | -0.02 | -0.2 | 0.16 | -0.33 | 0.12 | | SMCAUSvp | -0.02 | 0.44 | 0.11 | 0.52 | 0.02 | -0.01 | -0.17 | 0.2 | -0.32 | 0.05 | | SMINTEp | 0.06 | 0.59 | 0.19 | 0.14 | 0.1 | -0.06 | -0.1 | 0.05 | -0.2 | 0.35 | | SMCAUSr | -0.02 | -0.34 | 0 | 0.72 | -0.07 | 0.07 | 0.02 | 0.05 | 0.27 | -0.14 | | SMINTEr | -0.07 | -0.31 | 0.01 | 0.72 | -0.05 | 0.1 | 0.06 | 0.01 | 0.26 | -0.13 | | SMCAUSlsa | 0.03 | 0.03 | -0.29 | -0.15 | -0.03 | 0.34 | 0.35 | 0.48 | 0.1 | -0.02 | | SMCAUSwn | 0.04 | 0.1 | -0.14 | 0.32 | 0.08 | 0.08 | 0.02 | 0.73 | 0.11 | 0.02 | | SMTEMP | 0.67 | 0.64 | 0.17 | 0.04 | 0.07 | 0.02 | 0.01 | 0.01 | 0 | -0.26 | | SYNLE | 0.01 | -0.19 | -0.02 | 0.34 | 0.03 | 0.02 | -0.06 | 0.07 | 0.34 | -0.19 | | SYNNP | -0.05 | -0.04 | 0.03 | 0 | 0.14 | -0.09 | 0.61 | 0 | 0.1 | -0.32 | | SYNMEDpos | 0.63 | 0.56 | 0.2 | 0.04 | 0.02 | 0.05 | 0.01 | -0.05 | -0.05 | -0.38 | | SYNMEDwrd | 0.61 | 0.64 | 0.24 | 0.05 | 0.04 | 0.05 | -0.01 | -0.01 | -0.03 | -0.33 | | SYNMEDlem | 0.61 | 0.63 | 0.24 | 0.05 | 0.04 | 0.03 | -0.02 | -0.01 | -0.02 | -0.34 | | SYNSTRUTa | 0.06 | 0.8 | -0.01 | -0.05 | 0.08 | 0.19 | 0.1 | -0.06 | -0.16 | 0.1 | | SYNSTRUTt | 0.08 | 0.83 | 0.03 | -0.03 | 0.05 | 0.2 | 0.07 | -0.08 | -0.2 | 0.08 | | DRNP | 0.02 | -0.01 | -0.27 | -0.05 | 0.27 | 0.43 | 0.02 | 0.07 | -0.19 | 0.07 | | DRVP | 0.15 | -0.03 | 0.14 | -0.08 | -0.09 | 0.18 | -0.63 | 0.14 | 0.02 | -0.02 | | DRAP | -0.11 | 0.06 | 0.26 | 0.12 | -0.57 | -0.19 | 0.03 | -0.01 | -0.16 | 0.23 | | DRPP | 0.13 | 0.23 | 0.27 | 0.09 | 0.2 | 0.09 | 0.04 | -0.16 | 0.27 | 0.25 | | DRNEG | -0.13 | -0.23 | 0.1 | -0.1 | -0.33 | -0.01 | -0.18 | -0.01 | 0.28 | 0.05 | | WRDNOUN | -0.15 | 0.11 | 0.07 | -0.07 | 0.29 | 0.09 | 0.7 | -0.09 | -0.09 | -0.03 | | WRDVERB | -0.05 | -0.05 | 0.03 | -0.05 | 0.13 | -0.1 | -0.54 | 0.18 | 0.12 | -0.12 | | WRDADJ | -0.01 | -0.04 | -0.13 | 0.01 | -0.11 | 0.1 | 0.33 | 0.01 | -0.16 | -0.5 | | WRDADV | -0.11 | 0.01 | 0.29 | 0.2 | -0.7 | -0.18 | -0.04 | 0.02 | -0.01 | 0.17 | | WRDPRO | 0.13 | -0.08 | -0.25 | -0.09 | -0.1 | 0.16 | -0.6 | -0.03 | -0.21 | 0.13 | | WRDPRP3s | -0.14 | -0.02 | -0.1 | 0.13 | -0.01 | 0.02 | -0.17 | 0.12 | 0.05 | -0.17 | | WRDPRP3p | -0.06 | 0.04 | 0.12 | 0.25 | 0.04 | -0.08 | 0 | -0.01 | 0.13 | -0.1 | | WRDFRQc | -0.1 | -0.09 | -0.49 | 0.07 | -0.54 | 0.25 | -0.02 | -0.07 | 0.09 | 0.11 | | WRDFRQa | -0.11 | -0.21 | -0.59 | -0.13 | -0.21 | -0.01 | 0.1 | -0.44 | 0.07 | 0.04 | | WRDFRQmc | 0.09 | 0.64 | 0.01 | 0.13 | -0.03 | 0.07 | -0.16 | 0.25 | 0 | -0.08 | | WRDAOAc | 0.1 | -0.02 | 0.39 | 0.12 | 0.03 | 0 | -0.19 | 0.27 | 0.11 | -0.04 | | WRDFAMc | 0 | 0.02 | -0.28 | 0.15 | -0.19 | 0.19 | 0.02 | -0.15 | 0.31 | 0.18 | | WRDCNCc | 0.09 | 0.1 | 0.24 | -0.08 | 0.77 | -0.05 | 0.11 | 0.15 | 0.14 | 0.15 | | WRDIMGc | 0.09 | 0.04 | 0.22 | -0.14 | 0.86 | -0.03 | 0.12 | 0.06 | 0.03 | 0.02 | | WRDMEAc | 0.06 | 0.04 | 0.22 | 0.03 | 0.73 | -0.04 | 0.03 | -0.06 | -0.09 | 0.06 | | WRDPOLc | -0.01 | 0.08 | -0.23 | 0.32 | -0.03 | -0.03 | -0.13 | 0.66 | -0.04 | -0.05 | | WRDHYPn | 0.12 | 0.15 | 0.38 | 0.13 | 0.13 | 0 | -0.1 | 0.66 | 0.13 | -0.02 | | WRDHYPv | 0.01 | -0.01 | 0.02 | 0.14 | 0.27 | -0.19 | -0.22 | 0.39 | -0.03 | 0.26 | | WRDHYPnv | 0.01 | 0.08 | 0.37 | 0.05 | 0.38 | 0.07 | 0.22 | 0.6 | 0.09 | 0.06 | | RDFRE | 0.35 | 0.74 | -0.17 | -0.23 | -0.05 | -0.08 | -0.02 | 0.02 | -0.29 | -0.13 | | RDFKGL | -0.36 | -0.76 | 0.04 | 0.23 | 0.06 | 0.03 | -0.03 | 0.01 | 0.3 | 0.14 | | RDL2 | 0.5 | 0.5 | -0.34 | -0.01 | -0.27 | 0.29 | 0.01 | 0.01 | -0.04 | 0.24 |
Coh-Metrix Model 1e
This model used principal components scores from 7 min narrative writing samples in winter [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-------|:--------|:-------|:-------|:--------| | -10.9566 | 0.338 | 0.385 | 0.5608 | -0.2444 | 0.0443 | 0.0047 | -0.0256 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:-----|:------|:------|:------| | PC1 | 15.14 | 27.79 | 9.8 | 0 | 4.09 | 18.64 | 24.42 | 21.1 | | PC4 | 9.3 | 10.16 | 11.41 | 0 | 4.87 | 7.73 | 13.26 | 10.85 | | PC7 | 8.4 | 8.15 | 9.97 | 0 | 6.48 | 6.21 | 19.43 | 10.26 | | PC11 | 6.67 | 7.23 | 6.54 | 0 | 5.69 | 5.83 | 11.19 | 10.26 | | PC9 | 6.08 | 3.67 | 9.2 | 0 | 6.76 | 2 | 1.16 | 1.97 | | PC10 | 5.49 | 3.61 | 7.03 | 0 | 5.42 | 3.37 | 8.76 | 9.47 | | PC28 | 4.63 | 2.51 | 3.16 | 0 | 9.22 | 3.54 | 0 | 10.26 | | PC16 | 3.93 | 2.03 | 4.15 | 0 | 5.33 | 2.65 | 0 | 10.26 | | PC12 | 3.92 | 2.11 | 4.88 | 0 | 4.59 | 1.8 | 0 | 8.88 | | PC33 | 3.09 | 2.27 | 1.71 | 0 | 7.03 | 2.57 | 0 | 1.97 | | PC17 | 2.46 | 1.81 | 2.53 | 0 | 3.42 | 2.91 | 0 | 0.59 | | PC31 | 2.32 | 2.4 | 1.32 | 0 | 4.7 | 1.59 | 0 | 0 | | PC20 | 2.3 | 4.01 | 1.31 | 0 | 1.99 | 2.63 | 0 | 0 | | PC18 | 2.29 | 1.13 | 2.55 | 0 | 3.68 | 2.3 | 16.31 | 0.59 | | PC14 | 2.21 | 2.99 | 1.75 | 0 | 1.83 | 3.19 | 0 | 0.79 | | PC22 | 2.1 | 0.93 | 2.13 | 0 | 3.83 | 2.78 | 0 | 0 | | PC25 | 2.05 | 0.96 | 2.05 | 0 | 4.55 | 0 | 0 | 1.38 | | PC19 | 2.02 | 1.4 | 2.05 | 0 | 2.95 | 2.31 | 0 | 0.99 | | PC6 | 1.88 | 2.76 | 1.76 | 0 | 0.89 | 2.87 | 0 | 0 | | PC3 | 1.79 | 0.55 | 3.72 | 0 | 0.94 | 1 | 0 | 0 | | PC2 | 1.49 | 0.77 | 2.52 | 0 | 0.49 | 2.87 | 0.24 | 0 | | PC15 | 1.38 | 0 | 2.17 | 0 | 2.43 | 0.98 | 0 | 0 | | PC8 | 1.38 | 1.43 | 1.51 | 0 | 0.9 | 2.69 | 0 | 0 | | PC13 | 1.31 | 1.74 | 1.23 | 0 | 1.17 | 1.25 | 0 | 0 | | PC26 | 1.13 | 1.03 | 0.71 | 0 | 1.66 | 2.56 | 0 | 0 | | PC32 | 1.03 | 2.12 | 0.2 | 0 | 0.73 | 2.06 | 0 | 0 | | PC29 | 0.93 | 0.43 | 0.75 | 0 | 2.25 | 0.69 | 0 | 0 | | PC34 | 0.68 | 1.1 | 0.12 | 0 | 0.53 | 2.38 | 0 | 0 | | PC21 | 0.58 | 0.44 | 0.44 | 0 | 0.59 | 1.47 | 5.24 | 0.39 | | PC5 | 0.5 | 0.16 | 0.73 | 0 | 0.16 | 1.73 | 0 | 0 | | PC27 | 0.44 | 0.88 | 0.15 | 0 | 0.22 | 0.95 | 0 | 0 | | PC23 | 0.37 | 0.46 | 0.17 | 0 | 0.16 | 1.67 | 0 | 0 | | PC24 | 0.36 | 0.15 | 0.28 | 0 | 0.43 | 1.4 | 0 | 0 | | PC30 | 0.36 | 0.82 | 0 | 0 | 0 | 1.41 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components are displayed.
| Variable | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 17.16 | 14.47 | 6.31 | 5.42 | 5.10 | 5.09 | 4.07 | 3.84 | 3.40 | 3.20 | | Proportion Var | 0.18 | 0.15 | 0.07 | 0.06 | 0.05 | 0.05 | 0.04 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.18 | 0.33 | 0.39 | 0.45 | 0.50 | 0.55 | 0.59 | 0.63 | 0.67 | 0.70 | | Proportion Explained | 0.25 | 0.21 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 | 0.05 | | Cumulative Proportion | 0.25 | 0.46 | 0.56 | 0.64 | 0.71 | 0.79 | 0.85 | 0.90 | 0.95 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 | | DESWC | 0.18 | 0.16 | -0.09 | 0.21 | 0.05 | -0.09 | 0.19 | 0.21 | 0.63 | 0.38 | | DESPL | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 | | DESSL | -0.81 | -0.25 | -0.04 | 0.15 | 0 | -0.07 | 0.1 | 0.12 | 0.34 | 0.01 | | DESSLd | -0.04 | 0.46 | -0.08 | 0.14 | 0.1 | -0.15 | 0.07 | 0.17 | 0.01 | 0.51 | | DESWLsy | 0.14 | 0.21 | 0.02 | -0.1 | -0.16 | 0.84 | -0.05 | -0.06 | -0.06 | -0.05 | | DESWLsyd | 0.13 | 0.17 | -0.07 | -0.04 | -0.14 | 0.79 | -0.08 | 0.03 | -0.14 | -0.04 | | DESWLlt | 0.22 | 0.12 | 0.21 | -0.04 | -0.01 | 0.83 | -0.1 | 0.02 | 0.02 | -0.05 | | DESWLltd | 0.18 | 0.13 | -0.05 | -0.07 | -0.17 | 0.67 | -0.11 | 0.11 | -0.13 | -0.09 | | PCNARz | 0.33 | 0.64 | -0.37 | 0.18 | 0 | -0.25 | 0.34 | 0.28 | -0.03 | 0.05 | | PCNARp | 0.42 | 0.53 | -0.34 | 0.22 | -0.04 | -0.17 | 0.23 | 0.28 | 0 | 0.04 | | PCSYNz | 0.9 | -0.07 | 0.06 | -0.01 | 0.11 | 0.14 | -0.05 | -0.09 | -0.14 | -0.18 | | PCSYNp | 0.85 | -0.16 | 0.07 | 0.14 | 0.08 | 0.17 | -0.04 | -0.07 | 0.01 | -0.15 | | PCCNCz | -0.56 | -0.3 | 0.65 | -0.14 | 0.17 | -0.11 | -0.1 | 0.03 | -0.04 | -0.19 | | PCCNCp | -0.42 | -0.11 | 0.69 | 0.05 | 0.09 | -0.03 | -0.06 | 0.13 | 0.09 | -0.16 | | PCREFz | -0.36 | 0.73 | -0.06 | -0.07 | -0.02 | -0.19 | -0.04 | 0.08 | 0.16 | -0.46 | | PCREFp | -0.42 | 0.45 | -0.14 | -0.17 | -0.08 | -0.26 | 0 | 0.07 | 0.18 | -0.46 | | PCDCz | 0.14 | 0.24 | -0.15 | 0.86 | 0.18 | 0.01 | 0.13 | 0.04 | 0.15 | 0.06 | | PCDCp | 0.19 | 0.2 | -0.14 | 0.82 | 0.21 | 0.02 | 0.09 | -0.01 | 0.09 | 0.03 | | PCVERBz | -0.56 | -0.65 | -0.12 | 0.01 | -0.04 | -0.12 | -0.39 | 0.14 | 0.05 | 0 | | PCVERBp | -0.33 | -0.56 | -0.15 | 0.09 | 0.05 | -0.14 | -0.51 | 0.17 | -0.07 | 0.13 | | PCCONNz | 0.05 | -0.04 | -0.1 | 0.13 | -0.93 | 0.06 | -0.11 | -0.02 | 0.07 | -0.03 | | PCCONNp | 0.04 | -0.03 | -0.02 | -0.08 | -0.75 | 0 | -0.13 | -0.09 | -0.13 | -0.04 | | PCTEMPz | 0.68 | 0.62 | -0.03 | 0.12 | 0.05 | 0.15 | 0 | 0.11 | -0.07 | 0.25 | | PCTEMPp | 0.42 | 0.46 | 0.13 | 0.08 | 0.04 | 0.03 | 0.02 | 0.21 | -0.11 | 0.17 | | CRFNO1 | -0.11 | 0.73 | 0.26 | 0.11 | -0.01 | 0.12 | -0.16 | -0.08 | 0.1 | 0.08 | | CRFAO1 | 0.33 | 0.86 | -0.07 | 0.05 | 0.01 | 0.1 | 0.01 | 0.09 | -0.03 | -0.04 | | CRFSO1 | -0.1 | 0.77 | 0.27 | 0.09 | -0.03 | 0.16 | -0.1 | -0.12 | 0.12 | 0.14 | | CRFNOa | -0.16 | 0.77 | 0.25 | 0.07 | 0.05 | 0.07 | -0.12 | -0.07 | 0.13 | 0.12 | | CRFAOa | 0.31 | 0.88 | -0.08 | 0.03 | 0.01 | 0.1 | 0.03 | 0.1 | -0.06 | -0.01 | | CRFSOa | -0.14 | 0.78 | 0.25 | 0.08 | 0.02 | 0.13 | -0.1 | -0.11 | 0.11 | 0.14 | | CRFCWO1 | 0.24 | 0.86 | -0.11 | 0.03 | 0.05 | 0.05 | -0.05 | 0.08 | 0 | -0.19 | | CRFCWO1d | 0.8 | 0.14 | -0.04 | 0.01 | -0.06 | 0.1 | 0.01 | 0.07 | 0.14 | 0.03 | | CRFCWOa | 0.21 | 0.89 | -0.14 | 0.01 | 0.06 | 0.01 | -0.02 | 0.09 | -0.01 | -0.16 | | CRFCWOad | 0.82 | 0.2 | -0.03 | -0.02 | -0.07 | 0.02 | 0.05 | 0.09 | 0.17 | 0.13 | | CRFANP1 | 0.47 | 0.66 | -0.19 | 0.05 | 0.05 | 0.06 | 0.02 | 0.18 | -0.04 | -0.07 | | CRFANPa | 0.28 | 0.71 | -0.21 | 0.03 | 0.04 | 0.11 | 0.03 | 0.18 | -0.16 | -0.07 | | LSASS1 | 0.22 | 0.83 | 0.09 | 0 | -0.01 | 0.04 | 0.03 | 0.04 | 0.17 | 0.03 | | LSASS1d | 0.71 | 0.21 | 0.01 | -0.04 | -0.09 | 0.03 | 0.04 | 0.08 | 0.26 | 0.14 | | LSASSp | 0.17 | 0.87 | 0.06 | -0.01 | 0.04 | 0 | 0.06 | 0 | 0.17 | 0.02 | | LSASSpd | 0.72 | 0.28 | 0.04 | -0.08 | -0.09 | 0.07 | 0.06 | 0.05 | 0.32 | 0.1 | | LSAGN | 0.75 | 0.52 | 0.02 | -0.01 | -0.02 | 0.04 | 0.1 | 0.06 | 0.28 | 0.06 | | LSAGNd | 0.48 | 0.78 | 0.02 | 0.03 | 0.03 | 0.05 | 0.06 | 0.05 | 0.16 | 0.14 | | LDTTRc | -0.07 | -0.35 | 0.08 | -0.11 | -0.11 | 0.09 | 0.1 | 0.03 | -0.74 | 0.19 | | LDTTRa | 0.05 | -0.3 | 0.3 | -0.02 | -0.14 | 0.4 | 0.06 | -0.1 | -0.66 | 0.04 | | LDMTLD | 0.24 | -0.14 | 0.17 | 0.28 | -0.11 | 0.51 | 0.27 | 0.03 | 0.02 | 0.41 | | CNCAll | -0.12 | 0.07 | -0.06 | 0.32 | 0.86 | -0.16 | 0.08 | -0.15 | 0.02 | -0.07 | | CNCCaus | 0.03 | 0.08 | 0.03 | 0.84 | -0.1 | -0.02 | -0.14 | 0.04 | -0.06 | 0.06 | | CNCLogic | -0.07 | -0.09 | -0.31 | 0.62 | 0.35 | -0.01 | 0.33 | -0.07 | 0.19 | -0.13 | | CNCADC | -0.04 | -0.12 | -0.05 | 0.1 | 0.36 | 0.13 | 0.52 | 0.33 | -0.02 | 0.06 | | CNCTemp | 0.14 | 0.09 | -0.31 | 0.08 | 0.34 | -0.01 | 0.32 | -0.21 | 0.2 | -0.23 | | CNCTempx | 0.15 | -0.11 | 0.03 | 0.39 | 0.07 | 0.02 | 0.05 | -0.02 | -0.22 | -0.1 | | CNCAdd | -0.14 | 0.01 | 0.04 | -0.19 | 0.89 | -0.14 | -0.04 | -0.08 | -0.04 | -0.01 | | CNCPos | -0.08 | 0.11 | -0.05 | 0.33 | 0.73 | -0.2 | -0.16 | -0.26 | 0 | -0.1 | | CNCNeg | -0.07 | -0.13 | -0.11 | 0.03 | 0.37 | 0.11 | 0.55 | 0.31 | 0.02 | 0.01 | | SMCAUSv | 0.64 | 0.04 | 0.07 | 0.05 | -0.09 | 0.14 | -0.05 | 0.06 | -0.15 | -0.33 | | SMCAUSvp | 0.49 | 0.12 | 0.11 | 0.55 | -0.14 | 0.06 | -0.03 | 0.09 | -0.06 | -0.17 | | SMINTEp | 0.66 | 0.08 | 0.13 | 0.05 | -0.18 | 0.11 | 0.09 | 0.03 | 0.12 | -0.27 | | SMCAUSr | -0.36 | 0.16 | -0.05 | 0.58 | -0.13 | -0.13 | 0.05 | 0.08 | 0.18 | 0.3 | | SMINTEr | -0.43 | 0.12 | -0.08 | 0.68 | -0.06 | -0.07 | -0.06 | 0.04 | 0.12 | 0.25 | | SMCAUSlsa | -0.01 | 0.06 | -0.17 | 0.13 | 0.12 | 0.04 | -0.54 | 0.07 | 0.29 | -0.01 | | SMCAUSwn | 0.28 | 0.09 | -0.01 | 0.23 | 0.02 | 0.22 | -0.35 | 0.54 | 0.22 | 0.22 | | SMTEMP | 0.69 | 0.62 | -0.03 | 0.11 | 0.04 | 0.15 | -0.01 | 0.11 | -0.07 | 0.24 | | SYNLE | -0.13 | -0.11 | -0.1 | -0.14 | 0.05 | 0.15 | 0.11 | 0.03 | 0.18 | -0.06 | | SYNNP | 0 | -0.05 | 0.29 | -0.17 | 0 | 0.38 | -0.21 | -0.5 | 0.09 | 0.35 | | SYNMEDpos | 0.66 | 0.51 | -0.06 | 0.11 | 0.04 | 0.13 | 0.05 | 0.1 | -0.11 | 0.4 | | SYNMEDwrd | 0.71 | 0.53 | -0.04 | 0.12 | 0.03 | 0.17 | 0.01 | 0.09 | -0.07 | 0.35 | | SYNMEDlem | 0.7 | 0.54 | -0.04 | 0.12 | 0.03 | 0.16 | 0.02 | 0.09 | -0.08 | 0.35 | | SYNSTRUTa | 0.81 | 0.14 | -0.08 | 0.08 | -0.12 | 0.17 | -0.06 | 0.05 | -0.05 | -0.13 | | SYNSTRUTt | 0.85 | 0.15 | -0.05 | 0.06 | -0.12 | 0.16 | -0.05 | 0.02 | -0.04 | -0.15 | | DRNP | -0.04 | -0.1 | 0.15 | 0.02 | -0.43 | -0.03 | -0.26 | -0.02 | -0.12 | -0.28 | | DRVP | -0.05 | 0.07 | -0.08 | -0.14 | -0.05 | -0.14 | 0 | 0.65 | 0 | -0.02 | | DRAP | 0.09 | 0 | -0.21 | 0.22 | 0.26 | -0.09 | 0.42 | -0.01 | 0.06 | -0.36 | | DRPP | 0.11 | -0.02 | 0.05 | 0.09 | -0.01 | 0.06 | -0.45 | -0.03 | 0 | 0.05 | | DRNEG | 0.03 | 0.03 | -0.14 | 0.08 | 0.1 | -0.15 | 0.53 | -0.11 | 0.03 | 0.09 | | WRDNOUN | 0.04 | -0.06 | 0.49 | -0.12 | -0.16 | 0.28 | -0.41 | -0.4 | 0.02 | 0.06 | | WRDVERB | 0.17 | 0.17 | 0.05 | -0.13 | -0.06 | 0 | 0.17 | 0.65 | -0.12 | 0.03 | | WRDADJ | 0.05 | 0.01 | -0.09 | 0.04 | -0.06 | 0.37 | 0.01 | -0.26 | -0.03 | 0.29 | | WRDADV | 0.04 | 0.04 | -0.24 | 0.38 | 0.29 | 0 | 0.47 | -0.08 | 0.22 | -0.19 | | WRDPRO | -0.11 | 0.01 | -0.27 | -0.03 | -0.23 | -0.54 | 0.37 | 0.2 | -0.13 | -0.19 | | WRDPRP3s | 0.05 | 0.01 | -0.09 | 0.14 | -0.03 | -0.03 | 0.45 | -0.03 | 0.1 | 0.22 | | WRDPRP3p | -0.23 | -0.17 | -0.12 | 0.23 | 0.04 | 0.13 | -0.02 | 0.04 | 0.19 | 0.04 | | WRDFRQc | -0.12 | -0.25 | -0.58 | 0.26 | -0.15 | -0.24 | -0.05 | 0.31 | 0.05 | 0.06 | | WRDFRQa | -0.23 | -0.19 | -0.49 | -0.11 | 0.06 | -0.47 | -0.23 | -0.06 | -0.26 | 0.07 | | WRDFRQmc | 0.66 | -0.02 | -0.21 | 0.05 | 0.11 | 0.1 | -0.08 | 0.15 | -0.07 | 0.18 | | WRDAOAc | 0.12 | 0.14 | 0.17 | 0.16 | 0.04 | 0.32 | 0.03 | -0.09 | 0.07 | 0.13 | | WRDFAMc | -0.09 | -0.12 | -0.47 | 0.22 | -0.18 | -0.23 | -0.08 | 0.22 | 0.12 | -0.06 | | WRDCNCc | 0.03 | 0.02 | 0.83 | -0.12 | 0.04 | -0.03 | -0.06 | 0.02 | -0.09 | 0.05 | | WRDIMGc | 0.08 | 0.07 | 0.88 | -0.06 | -0.05 | 0 | -0.11 | -0.02 | -0.15 | 0.03 | | WRDMEAc | 0.06 | 0.04 | 0.71 | 0.06 | -0.19 | 0.03 | -0.13 | 0.17 | -0.2 | 0.03 | | WRDPOLc | 0.05 | 0.15 | 0.08 | 0.3 | -0.06 | 0.01 | -0.27 | 0.67 | 0.18 | 0.11 | | WRDHYPn | 0.24 | 0.16 | 0.25 | 0.27 | -0.11 | -0.07 | 0.07 | 0.3 | 0.17 | -0.18 | | WRDHYPv | 0.21 | 0 | 0.23 | 0.1 | -0.12 | 0.2 | -0.02 | 0.54 | 0.31 | -0.01 | | WRDHYPnv | 0.13 | 0.08 | 0.61 | 0.11 | -0.23 | 0.22 | -0.34 | 0.12 | 0.24 | -0.05 | | RDFRE | 0.82 | 0.19 | 0.04 | -0.11 | 0.05 | -0.19 | -0.09 | -0.1 | -0.32 | 0 | | RDFKGL | -0.81 | -0.23 | -0.04 | 0.15 | -0.02 | 0.02 | 0.1 | 0.11 | 0.34 | 0.01 | | RDL2 | 0.44 | 0.48 | -0.44 | 0.2 | -0.1 | -0.03 | -0.09 | 0.25 | 0.01 | -0.15 |
Coh-Metrix Model 1f
This model used principal component scores from 7 min narrative writing samples in spring [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model gbm = stochastic gradient boosted trees pls = partial least squares regression svm = support vector machines enet = elastic net regression rf = random forest regression mars = bagged multivariate adaptive regression splines cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:------|:--------| | -16.5845 | 0.1071 | 0.5091 | 0.6984 | -0.2708 | -0.0323 | 0.036 | -0.0221 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
PC1 = scores on 1st principal component extracted, ...
Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0).
| Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:------|:------|:------|:-----| | PC7 | 14.02 | 14.78 | 14.62 | 0 | 12.36 | 11.93 | 19.81 | 10 | | PC1 | 13.63 | 30.49 | 12.95 | 0 | 6.54 | 26.18 | 26.11 | 20 | | PC8 | 12.82 | 12.75 | 13.42 | 0 | 11.91 | 8.44 | 15.73 | 10 | | PC6 | 10.25 | 12.69 | 10.89 | 0 | 7.93 | 8.88 | 10.09 | 16.6 | | PC3 | 5.8 | 2.48 | 8 | 0 | 3.23 | 3.41 | 0.8 | 10 | | PC9 | 4.73 | 3.43 | 5.34 | 0 | 4.67 | 2.71 | 0 | 6.6 | | PC4 | 4.32 | 2.1 | 5.78 | 0 | 2.94 | 0.55 | 0.09 | 6.6 | | PC16 | 3.92 | 1.27 | 3.57 | 0 | 6.22 | 0.83 | 0 | 3.4 | | PC20 | 3.79 | 3.76 | 2.76 | 0 | 6.1 | 2.84 | 0 | 6.6 | | PC24 | 3.35 | 2.04 | 2.26 | 0 | 6.11 | 5.13 | 0 | 3.4 | | PC21 | 2.5 | 0 | 1.85 | 0 | 3.81 | 0.86 | 7.63 | 3.4 | | PC32 | 2.2 | 0.74 | 1.2 | 0 | 5.05 | 1.77 | 0 | 0 | | PC18 | 2.06 | 0.28 | 1.83 | 0 | 3.34 | 0.48 | 0 | 3.4 | | PC15 | 2.06 | 0.86 | 2.12 | 0 | 2.84 | 1.45 | 0 | 0 | | PC14 | 1.87 | 0.22 | 2.13 | 0 | 2.42 | 0.49 | 0 | 0 | | PC19 | 1.76 | 0.83 | 1.58 | 0 | 2.82 | 1.1 | 0 | 0 | | PC10 | 1.73 | 0.88 | 2.1 | 0 | 1.55 | 3.01 | 0 | 0 | | PC31 | 1.69 | 0.48 | 1.03 | 0 | 3.81 | 0.38 | 0 | 0 | | PC12 | 1.2 | 0.41 | 1.48 | 0 | 1.15 | 1.21 | 0 | 0 | | PC30 | 1.06 | 0.58 | 0.76 | 0 | 1.97 | 1.51 | 0 | 0 | | PC22 | 1.02 | 0.47 | 0.99 | 0 | 1.56 | 0.27 | 0 | 0 | | PC17 | 0.83 | 1.24 | 0.22 | 0 | 0 | 3.14 | 13.07 | 0 | | PC26 | 0.72 | 0.25 | 0.7 | 0 | 1.14 | 0.26 | 0 | 0 | | PC5 | 0.58 | 1.09 | 0.47 | 0 | 0 | 2.15 | 4.42 | 0 | | PC25 | 0.49 | 0.83 | 0.54 | 0 | 0.47 | 0.05 | 0 | 0 | | PC13 | 0.46 | 0.3 | 0.67 | 0 | 0.07 | 2.34 | 0 | 0 | | PC2 | 0.35 | 0.99 | 0.31 | 0 | 0 | 0 | 2.25 | 0 | | PC11 | 0.23 | 1.78 | 0 | 0 | 0 | 3.42 | 0 | 0 | | PC28 | 0.21 | 0.07 | 0.31 | 0 | 0 | 1.49 | 0 | 0 | | PC29 | 0.11 | 0.55 | 0.08 | 0 | 0 | 0.75 | 0 | 0 | | PC23 | 0.09 | 0.33 | 0.01 | 0 | 0 | 2.16 | 0 | 0 | | PC33 | 0.07 | 0.57 | 0.03 | 0 | 0 | 0.32 | 0 | 0 | | PC27 | 0.05 | 0.43 | 0 | 0 | 0 | 0.5 | 0 | 0 |
Proportion of Variance by Varimax Rotated Component (RC)
Due to space limitations, loadings for only the first ten principal components are displayed.
| Variable | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 18.09 | 16.04 | 6.14 | 4.86 | 4.81 | 4.44 | 4.37 | 4.20 | 3.86 | 2.98 | | Proportion Var | 0.19 | 0.17 | 0.06 | 0.05 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.19 | 0.35 | 0.42 | 0.47 | 0.51 | 0.56 | 0.61 | 0.65 | 0.69 | 0.72 | | Proportion Explained | 0.26 | 0.23 | 0.09 | 0.07 | 0.07 | 0.06 | 0.06 | 0.06 | 0.06 | 0.04 | | Cumulative Proportion | 0.26 | 0.49 | 0.58 | 0.65 | 0.72 | 0.78 | 0.84 | 0.90 | 0.96 | 1.00 |
Varimax Rotated Loadings
| Metric | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 | | DESWC | 0.17 | 0.32 | 0.15 | 0.44 | -0.02 | 0.18 | 0.04 | -0.17 | 0.38 | 0.43 | | DESPL | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 | | DESSL | -0.75 | -0.24 | 0.1 | 0.12 | -0.09 | 0.12 | -0.11 | -0.15 | 0.25 | 0.25 | | DESSLd | 0.02 | 0.66 | 0.16 | 0.1 | -0.03 | 0.1 | 0.07 | 0.06 | -0.06 | 0.2 | | DESWLsy | 0.09 | -0.08 | 0 | -0.05 | -0.14 | -0.16 | 0.86 | 0.02 | 0.06 | -0.13 | | DESWLsyd | 0.03 | -0.07 | 0 | -0.04 | -0.07 | 0.06 | 0.82 | -0.1 | -0.04 | -0.09 | | DESWLlt | 0.14 | -0.17 | -0.05 | -0.01 | -0.12 | -0.32 | 0.72 | 0.1 | 0.17 | -0.07 | | DESWLltd | 0.12 | 0.06 | 0.09 | -0.05 | 0.15 | -0.05 | 0.74 | 0.06 | -0.13 | 0.18 | | PCNARz | 0.35 | 0.7 | 0.16 | 0.21 | 0.45 | 0.21 | -0.14 | 0.03 | -0.03 | 0.17 | | PCNARp | 0.47 | 0.57 | 0.1 | 0.27 | 0.41 | 0.2 | -0.13 | -0.02 | -0.02 | 0.16 | | PCSYNz | 0.89 | -0.07 | -0.03 | -0.17 | 0.08 | -0.17 | 0.09 | 0.06 | -0.07 | -0.23 | | PCSYNp | 0.88 | -0.13 | 0.05 | -0.15 | 0.01 | -0.14 | 0.02 | 0.04 | 0.08 | -0.11 | | PCCNCz | -0.64 | -0.38 | -0.18 | -0.4 | -0.06 | -0.31 | -0.04 | -0.22 | 0.07 | 0.23 | | PCCNCp | -0.48 | -0.13 | -0.1 | -0.45 | -0.04 | -0.36 | 0.03 | -0.14 | 0.05 | 0.22 | | PCREFz | -0.4 | 0.72 | 0.01 | -0.14 | 0.29 | 0.09 | -0.24 | -0.13 | 0.1 | -0.1 | | PCREFp | -0.49 | 0.4 | -0.04 | -0.04 | 0.4 | 0.15 | -0.24 | -0.1 | 0.05 | -0.05 | | PCDCz | 0.17 | 0.3 | 0.9 | 0.06 | 0.06 | 0.07 | 0 | 0.06 | 0.13 | 0.02 | | PCDCp | 0.18 | 0.32 | 0.79 | 0.15 | 0.09 | 0.08 | 0.05 | 0.04 | 0.09 | 0.07 | | PCVERBz | -0.62 | -0.62 | -0.01 | -0.1 | -0.18 | 0.3 | -0.09 | -0.05 | 0.2 | -0.09 | | PCVERBp | -0.41 | -0.49 | 0.06 | -0.17 | -0.22 | 0.43 | -0.1 | -0.03 | 0.16 | -0.11 | | PCCONNz | 0.24 | 0.02 | 0.09 | -0.06 | -0.06 | 0.08 | -0.01 | 0.9 | 0.11 | 0.03 | | PCCONNp | 0.13 | -0.07 | 0.03 | -0.24 | 0.12 | -0.05 | -0.12 | 0.75 | 0.11 | -0.14 | | PCTEMPz | 0.72 | 0.62 | 0.09 | 0.09 | 0.01 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 | | PCTEMPp | 0.47 | 0.54 | 0.14 | 0.17 | -0.11 | 0.02 | 0.17 | 0.07 | 0 | 0.17 | | CRFNO1 | 0.07 | 0.7 | 0.15 | 0.07 | -0.37 | -0.13 | -0.22 | -0.03 | 0.11 | -0.07 | | CRFAO1 | 0.36 | 0.85 | 0.06 | 0.07 | 0.17 | 0.02 | 0.11 | 0 | 0.06 | 0.06 | | CRFSO1 | 0.09 | 0.77 | 0.1 | -0.01 | -0.32 | -0.16 | -0.22 | -0.07 | 0.06 | -0.11 | | CRFNOa | -0.03 | 0.77 | 0.08 | 0.01 | -0.32 | -0.08 | -0.21 | -0.05 | 0.08 | -0.1 | | CRFAOa | 0.3 | 0.86 | 0.07 | 0.08 | 0.25 | 0.07 | 0.12 | 0.02 | 0.05 | 0.06 | | CRFSOa | -0.01 | 0.81 | 0.07 | -0.04 | -0.28 | -0.12 | -0.2 | -0.09 | 0.05 | -0.1 | | CRFCWO1 | 0.27 | 0.89 | 0.08 | -0.08 | 0.17 | 0.05 | -0.01 | -0.04 | 0.08 | -0.03 | | CRFCWO1d | 0.81 | 0.27 | -0.05 | 0.07 | -0.03 | 0.02 | 0.02 | -0.04 | 0.03 | 0.06 | | CRFCWOa | 0.18 | 0.9 | 0.08 | -0.07 | 0.19 | 0.07 | 0 | -0.04 | 0.07 | -0.04 | | CRFCWOad | 0.83 | 0.33 | -0.02 | 0.12 | -0.01 | 0.02 | 0.02 | 0.02 | 0.03 | 0.1 | | CRFANP1 | 0.38 | 0.8 | 0.09 | 0.08 | 0.29 | 0.05 | 0.12 | 0.03 | 0.03 | 0.08 | | CRFANPa | 0.18 | 0.84 | 0.11 | 0.04 | 0.29 | 0.11 | 0.15 | 0.05 | 0.02 | 0.04 | | LSASS1 | 0.29 | 0.81 | 0.07 | -0.08 | 0.11 | -0.13 | -0.14 | -0.03 | 0.03 | 0.01 | | LSASS1d | 0.67 | 0.34 | 0.01 | 0.13 | -0.09 | -0.13 | -0.16 | 0.02 | 0.03 | 0.1 | | LSASSp | 0.19 | 0.85 | 0.06 | -0.1 | 0.11 | -0.07 | -0.09 | -0.03 | 0.03 | -0.02 | | LSASSpd | 0.76 | 0.38 | -0.02 | 0.09 | -0.12 | -0.1 | -0.1 | 0.06 | 0.05 | 0.09 | | LSAGN | 0.73 | 0.58 | -0.01 | 0.13 | 0.09 | -0.05 | -0.01 | -0.04 | 0.12 | 0.04 | | LSAGNd | 0.59 | 0.72 | 0.03 | 0.02 | -0.05 | -0.08 | -0.07 | 0 | 0.04 | 0.02 | | LDTTRc | -0.11 | -0.48 | 0.03 | -0.19 | 0.15 | -0.09 | 0.22 | 0.27 | -0.56 | 0.16 | | LDTTRa | -0.03 | -0.46 | 0.01 | -0.15 | -0.02 | -0.31 | 0.18 | 0.5 | -0.42 | -0.11 | | LDMTLD | 0.19 | -0.13 | 0.15 | 0.25 | -0.03 | -0.16 | 0.28 | 0.37 | -0.19 | 0.32 | | CNCAll | -0.3 | 0.07 | 0.62 | -0.12 | 0.17 | -0.01 | -0.06 | -0.62 | -0.02 | -0.09 | | CNCCaus | 0.04 | 0.02 | 0.86 | -0.19 | 0 | 0.12 | 0.13 | 0.03 | -0.01 | -0.08 | | CNCLogic | -0.09 | 0.11 | 0.78 | 0.31 | 0.08 | 0.06 | -0.09 | -0.01 | -0.05 | -0.13 | | CNCADC | 0.05 | -0.11 | -0.1 | 0.71 | -0.04 | 0.06 | 0.12 | -0.24 | -0.16 | 0.05 | | CNCTemp | 0.05 | 0.24 | 0.35 | 0.16 | 0.09 | -0.15 | -0.26 | 0.14 | 0.14 | 0.03 | | CNCTempx | -0.02 | 0.11 | 0.14 | 0.09 | -0.16 | 0.33 | 0.08 | -0.01 | -0.19 | 0.25 | | CNCAdd | -0.36 | -0.05 | 0 | -0.12 | 0.11 | -0.07 | -0.09 | -0.84 | -0.11 | -0.05 | | CNCPos | -0.27 | 0.11 | 0.61 | -0.31 | 0.19 | -0.05 | -0.1 | -0.53 | 0.03 | -0.1 | | CNCNeg | 0 | -0.14 | -0.08 | 0.63 | -0.06 | 0.08 | 0.07 | -0.22 | -0.17 | 0.02 | | SMCAUSv | 0.76 | -0.13 | 0.04 | -0.2 | 0.05 | -0.1 | 0.23 | 0.11 | 0.13 | 0.06 | | SMCAUSvp | 0.58 | -0.1 | 0.55 | -0.22 | 0.01 | -0.03 | 0.22 | 0.15 | 0.13 | -0.03 | | SMINTEp | 0.62 | 0.03 | -0.13 | -0.29 | 0.33 | -0.08 | -0.06 | 0.16 | 0.08 | -0.01 | | SMCAUSr | -0.25 | 0.24 | 0.61 | 0.15 | -0.05 | 0.2 | -0.11 | 0.04 | 0.1 | 0.13 | | SMINTEr | -0.23 | 0.03 | 0.66 | 0.03 | -0.13 | 0.26 | 0.11 | -0.11 | 0.13 | 0.18 | | SMCAUSlsa | -0.16 | 0.03 | -0.08 | -0.11 | -0.21 | 0.43 | 0.17 | -0.07 | 0.38 | -0.38 | | SMCAUSwn | 0.11 | 0.06 | 0.17 | -0.08 | 0.17 | 0.12 | 0.02 | 0.09 | 0.71 | 0.06 | | SMTEMP | 0.73 | 0.62 | 0.07 | 0.08 | 0 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 | | SYNLE | -0.17 | -0.03 | -0.01 | 0.03 | 0.02 | -0.18 | 0.08 | 0.23 | 0.35 | 0.02 | | SYNNP | -0.03 | -0.11 | -0.04 | -0.14 | -0.65 | -0.04 | 0.11 | 0.03 | 0 | 0.01 | | SYNMEDpos | 0.74 | 0.57 | 0.07 | 0.11 | 0.01 | 0.02 | 0.12 | 0.04 | 0.04 | 0.12 | | SYNMEDwrd | 0.76 | 0.56 | 0.07 | 0.11 | 0 | 0 | 0.14 | 0.07 | 0.05 | 0.13 | | SYNMEDlem | 0.76 | 0.55 | 0.07 | 0.11 | -0.01 | 0 | 0.14 | 0.08 | 0.05 | 0.14 | | SYNSTRUTa | 0.77 | 0.18 | -0.04 | -0.11 | 0.04 | 0.05 | 0.05 | 0.15 | 0.07 | -0.01 | | SYNSTRUTt | 0.82 | 0.16 | -0.08 | -0.11 | 0.04 | 0.04 | 0.01 | 0.18 | 0.08 | 0.01 | | DRNP | -0.17 | 0.01 | -0.27 | -0.2 | -0.15 | 0.04 | -0.25 | 0.33 | -0.36 | 0.1 | | DRVP | 0.16 | 0.04 | 0.09 | -0.05 | 0.64 | -0.04 | 0.03 | -0.04 | 0.22 | -0.22 | | DRAP | -0.1 | 0.09 | 0.25 | 0.55 | 0.08 | -0.01 | -0.11 | -0.01 | 0.25 | -0.07 | | DRPP | 0.04 | -0.06 | -0.09 | -0.13 | -0.13 | 0 | 0.02 | 0.18 | 0 | 0.66 | | DRNEG | 0.03 | 0.03 | -0.11 | 0.5 | 0.05 | 0.06 | -0.08 | 0.13 | -0.12 | -0.06 | | WRDNOUN | 0.05 | -0.06 | -0.13 | -0.26 | -0.71 | -0.31 | 0.03 | 0.24 | 0.05 | 0.02 | | WRDVERB | 0.23 | 0.1 | 0.08 | -0.06 | 0.61 | -0.04 | 0.07 | 0.07 | 0.21 | 0.07 | | WRDADJ | 0.09 | -0.11 | -0.09 | 0.09 | -0.24 | 0.1 | 0.31 | 0.1 | -0.1 | -0.56 | | WRDADV | -0.09 | 0.14 | 0.33 | 0.71 | 0.06 | 0.06 | -0.1 | -0.05 | 0.19 | -0.08 | | WRDPRO | -0.04 | 0.14 | -0.08 | 0 | 0.61 | 0.27 | -0.34 | 0.06 | -0.27 | 0.02 | | WRDPRP3s | 0.03 | 0.12 | 0 | 0.21 | 0.06 | -0.03 | -0.04 | 0.08 | 0.01 | 0.09 | | WRDPRP3p | -0.06 | -0.18 | -0.05 | -0.03 | 0 | -0.17 | 0.07 | 0.18 | 0.04 | -0.3 | | WRDFRQc | -0.08 | -0.06 | 0.23 | 0.22 | 0.16 | 0.75 | -0.34 | 0.07 | 0.04 | -0.14 | | WRDFRQa | -0.26 | -0.09 | 0.17 | 0.07 | 0.13 | 0.58 | -0.43 | -0.17 | -0.25 | 0.22 | | WRDFRQmc | 0.79 | 0.09 | 0.01 | 0.04 | 0.12 | 0.07 | 0.05 | 0 | 0.09 | 0.08 | | WRDAOAc | 0.22 | -0.06 | 0.11 | 0.24 | -0.11 | -0.2 | 0.07 | 0.1 | 0.07 | -0.14 | | WRDFAMc | -0.02 | -0.03 | 0.12 | 0.02 | 0.12 | 0.67 | -0.08 | 0.09 | 0.19 | 0.09 | | WRDCNCc | 0.04 | 0.03 | -0.31 | -0.41 | -0.07 | -0.5 | 0.03 | 0.02 | 0.1 | 0.51 | | WRDIMGc | 0.01 | -0.02 | -0.38 | -0.48 | -0.16 | -0.48 | 0.12 | -0.01 | 0.05 | 0.38 | | WRDMEAc | 0.11 | -0.11 | -0.36 | -0.47 | -0.13 | -0.18 | 0.25 | -0.1 | 0.09 | 0.37 | | WRDPOLc | 0.15 | 0.08 | 0.25 | -0.21 | 0.13 | 0.2 | -0.04 | 0.15 | 0.67 | -0.04 | | WRDHYPn | 0.11 | 0.25 | 0.03 | 0.04 | 0.22 | -0.32 | -0.03 | 0.07 | 0.48 | 0.07 | | WRDHYPv | 0.14 | 0.04 | -0.01 | -0.07 | 0.57 | -0.12 | 0.01 | -0.03 | 0.46 | 0.06 | | WRDHYPnv | 0.04 | 0.16 | -0.1 | -0.23 | -0.23 | -0.52 | -0.02 | 0.14 | 0.49 | 0.03 | | RDFRE | 0.79 | 0.26 | -0.08 | -0.09 | 0.14 | -0.09 | -0.13 | 0.11 | -0.23 | -0.18 | | RDFKGL | -0.75 | -0.25 | 0.1 | 0.11 | -0.11 | 0.11 | -0.03 | -0.15 | 0.26 | 0.24 | | RDL2 | 0.48 | 0.59 | 0.15 | 0.01 | 0.2 | 0.44 | -0.16 | 0.08 | 0.1 | -0.1 |
Coh-Metrix Model 2 {#cohmetrix-model-2}
General Description
Coh-Metrix Model 2 is a simplified version of Model 1. Model 2 is recommended for use over Model 1.
Coh-Metrix Model 2 is an ensemble (formed by averaging predicted quality scores) of the three sub-models described below.
Highly correlated Coh-Metrix metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019].
Coh-Metrix Model 2a
This model was trained on fall data in [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model pls = partial least squares regression rf = random forest regression mars = bagged multivariate adaptive regression splines gbm = stochastic gradient boosted trees svm = support vector machines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | rf | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------|:------|:-------| | -11.0081 | 0.1741 | 0.0413 | 0.1875 | 0.2353 | 0.206 | 0.2108 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | rf | mars | gbm | svm | cube | |:----------|:--------|:-----|:------|:------|:------|:-----|:------| | DESWC | 29.91 | 5.86 | 14.51 | 55.78 | 44.87 | 5.96 | 36.49 | | DESWLlt | 8.67 | 2.98 | 2.69 | 18.79 | 3.07 | 2.39 | 17.89 | | LDMTLD | 7.35 | 4.01 | 5.57 | 0 | 9.55 | 3.9 | 17.89 | | WRDHYPn | 7.16 | 2.9 | 2.68 | 9.55 | 3.85 | 2.29 | 17.89 | | LDTTRa | 2.84 | 2.01 | 0.97 | 11.14 | 0.26 | 1.12 | 1.05 | | CNCPos | 1.36 | 0.88 | 1.11 | 4.75 | 0.05 | 1.28 | 0.35 | | DESWLsy | 1.3 | 2.09 | 1.9 | 0 | 1.66 | 1.54 | 1.05 | | CNCTempx | 1.11 | 0.89 | 1.03 | 0 | 1.63 | 2.48 | 0.35 | | CNCLogic | 1.09 | 1.42 | 1.39 | 0 | 1.81 | 1.7 | 0.35 | | PCDCp | 1.08 | 2 | 2.56 | 0 | 1.71 | 1.36 | 0 | | DESPL | 1.04 | 3.22 | 2.18 | 0 | 0.04 | 2.15 | 0 | | DESWLltd | 1.02 | 2.3 | 1.13 | 0 | 1.26 | 1.6 | 0 | | WRDFRQa | 0.96 | 1.86 | 0.58 | 0 | 0.17 | 1.97 | 1.05 | | DESWLsyd | 0.92 | 1.97 | 1.87 | 0 | 1.21 | 1.29 | 0 | | DESSLd | 0.9 | 1.67 | 2 | 0 | 1.27 | 1.36 | 0 | | CNCTemp | 0.89 | 0.93 | 1.39 | 0 | 1.08 | 1.89 | 0.35 | | LSAGN | 0.87 | 2.55 | 1.18 | 0 | 0.22 | 1.8 | 0 | | LSASSpd | 0.86 | 1.95 | 0.86 | 0 | 0.36 | 1.79 | 0.35 | | CNCADC | 0.85 | 1.12 | 1.41 | 0 | 0.67 | 2.37 | 0 | | DRPP | 0.84 | 2.18 | 2.11 | 0 | 0.57 | 1.4 | 0 | | PCCONNz | 0.82 | 1.09 | 0.4 | 0 | 1.61 | 1.36 | 0 | | WRDPRO | 0.82 | 1.77 | 1.44 | 0 | 0.73 | 1.61 | 0 | | SYNSTRUTa | 0.8 | 0.85 | 3.07 | 0 | 0.58 | 1.4 | 0.7 | | CRFCWO1d | 0.79 | 1.67 | 1.39 | 0 | 0.43 | 1.88 | 0 | | LSASS1d | 0.78 | 1.55 | 0.48 | 0 | 0.78 | 1.72 | 0 | | SMCAUSwn | 0.76 | 1.4 | 1.04 | 0 | 0.31 | 2.16 | 0 | | SYNMEDpos | 0.75 | 1.75 | 0.6 | 0 | 0.77 | 1.36 | 0 | | SMINTEp | 0.73 | 0.75 | 0.85 | 0 | 0.79 | 1.66 | 0.35 | | LDTTRc | 0.73 | 1.82 | 0.7 | 0 | 0.72 | 1.23 | 0 | | CRFCWOad | 0.73 | 1.51 | 1.12 | 0 | 0.38 | 1.81 | 0 | | WRDVERB | 0.71 | 1.04 | 0.93 | 0 | 0.58 | 0.86 | 1.05 | | WRDFAMc | 0.7 | 1.09 | 0.5 | 0 | 1.32 | 1.08 | 0 | | WRDHYPnv | 0.69 | 1.9 | 0.91 | 0 | 0.13 | 1.22 | 0.35 | | WRDFRQmc | 0.69 | 1.35 | 2.5 | 0 | 1.3 | 0.4 | 0 | | WRDCNCc | 0.69 | 1.07 | 0.44 | 0 | 0.58 | 0.8 | 1.05 | | PCNARz | 0.68 | 1.29 | 1.29 | 0 | 0.58 | 1.1 | 0.35 | | WRDPOLc | 0.67 | 0.85 | 0.77 | 0 | 0.51 | 1.97 | 0 | | RDFRE | 0.66 | 1.29 | 0.91 | 0 | 0.76 | 1.23 | 0 | | CRFNOa | 0.63 | 0.74 | 1.34 | 0 | 0.56 | 1.67 | 0 | | PCVERBz | 0.63 | 1.55 | 0.67 | 0 | 0.28 | 1.09 | 0.35 | | LSAGNd | 0.61 | 1.5 | 0.95 | 0 | 0.24 | 1.38 | 0 | | SMCAUSvp | 0.59 | 0.93 | 0.73 | 0 | 0.36 | 1.67 | 0 | | WRDADV | 0.58 | 1.3 | 0.95 | 0 | 0.13 | 1.55 | 0 | | WRDAOAc | 0.58 | 1.66 | 0.46 | 0 | 0.27 | 1.18 | 0 | | DRNP | 0.57 | 1.73 | 1.5 | 0 | 0.27 | 0.82 | 0 | | WRDHYPv | 0.56 | 0.28 | 1.01 | 0 | 0.83 | 1.47 | 0 | | DRVP | 0.56 | 0.82 | 0.62 | 0 | 0.82 | 0.75 | 0.35 | | PCNARp | 0.54 | 1.76 | 1.29 | 0 | 0 | 1 | 0 | | CNCCaus | 0.51 | 1.11 | 0.58 | 0 | 0.11 | 1.46 | 0 | | CRFCWOa | 0.49 | 0.46 | 0.56 | 0 | 0.37 | 1.57 | 0 | | SMCAUSr | 0.47 | 1.58 | 1.07 | 0 | 0.47 | 0.29 | 0 | | SMCAUSlsa | 0.46 | 0.24 | 0.86 | 0 | 0.63 | 1.26 | 0 | | LSASSp | 0.46 | 0.79 | 1.01 | 0 | 0.15 | 1.31 | 0 | | SMINTEr | 0.45 | 1.83 | 0.9 | 0 | 0.13 | 0.45 | 0 | | DRAP | 0.44 | 0.46 | 1.01 | 0 | 0.41 | 1.17 | 0 | | DRNEG | 0.44 | 0.76 | 0.43 | 0 | 0.09 | 1.41 | 0 | | WRDNOUN | 0.43 | 0.56 | 0.96 | 0 | 0.75 | 0.66 | 0 | | WRDFRQc | 0.43 | 0.54 | 0.67 | 0 | 0.48 | 1.04 | 0 | | WRDPRP3s | 0.43 | 0.76 | 0.84 | 0 | 0.74 | 0.56 | 0 | | CRFANPa | 0.41 | 0.76 | 1.36 | 0 | 0.18 | 0.96 | 0 | | SYNLE | 0.4 | 1.13 | 0.95 | 0 | 0.1 | 0.77 | 0 | | PCTEMPp | 0.4 | 1.05 | 0.22 | 0 | 0.55 | 0.47 | 0 | | WRDADJ | 0.39 | 0.99 | 0.94 | 0 | 0.2 | 0.74 | 0 | | RDL2 | 0.39 | 0.31 | 1.06 | 0 | 0.39 | 1.09 | 0 | | WRDIMGc | 0.37 | 0.14 | 0.74 | 0 | 0.3 | 0.93 | 0.35 | | PCVERBp | 0.36 | 0.99 | 1.39 | 0 | 0 | 0.75 | 0 | | PCCNCz | 0.35 | 1.18 | 0.1 | 0 | 0.32 | 0.39 | 0 | | WRDMEAc | 0.29 | 0.21 | 0.45 | 0 | 0.44 | 0.71 | 0 | | PCREFz | 0.29 | 0.47 | 1.07 | 0 | 0.3 | 0.53 | 0 | | SMCAUSv | 0.25 | 0.39 | 0.77 | 0 | 0.13 | 0.66 | 0 | | SYNNP | 0.22 | 0.03 | 0.64 | 0 | 0.58 | 0.33 | 0 | | PCSYNp | 0.21 | 0.68 | 1.55 | 0 | 0.15 | 0 | 0 | | PCCONNp | 0.17 | 0 | 0.33 | 0 | 0.11 | 0.66 | 0 | | PCCNCp | 0.16 | 0.57 | 0.78 | 0 | 0 | 0.16 | 0 | | PCREFp | 0.15 | 0.06 | 0.74 | 0 | 0 | 0.58 | 0 | | WRDPRP3p | 0.14 | 0.84 | 0 | 0 | 0 | 0.03 | 0 |
Coh-Metrix Model 2b
This model was trained on winter data [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model mars = bagged multivariate adaptive regression splines gbm = stochastic gradient boosted trees svm = support vector machines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------| | -7.2585 | 0.2289 | 0.5300 | 0.1527 | 0.1150 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | mars | gbm | svm | cube | |:----------|:--------|:------|:-----|:-----|:------| | DESWC | 30.39 | 45.46 | 34.5 | 4.37 | 16.04 | | LSAGN | 7.18 | 0 | 9.31 | 2.85 | 17.43 | | DESWLlt | 6.73 | 19 | 2.42 | 1.37 | 9.31 | | LDMTLD | 5.59 | 0 | 8.91 | 2.47 | 5.54 | | SYNLE | 5.43 | 13.58 | 3.65 | 1.79 | 2.18 | | WRDIMGc | 3.6 | 9.36 | 1.84 | 1.22 | 3.37 | | WRDNOUN | 3.19 | 6.64 | 1.38 | 1.77 | 6.53 | | CNCAdd | 1.87 | 5.96 | 0.45 | 1.02 | 1.39 | | WRDVERB | 1.42 | 0 | 2.02 | 1.67 | 1.19 | | SMCAUSwn | 1.25 | 0 | 1.83 | 2.04 | 0 | | DESWLltd | 1.16 | 0 | 1.21 | 1.53 | 2.77 | | CRFCWO1d | 1.09 | 0 | 1.18 | 2.17 | 1.39 | | WRDHYPnv | 1.02 | 0 | 0.98 | 1.36 | 2.77 | | CRFNOa | 0.99 | 0 | 1.37 | 1.9 | 0 | | RDFRE | 0.99 | 0 | 1.54 | 0.25 | 1.39 | | LSAGNd | 0.85 | 0 | 0.53 | 1.76 | 2.77 | | DESWLsy | 0.77 | 0 | 1.11 | 1.28 | 0 | | SYNMEDpos | 0.76 | 0 | 0.29 | 2 | 2.77 | | WRDPRP3s | 0.74 | 0 | 0.91 | 1.53 | 0.4 | | DESPL | 0.73 | 0 | 0.44 | 2.34 | 1.39 | | CNCAll | 0.72 | 0 | 0.44 | 0.96 | 3.17 | | RDL2 | 0.71 | 0 | 0.6 | 1.65 | 1.39 | | PCCNCz | 0.71 | 0 | 0.62 | 1.6 | 1.39 | | PCVERBz | 0.69 | 0 | 0.82 | 1.78 | 0 | | WRDPRO | 0.69 | 0 | 0.69 | 1.18 | 1.39 | | CNCTemp | 0.65 | 0 | 0.71 | 1.87 | 0 | | WRDFRQc | 0.65 | 0 | 0.98 | 0.99 | 0 | | WRDFRQmc | 0.63 | 0 | 0.56 | 1.25 | 1.39 | | DRVP | 0.62 | 0 | 0.93 | 0.92 | 0 | | LSASS1d | 0.61 | 0 | 0.65 | 1.87 | 0 | | SMCAUSlsa | 0.6 | 0 | 0.94 | 0.77 | 0 | | CRFCWOad | 0.6 | 0 | 0.61 | 1.92 | 0 | | WRDMEAc | 0.59 | 0 | 0.53 | 1.4 | 0.99 | | PCTEMPp | 0.56 | 0 | 0.78 | 1.02 | 0 | | SMCAUSv | 0.56 | 0 | 0.85 | 0.83 | 0 | | PCCNCp | 0.56 | 0 | 0.02 | 1.62 | 2.77 | | WRDFRQa | 0.53 | 0 | 0.73 | 1.01 | 0 | | LSASSp | 0.53 | 0 | 0.54 | 1.71 | 0 | | LDTTRc | 0.52 | 0 | 0.64 | 1.3 | 0 | | DRPP | 0.51 | 0 | 0.7 | 1.02 | 0 | | PCREFp | 0.5 | 0 | 0 | 0.7 | 3.56 | | CRFCWO1 | 0.47 | 0 | 0.4 | 1.75 | 0 | | SMCAUSvp | 0.46 | 0 | 0.47 | 1.42 | 0 | | PCNARz | 0.45 | 0 | 0.26 | 1.64 | 0.59 | | SYNNP | 0.45 | 0 | 0.3 | 0.91 | 1.39 | | PCSYNz | 0.45 | 0 | 0.32 | 0.87 | 1.39 | | SMINTEp | 0.45 | 0 | 0.4 | 1.66 | 0 | | LDTTRa | 0.43 | 0 | 0.23 | 1.06 | 1.39 | | DESWLsyd | 0.42 | 0 | 0.35 | 1.62 | 0 | | CRFANPa | 0.41 | 0 | 0.34 | 1.59 | 0 | | SMINTEr | 0.41 | 0 | 0.6 | 0.69 | 0 | | CNCLogic | 0.4 | 0 | 0.51 | 0.89 | 0 | | WRDAOAc | 0.4 | 0 | 0.58 | 0.69 | 0 | | WRDHYPv | 0.4 | 0 | 0.36 | 1.45 | 0 | | CNCNeg | 0.4 | 0 | 0.32 | 1.17 | 0.59 | | CNCCaus | 0.38 | 0 | 0.47 | 0.92 | 0 | | WRDFAMc | 0.37 | 0 | 0.46 | 0.91 | 0 | | SYNSTRUTa | 0.36 | 0 | 0.36 | 1.2 | 0 | | CRFAOa | 0.35 | 0 | 0.19 | 1.68 | 0 | | WRDADV | 0.34 | 0 | 0.32 | 1.19 | 0 | | SMCAUSr | 0.33 | 0 | 0.61 | 0.1 | 0 | | DESSLd | 0.32 | 0 | 0.18 | 1.5 | 0 | | PCCONNp | 0.29 | 0 | 0.46 | 0.37 | 0 | | WRDPOLc | 0.28 | 0 | 0.28 | 0.93 | 0 | | WRDADJ | 0.28 | 0 | 0.4 | 0.51 | 0 | | DRAP | 0.26 | 0 | 0.24 | 0.87 | 0 | | DRNP | 0.26 | 0 | 0.33 | 0.57 | 0 | | WRDHYPn | 0.26 | 0 | 0.27 | 0.84 | 0 | | CNCTempx | 0.25 | 0 | 0.23 | 0.86 | 0 | | DRNEG | 0.23 | 0 | 0.13 | 1.08 | 0 | | PCREFz | 0.23 | 0 | 0.24 | 0.7 | 0 | | PCVERBp | 0.22 | 0 | 0 | 1.48 | 0 | | PCNARp | 0.2 | 0 | 0.01 | 1.35 | 0 | | PCDCp | 0.19 | 0 | 0.11 | 0.9 | 0 | | PCSYNp | 0.09 | 0 | 0.02 | 0.56 | 0 | | WRDPRP3p | 0 | 0 | 0 | 0 | 0 |
Coh-Metrix Model 2c
This model was trained on spring data [@Mercer2019].
Algorithm Weightings in Ensemble
Abbreviations: all = ensemble model pls = partial least squares regression mars = bagged multivariate adaptive regression splines gbm = stochastic gradient boosted trees * svm = support vector machines
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | svm | |:----------|:-------|:-------|:------|:-------| | -10.8192 | 0.0374 | 0.2735 | 0.243 | 0.5377 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
| Metric | overall | pls | mars | gbm | svm | |:----------|:--------|:-----|:------|:------|:-----| | DESWC | 22.58 | 5.76 | 48.45 | 37.37 | 3.91 | | WRDVERB | 7.27 | 1.88 | 23.22 | 2.92 | 1.5 | | DESWLltd | 5.72 | 1.75 | 16.17 | 3.85 | 1.53 | | CRFCWOa | 4.57 | 1.9 | 12.16 | 1.14 | 2.44 | | PCNARp | 2.04 | 2.72 | 0 | 3.23 | 2.49 | | PCDCz | 1.78 | 1.05 | 0 | 3.24 | 2.07 | | CRFANPa | 1.67 | 1.23 | 0 | 2.21 | 2.3 | | LSASS1d | 1.67 | 1.5 | 0 | 1.62 | 2.56 | | WRDHYPn | 1.59 | 2.38 | 0 | 3.29 | 1.58 | | SYNSTRUTa | 1.59 | 1.74 | 0 | 1.28 | 2.54 | | LDMTLD | 1.57 | 1.79 | 0 | 2.47 | 1.95 | | LSAGN | 1.57 | 1.72 | 0 | 1.17 | 2.55 | | SMCAUSvp | 1.53 | 0.97 | 0 | 1.91 | 2.17 | | DESSLd | 1.52 | 0.87 | 0 | 1.28 | 2.45 | | LSAGNd | 1.49 | 2.29 | 0 | 0.11 | 2.81 | | WRDFRQmc | 1.41 | 2.04 | 0 | 1.45 | 2.06 | | DESPL | 1.41 | 3.29 | 0 | 0.85 | 2.25 | | PCVERBz | 1.25 | 1.73 | 0 | 0.62 | 2.13 | | SYNMEDpos | 1.24 | 1.78 | 0 | 0.72 | 2.08 | | LSASSp | 1.22 | 1.97 | 0 | 0.13 | 2.28 | | SMCAUSv | 1.16 | 0.91 | 0 | 1.16 | 1.78 | | CRFCWO1d | 1.15 | 1.54 | 0 | 0.21 | 2.14 | | SMCAUSwn | 1.09 | 2.26 | 0 | 0.94 | 1.63 | | CNCTempx | 1.08 | 0.26 | 0 | 0.57 | 1.92 | | WRDHYPv | 1.06 | 2.37 | 0 | 1.41 | 1.36 | | SMCAUSlsa | 1.03 | 1.59 | 0 | 1.07 | 1.5 | | WRDNOUN | 1 | 2.13 | 0 | 1.68 | 1.11 | | PCDCp | 1 | 1.81 | 0 | 0.23 | 1.8 | | PCVERBp | 0.93 | 1.09 | 0 | 0.05 | 1.79 | | PCTEMPp | 0.92 | 1.68 | 0 | 0.51 | 1.52 | | LDTTRc | 0.9 | 1.26 | 0 | 1.34 | 1.14 | | WRDPRP3s | 0.87 | 1.36 | 0 | 1.67 | 0.92 | | DESWLlt | 0.85 | 2.05 | 0 | 1.12 | 1.07 | | CNCTemp | 0.85 | 0.85 | 0 | 0.9 | 1.27 | | RDL2 | 0.84 | 2.17 | 0 | 0.54 | 1.31 | | DRPP | 0.83 | 1.78 | 0 | 1.37 | 0.94 | | PCCNCz | 0.8 | 1.87 | 0 | 0.31 | 1.35 | | DRNP | 0.8 | 1.62 | 0 | 0.79 | 1.16 | | LDTTRa | 0.79 | 2.31 | 0 | 0.24 | 1.34 | | WRDAOAc | 0.78 | 0.69 | 0 | 0.78 | 1.17 | | RDFKGL | 0.73 | 2.06 | 0 | 0.13 | 1.27 | | SYNNP | 0.68 | 0.65 | 0 | 0.24 | 1.23 | | CNCPos | 0.68 | 0.82 | 0 | 0.67 | 1.03 | | CNCCaus | 0.67 | 0.73 | 0 | 0.63 | 1.02 | | WRDADV | 0.67 | 1.9 | 0 | 0.66 | 0.93 | | PCSYNz | 0.66 | 1.76 | 0 | 0.33 | 1.07 | | WRDPRO | 0.64 | 0.73 | 0 | 0.9 | 0.84 | | CNCLogic | 0.64 | 0.41 | 0 | 0.71 | 0.95 | | PCCNCp | 0.64 | 1.15 | 0 | 0 | 1.22 | | DRVP | 0.63 | 0.79 | 0 | 0.32 | 1.08 | | WRDADJ | 0.62 | 1.12 | 0 | 0.38 | 1 | | SMINTEp | 0.62 | 1.06 | 0 | 0.53 | 0.95 | | DRNEG | 0.6 | 0.73 | 0 | 0.06 | 1.13 | | WRDPOLc | 0.6 | 0.89 | 0 | 0.61 | 0.88 | | WRDHYPnv | 0.58 | 0.02 | 0 | 0.17 | 1.09 | | WRDFRQa | 0.58 | 0.36 | 0 | 0.35 | 0.99 | | SYNLE | 0.57 | 0.09 | 0 | 0.62 | 0.87 | | SMCAUSr | 0.56 | 0.05 | 0 | 0.3 | 1 | | DRAP | 0.55 | 1.23 | 0 | 0.32 | 0.89 | | PCREFz | 0.53 | 1.2 | 0 | 0.46 | 0.78 | | DESWLsy | 0.5 | 0.85 | 0 | 0.43 | 0.76 | | WRDMEAc | 0.49 | 0.68 | 0 | 0.7 | 0.63 | | PCSYNp | 0.48 | 1.45 | 0 | 0.02 | 0.86 | | CNCADC | 0.46 | 1.61 | 0 | 0.41 | 0.63 | | WRDFRQc | 0.42 | 0.3 | 0 | 0.83 | 0.45 | | WRDCNCc | 0.39 | 1.3 | 0 | 0.39 | 0.53 | | DESWLsyd | 0.39 | 0.32 | 0 | 0.33 | 0.63 | | SMINTEr | 0.26 | 0.37 | 0 | 0.11 | 0.44 | | PCCONNz | 0.23 | 0.53 | 0 | 0.23 | 0.32 | | PCREFp | 0.23 | 0.32 | 0 | 0.01 | 0.44 | | WRDFAMc | 0.1 | 0 | 0 | 0.24 | 0.09 | | CRFNO1 | 0.09 | 0.98 | 0 | 0.1 | 0.08 | | PCCONNp | 0.08 | 1.18 | 0 | 0.04 | 0.06 | | WRDPRP3p | 0.02 | 0.44 | 0 | 0.03 | 0 |
Coh-Metrix Model 3 {#cohmetrix-model-3}
General Description
Coh-Metrix Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below.
The models were trained on Coh-Metrix scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons).
Highly correlated CohMetrix metrics (r > |.90|) were excluded during pre-processing (see section on Scoring Model Development for more details).
More details on the sample will be provided once peer review is complete on the main study using this model.
CohMetrix Model 3narr
This model was trained on CohMetrix scores from 15 min narrative writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression mars = bagged multivariate adaptive regression splines enet = elastic net regression * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | 0.0000 | 0.1419 | 0.3143 | 0.0729 | 0.0816 | 0.1792 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |gbm |mars |enet |cube | |---------|-------|----|-----|-----|-----|-----| |DESWC |24.87 |4.56|36.54|23.26|28.45|14.11| |WRDHYPn |8.87 |1.84|2.7 |14.41|7.18 |4.52 | |WRDNOUN |7.12 |2.27|2.89 |10.61|7.1 |4.37 | |DESSL |7 |1.2 |0.27 |14.41|0 |0.77 | |SYNNP |5.71 |1.81|3.49 |7.61 |8.68 |2.38 | |DESWLlt |5.25 |0.86|1.94 |7.61 |5.68 |4.14 | |LDVOCD |4.17 |2.88|3.99 |6.34 |0 |0.69 | |LDTTRa |2.6 |3.58|4.54 |0 |7.74 |3.99 | |SMCAUSwn |2.18 |2 |5.72 |0 |3.96 |2.15 | |SYNLE |2.05 |0.5 |1.59 |3.39 |0 |0.31 | |WRDPRP1s |1.71 |0.5 |0.83 |2.89 |0 |0.84 | |WRDHYPnv |1.55 |0.44|0.28 |2.61 |0 |1.61 | |PCDCp |1.44 |1.37|0.04 |2.61 |0.04 |1 | |PCREFp |1.41 |0.55|0 |2.65 |0 |1 | |PCNARz |1.32 |2.01|3.19 |0 |0 |3.14 | |CRFANPa |1.3 |1.59|3.69 |0 |0 |2.3 | |LSAGN |1.29 |2.26|2.08 |0 |2.54 |2.99 | |WRDFRQmc |0.94 |1.95|0.78 |0 |2.77 |2.68 | |CNCLogic |0.91 |0.73|0.37 |1.61 |0 |0.23 | |PCDCz |0.91 |1.12|3.1 |0 |0 |0.77 | |SYNMEDpos|0.9 |1.78|0.13 |0 |4.2 |2.61 | |SMCAUSlsa|0.84 |0.78|0.76 |0 |1.34 |3.37 | |PCCONNz |0.75 |1.46|1.11 |0 |2.09 |1.46 | |DRPP |0.65 |1.2 |0.36 |0 |2.51 |1.76 | |WRDAOAc |0.65 |1.77|1.13 |0 |1.24 |1.23 | |WRDPRO |0.63 |1.35|1.42 |0 |0 |1.53 | |DESPL |0.61 |2.71|0.51 |0 |1.6 |1.46 | |PCREFz |0.59 |1.06|0.56 |0 |2.75 |0.92 | |WRDMEAc |0.53 |0.99|1.17 |0 |0.82 |0.84 | |LDTTRc |0.52 |2.28|0.15 |0 |0 |2.68 | |DRNP |0.49 |0.62|0.24 |0 |1.25 |1.92 | |DESWLsy |0.49 |1.25|0.33 |0 |0.72 |1.99 | |WRDPOLc |0.44 |1.45|0.91 |0 |0 |1.07 | |SMINTEr |0.4 |0.35|0.26 |0 |2.03 |0.84 | |PCSYNz |0.39 |0.91|0.59 |0 |0 |1.46 | |PCCONNp |0.37 |2.07|0.2 |0 |1.9 |0.31 | |SMINTEp |0.35 |0.56|1.24 |0 |0 |0.15 | |DRPVAL |0.34 |1.5 |0.33 |0 |1.65 |0.23 | |LSASS1d |0.33 |1.49|0.06 |0 |0 |1.76 | |PCCNCz |0.32 |1.44|0.1 |0 |0 |1.61 | |CRFCWOa |0.3 |1.73|0.04 |0 |0 |1.46 | |RDL2 |0.29 |1.91|0.31 |0 |0 |0.92 | |CNCPos |0.28 |0.22|0.87 |0 |0 |0.38 | |PCVERBz |0.28 |1.5 |0.14 |0 |0 |1.23 | |LSAGNd |0.28 |1.86|0.19 |0 |0 |1.07 | |CRFCWO1d |0.25 |1.57|0.74 |0 |0 |0 | |WRDFRQa |0.25 |0.44|0.67 |0 |0 |0.46 | |CNCCaus |0.25 |0.36|0.29 |0 |0 |1.15 | |CRFAOa |0.24 |1.74|0.04 |0 |0 |1.07 | |DESWLltd |0.23 |0.2 |0.49 |0 |0 |0.69 | |WRDADJ |0.22 |0.25|0.43 |0 |0 |0.69 | |WRDPRP3p |0.21 |1.25|0.51 |0 |0 |0.23 | |LDMTLD |0.21 |0.26|0.42 |0 |0 |0.69 | |CRFCWOad |0.21 |1.68|0.35 |0 |0 |0.38 | |CNCTemp |0.2 |0.55|0.06 |0 |1.45 |0.15 | |WRDIMGc |0.19 |0.4 |0.22 |0 |0 |0.84 | |DESWLsyd |0.19 |0.69|0.26 |0 |0 |0.69 | |WRDVERB |0.18 |0.55|0.44 |0 |0 |0.31 | |CRFNOa |0.15 |1.06|0.07 |0 |0 |0.61 | |CNCADC |0.14 |1.46|0.12 |0 |0 |0.31 | |WRDHYPv |0.14 |0.35|0.32 |0 |0 |0.31 | |WRDFRQc |0.14 |0.58|0.38 |0 |0 |0.15 | |WRDCNCc |0.14 |0.37|0.51 |0 |0 |0 | |LSASSp |0.14 |1.54|0.08 |0 |0 |0.38 | |LSASSpd |0.14 |1.55|0.05 |0 |0 |0.46 | |PCTEMPp |0.13 |1.36|0.06 |0 |0 |0.38 | |DRVP |0.12 |0.92|0.15 |0 |0 |0.31 | |WRDPRP2 |0.12 |1.54|0.13 |0 |0.23 |0 | |DRGERUND |0.12 |0.49|0.15 |0 |0 |0.46 | |SMCAUSvp |0.11 |0.54|0.28 |0 |0 |0.15 | |PCSYNp |0.1 |0.68|0.15 |0 |0 |0.23 | |DRINF |0.1 |0.68|0.11 |0 |0 |0.31 | |DRAP |0.09 |0.43|0.29 |0 |0 |0 | |WRDADV |0.09 |0.6 |0.28 |0 |0 |0 | |DESSLd |0.09 |1 |0.12 |0 |0.1 |0.08 | |SYNSTRUTa|0.08 |1.31|0.09 |0 |0 |0 | |PCCNCp |0.07 |1.21|0 |0 |0 |0.15 | |WRDFAMc |0.07 |0.63|0.2 |0 |0 |0 | |SMCAUSv |0.06 |0.53|0.14 |0 |0 |0 | |CNCTempx |0.05 |0 |0.16 |0 |0 |0.08 | |SMCAUSr |0.04 |0.77|0.02 |0 |0 |0 | |PCVERBp |0.04 |0.9 |0 |0 |0 |0 | |WRDPRP1p |0.03 |0.68|0.02 |0 |0 |0 | |WRDPRP3s |0.02 |0.4 |0 |0 |0 |0 | |DRNEG |0.01 |0.24|0.02 |0 |0 |0 |
Coh-Metrix Model 3exp
This model was trained on Coh-Metrix scores from 15 min expository writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression gbm = stochastic gradient boosted trees mars = bagged multivariate adaptive regression splines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | cube | |:----------|:-------|:-------|:-------|:-------| | -0.0577 | 0.1306 | 0.3136 | 0.3991 | 0.1752 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|mars |pls |gbm |cube | |---------|-------|-----|----|-----|-----| |DESWC |26.13 |25.33|5.56|47.08|15.82| |LSAGN |3.77 |0 |2.75|5.56 |4.35 | |LDTTRa |3.68 |0 |4.12|4.55 |3.68 | |DESSLd |3.13 |9.32 |1.26|2.42 |3.51 | |DESWLlt |2.98 |10.88|0.99|1.35 |4.35 | |WRDPRP2 |2.25 |0 |2.09|2.72 |3.18 | |LDVOCD |2.16 |2.46 |3.82|0.71 |2.26 | |DRPP |2.11 |0 |2.07|3.28 |1.09 | |WRDPOLc |2.1 |12.49|0.93|0.31 |0.5 | |LDTTRc |1.96 |0 |3.31|1.89 |1.17 | |SMCAUSwn |1.62 |0 |0.98|2.12 |2.85 | |WRDNOUN |1.61 |3.23 |1.38|0.53 |3.26 | |WRDPRP1s |1.55 |5.37 |1.05|0.83 |1.26 | |WRDPRP1p |1.5 |5.2 |0.08|0.9 |2.68 | |CNCTemp |1.39 |7.18 |1.01|0.27 |0.33 | |PCNARz |1.37 |0 |2.21|0.63 |2.59 | |LSASS1d |1.29 |4.14 |1.67|0.51 |0.25 | |PCREFz |1.27 |5.37 |1.14|0.18 |0.92 | |PCCONNz |1.25 |0 |1.16|2.11 |0.42 | |DRNP |1.22 |3.67 |0.17|1.18 |1.34 | |WRDMEAc |1.2 |5.37 |0.64|0.47 |0.75 | |WRDFRQa |1.19 |0 |1.04|1.52 |1.59 | |PCCONNp |1.19 |0 |2.2 |0.62 |1.59 | |SYNMEDpos|1.19 |0 |1.84|0.24 |3.1 | |WRDHYPn |1.18 |0 |0.89|1.18 |2.59 | |DESPL |1.03 |0 |2.71|0.38 |0.25 | |WRDHYPnv |1.02 |0 |0.83|0.73 |2.76 | |PCCNCz |0.94 |0 |1.35|0.26 |2.43 | |RDL2 |0.93 |0 |1.89|0.83 |0.17 | |LSASSp |0.9 |0 |1.68|0.69 |0.67 | |PCVERBz |0.9 |0 |1.38|0.38 |1.92 | |WRDHYPv |0.87 |0 |0.92|0.94 |1.26 | |LSASSpd |0.84 |0 |1.66|0.21 |1.42 | |WRDADJ |0.82 |0 |1.39|0.89 |0.25 | |CRFCWOa |0.81 |0 |1.79|0.37 |0.67 | |CRFANPa |0.78 |0 |1.28|0.52 |1.09 | |LSAGNd |0.77 |0 |2.06|0.08 |0.59 | |PCREFp |0.77 |0 |0.97|0 |2.76 | |CRFAOa |0.74 |0 |1.87|0.02 |0.92 | |DESSL |0.74 |0 |0.94|0.46 |1.59 | |CRFCWO1d |0.69 |0 |1.77|0.29 |0.17 | |SYNNP |0.69 |0 |1.25|0.31 |1.09 | |CRFNOa |0.64 |0 |1.2 |0.47 |0.5 | |PCTEMPp |0.62 |0 |1.26|0.49 |0.25 | |WRDAOAc |0.61 |0 |1.21|0.47 |0.33 | |CNCNeg |0.6 |0 |1.7 |0.2 |0 | |DRAP |0.58 |0 |1.08|0.21 |0.92 | |DRGERUND |0.57 |0 |0.97|0.69 |0 | |PCDCz |0.55 |0 |1.33|0.16 |0.42 | |PCDCp |0.53 |0 |1.35|0.09 |0.42 | |DESWLsy |0.53 |0 |0.58|0.34 |1.26 | |PCSYNz |0.51 |0 |0.75|0.12 |1.34 | |DESWLltd |0.5 |0 |0.25|0.53 |1.26 | |SMCAUSr |0.47 |0 |1.38|0.11 |0 | |WRDFRQmc |0.47 |0 |1.21|0.11 |0.33 | |WRDIMGc |0.45 |0 |0.32|0.27 |1.42 | |LDMTLD |0.45 |0 |0.51|0.64 |0.25 | |SMCAUSlsa|0.44 |0 |0.33|0.24 |1.42 | |SYNSTRUTa|0.43 |0 |1.13|0.22 |0 | |WRDADV |0.42 |0 |0.56|0.12 |1.17 | |CNCTempx |0.37 |0 |1.07|0.09 |0 | |WRDFRQc |0.37 |0 |0.68|0.16 |0.59 | |CNCLogic |0.36 |0 |0.95|0.17 |0 | |SMINTEr |0.36 |0 |1.12|0.03 |0 | |DRINF |0.36 |0 |0.89|0.21 |0 | |DESWLsyd |0.36 |0 |0.28|0.55 |0.33 | |SMINTEp |0.35 |0 |0.99|0.07 |0.08 | |SMCAUSvp |0.33 |0 |0.99|0.06 |0 | |CNCPos |0.33 |0 |0.17|0.61 |0.25 | |PCVERBp |0.31 |0 |0.49|0.01 |0.92 | |WRDPRP3s |0.29 |0 |0.42|0.26 |0.33 | |PCCNCp |0.29 |0 |0.91|0.03 |0 | |WRDPRO |0.29 |0 |0.64|0.14 |0.25 | |WRDFAMc |0.26 |0 |0.55|0.24 |0 | |DRVP |0.25 |0 |0.57|0.18 |0 | |SMCAUSv |0.23 |0 |0.69|0.04 |0 | |SYNLE |0.2 |0 |0.03|0.33 |0.33 | |PCSYNp |0.2 |0 |0.53|0.02 |0.17 | |WRDPRP3p |0.19 |0 |0.25|0.29 |0 | |CNCCaus |0.18 |0 |0.47|0.09 |0 | |WRDVERB |0.17 |0 |0.1 |0.34 |0 | |DRNEG |0.03 |0 |0 |0.08 |0 |
Coh-Metrix Model 3per
This model was trained on Coh-Metrix scores from 15 min persuasive writing samples.
Algorithm Weightings in Ensemble
Abbreviations: overall = ensemble model pls = partial least squares regression gbm = stochastic gradient boosted trees mars = bagged multivariate adaptive regression splines * cube = cubist regression
The table below presents the linear weightings of each algorithm for the ensemble model.
| Intercept | pls | mars | gbm | cube | |:----------|:-------|:-------|:-------|:-------| | -0.0381 | 0.0558 | 0.4924 | 0.4425 | 0.0259 |
Metric Importance in Each Algorithm and Ensemble
Each column sums to approximately 100, allowing for rounding (values describe relative contribution to the model).
|Metric |overall|pls |mars |gbm |cube | |---------|-------|----|-----|-----|-----| |DESWC |32.09 |4.68|34.34|33.8 |19.05| |WRDHYPn |10.41 |2.03|17.13|4.3 |5.17 | |LDVOCD |9.59 |3.43|0 |21.44|2.53 | |DESWLlt |8.27 |1.29|13.16|3.77 |7.4 | |LSAGN |6.13 |2.92|8.63 |3.88 |4.05 | |WRDNOUN |4.46 |1.39|7.36 |1.66 |3.95 | |WRDADV |3.05 |1.18|5.53 |0.52 |3.24 | |WRDFRQa |2.13 |0.25|3.86 |0.55 |0.2 | |SMCAUSwn |2.11 |1.08|3.7 |0.54 |1.01 | |CNCAdd |1.76 |0.66|3.37 |0.19 |0.2 | |WRDADJ |1.67 |0.59|2.93 |0.26 |4.15 | |LDTTRa |1.47 |4.05|0 |2.58 |4.76 | |DESSC |1.38 |3.27|0 |2.6 |2.74 | |LDTTRc |1.17 |3.17|0 |2.21 |1.32 | |DESWLltd |0.63 |1.31|0 |1.2 |1.42 | |WRDPRO |0.57 |1.57|0 |1.1 |0.3 | |PCDCz |0.5 |1.55|0 |0.94 |0.3 | |DESSLd |0.47 |1.61|0 |0.84 |0.61 | |CRFCWO1d |0.47 |2.36|0 |0.74 |0.81 | |DRNEG |0.46 |1.75|0 |0.72 |1.82 | |WRDPOLc |0.46 |0.78|0 |0.89 |1.22 | |SYNNP |0.45 |0.82|0 |0.92 |0 | |CNCCaus |0.44 |0.24|0 |0.98 |0 | |WRDPRP3p |0.4 |0.59|0 |0.82 |0.3 | |WRDHYPv |0.39 |0.94|0 |0.76 |0.2 | |LDMTLD |0.35 |0.38|0 |0.67 |1.72 | |WRDAOAc |0.34 |1.9 |0 |0.52 |0.3 | |CNCPos |0.32 |0.48|0 |0.66 |0 | |WRDMEAc |0.32 |1.11|0 |0.56 |0.61 | |CRFANPa |0.31 |1.37|0 |0.52 |0.2 | |DRVP |0.31 |0.47|0 |0.58 |1.22 | |WRDFRQc |0.31 |0.22|0 |0.65 |0.71 | |SMCAUSlsa|0.26 |0.65|0 |0.5 |0 | |LSASS1d |0.26 |2.04|0 |0.34 |0 | |CRFAO1 |0.25 |2.03|0 |0.3 |0.2 | |WRDHYPnv |0.25 |0.33|0 |0.48 |0.71 | |CNCTempx |0.23 |0.5 |0 |0.32 |2.23 | |SYNSTRUTa|0.22 |0.75|0 |0.39 |0.41 | |SYNMEDpos|0.21 |2.04|0 |0.14 |1.42 | |WRDPRP1s |0.2 |0.81|0 |0.26 |1.62 | |PCVERBz |0.2 |1.75|0 |0.15 |1.62 | |CRFCWO1 |0.19 |1.81|0 |0.18 |0.41 | |RDL2 |0.19 |1.54|0 |0.22 |0.51 | |PCNARz |0.18 |2.01|0 |0.05 |1.72 | |LSAGNd |0.18 |2.08|0 |0.1 |0.91 | |RDFKGL |0.17 |0.52|0 |0.19 |2.13 | |PCSYNz |0.17 |0.67|0 |0.19 |2.13 | |LSASSpd |0.15 |1.95|0 |0.08 |0.2 | |SMCAUSv |0.15 |0.37|0 |0.26 |0.61 | |SYNLE |0.14 |0.44|0 |0.26 |0 | |RDFRE |0.14 |0.45|0 |0.14 |2.13 | |DRNP |0.14 |0.41|0 |0.28 |0 | |PCDCp |0.14 |1.89|0 |0 |1.62 | |SMCAUSr |0.13 |1.29|0 |0.13 |0 | |LSASS1 |0.13 |1.84|0 |0.04 |0.41 | |CRFCWOad |0.13 |1.93|0 |0.04 |0.3 | |WRDPRP2 |0.12 |1.32|0 |0.1 |0 | |WRDFAMc |0.12 |0.32|0 |0.23 |0.1 | |PCREFz |0.12 |1.13|0 |0.06 |1.42 | |DESWLsy |0.11 |0.61|0 |0.13 |0.51 | |CNCLogic |0.11 |0.55|0 |0.17 |0.1 | |CRFNO1 |0.11 |1.61|0 |0.04 |0.2 | |DRGERUND |0.1 |0.38|0 |0.16 |0.41 | |DRPP |0.1 |0.34|0 |0.19 |0 | |PCTEMPp |0.1 |1.16|0 |0.07 |0.2 | |SMINTEr |0.1 |1.5 |0 |0.03 |0.2 | |PCCNCz |0.1 |1.31|0 |0.05 |0.41 | |DESWLsyd |0.1 |0.76|0 |0.13 |0.3 | |CNCNeg |0.09 |0.55|0 |0.13 |0.3 | |WRDVERB |0.08 |0.23|0 |0.15 |0 | |SMCAUSvp |0.08 |0.24|0 |0.14 |0.2 | |PCCONNz |0.08 |0.49|0 |0.11 |0.3 | |PCCONNp |0.07 |1.11|0 |0.01 |0 | |DRAP |0.07 |0.12|0 |0.14 |0 | |WRDCNCc |0.07 |0.03|0 |0.14 |0.2 | |WRDFRQmc |0.07 |1.06|0 |0.03 |0 | |PCVERBp |0.07 |1.19|0 |0 |0.3 | |PCREFp |0.06 |0.92|0 |0 |0.3 | |PCCNCp |0.05 |0.83|0 |0 |0 | |WRDPRP1p |0.05 |0.16|0 |0.09 |0 | |WRDIMGc |0.05 |0 |0 |0.09 |0.51 | |SMINTEp |0.05 |0.71|0 |0.03 |0 | |DRINF |0.05 |0.3 |0 |0.06 |0.3 | |CNCADC |0.05 |0.44|0 |0.05 |0.2 | |CNCTemp |0.04 |0.58|0 |0.02 |0 | |PCSYNp |0.04 |0.42|0 |0.01 |0.71 | |DRPVAL |0.01 |0.07|0 |0.01 |0 |
Automated Written Expression CBM (aWE-CBM) Model 1 {#awecbm-model-1}
General Description
Total Words Written (TWW) scores are generated directly from the GAMET word count score. Words Spelled Correctly (WSC) scores are generated by subtracting the GAMET misspelling score from the GAMET word count score.
Correct Word Sequences (CWS) and Correct Minus Incorrect Word Sequences (CIWS) scores are based on ensemble models originally trained to predict CBM scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019]. More details on the sample are available in [@Mercer2019].
The CWS and CIWS models are detailed below (from Mercer et al., 2021).
Correct Word Sequences Model
| Metric | Overall | GBM | SVM | ENET | MARS | |:------------|:--------|:------|:------|:------|:------| | Word Count | 75.48 | 86.79 | 67.10 | 77.17 | 77.84 | | Spelling | 14.26 | 0.62 | 0.00 | 21.41 | 22.05 | | %Spelling | 8.78 | 12.28 | 27.95 | 0.40 | 0.11 | | Grammar | 0.85 | 0.05 | 2.77 | 0.11 | 0.00 | | %Grammar | 0.01 | 0.06 | 0.01 | 0.00 | 0.00 | | Duplication | 0.04 | 0.12 | 0.12 | 0.00 | 0.00 | | Typography | 0.38 | 0.08 | 1.33 | 0.00 | 0.00 | | White Space | 0.20 | 0.00 | 0.71 | 0.92 | 0.00 |
Note. The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following regression equation was used to weight the algorithms in the CWS ensemble model: .162 + .074 * GBM + .281 * SVM + .001 * ENET + .642 * MARS.
Correct Minus Incorrect Word Sequences Model
| Metric | Overall | GBM | SVM | ENET | MARS | |:------------|:--------|:------|:------|:------|:------| | Word Count | 55.60 | 55.76 | 47.57 | 61.43 | 61.35 | | Spelling | 19.25 | 1.48 | 6.57 | 35.80 | 35.04 | | %Spelling | 22.31 | 41.99 | 42.74 | 0.00 | 0.00 | | Grammar | 0.82 | 0.00 | 1.69 | 0.00 | 0.62 | | %Grammar | 0.04 | 0.23 | 0.00 | 0.00 | 0.00 | | Duplication | 0.28 | 0.10 | 0.76 | 0.00 | 0.00 | | Typography | 1.37 | 0.41 | 0.07 | 1.55 | 2.97 | | White Space | 0.34 | 0.04 | 0.60 | 1.22 | 0.00 |
Note. The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following equation was used for the CIWS model: -.170 + .180 * GBM + .346 * SVM + .100 * ENET + .375 * MARS.
References
Try the writeAlizer package in your browser
Any scripts or data that you put into this service are public.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.