In rENA: Epistemic Network Analysis

r ub = enaset[["_function.params"]]$units.by r meta = enaset$meta.data[, ub, with = F] r u = enaset$enadata$units r cvb = enaset[["_function.params"]]$conversations.by r cv = enaset[["_function.params"]]$conversation r cvEx = as.vector(as.matrix((enaset$model$raw.input[1,cvb,with=F]))[1,]) r codes = enaset$rotation$codes r wb = enaset[["_function.params"]]$window.size.back r wgt = enaset[["_function.params"]]$weight.by r normed = identical(enaset[["_function.params"]]$norm.by, rENA:::fun_sphere_norm) r corr = ena.correlations(enaset) r corrRnd = apply(apply(matrix(corr), 1, unlist), c(1,2), round, sig.dig) r if(theory == T) paste0("## ENA Theory")

r if(theory == T) paste0("Epistemic Network Analysis (ENA) is a technique for modeling the structure of connections in data. ENA assumes: (1) that it is possible to systematically identify a set of meaningful features in the data (*Codes*); (2) that the data has local structure (*conversations*); and (3) that an important feature of the data is the way that Codes are connected to one another within conversations (Bowman et al., 2021; Shaffer, 2017; Shaffer, Collier, & Ruis, 2016; Shaffer & Ruis, 2017). For example, if a team is working on a design project, they talk about important Codes such as production processes, design specifications, budget, and so on. They have a series of conversations at design meetings, and a key part of understanding their design process is modeling how they think about the relationships between production processes, specifications, budget, and other key parts of their design work (Arastoopour et al., 2016). In an interview transcript, each answer to a question might be a unique conversation (Eagan & Hamilton, 2018; Zörgő & Peters, 2019), or in a collection of documents, each document or section of a document (Ruis, 2018). ENA models the connections among Codes by quantifying the co-occurrence of Codes within conversations, producing a weighted network of co-occurrences, along with associated visualizations for each unit of analysis in the data. Critically, ENA analyzes all of the networks simultaneously, resulting in a set of networks that can be compared both visually and statistically.")

r if(theory == T) paste0("While ENA was originally designed to address challenges in learning analytics (Shaffer et al., 2009), the method is not limited to analyses of learning data. For example, ENA has been used to analyze (a) task performance (D’Angelo et al., 2020; Ruis et al., 2018); (b) gaze patterns (Andrist et al., 2015; Brückner et al., 2020); (c) team communication (Sullivan et al., 2018; Wooldridge et al., 2018); (d) governmental communication and policy (Schnaider et al., 2021); and social media (Dubovi & Tabak, 2021; Misiejuk et al., 2021). The key assumption of the method is that the structure of connections in the data is meaningful. ENA is thus a useful technique for modeling [your topic] because it can model the relationships among [features of interest in your data] as they occur within [your conversations].")

r if(methods == T) paste0("## ENA Methods")

r if(methods == T) paste0("In this study, we applied epistemic network analysis (Bowman et al., 2021; Shaffer, 2017; Shaffer, Collier, & Ruis, 2016; Shaffer & Ruis, 2017) to our data using the ENA", ifelse(tool=="rENA", paste0(" package for the statistical programming language R (version ", tool.version, ") (Marquart, Swiecki, et al., 2018)"), paste0(" Web Tool (version ", tool.version, ") (Marquart, Hinojosa, Swiecki, Eagan, & Shaffer, 2018)")))

r if(methods == T) paste0("We defined the units of analysis as all lines of data associated with a single value of ", ub[1], if(length(ub) > 1)" subsetted by ", concatenate::cc_and(ub[2:length(ub)], oxford=T), ". For example, one unit consisted of all the lines associated with ", concatenate::cc_and(apply(matrix(c(colnames(meta)[-1], t(meta[1,-c(1)])), ncol = 2, nrow = ncol(meta)-1), 1, paste, collapse = " "), oxford = T),".")

r if(methods == T) paste0(if(identical(wb, "Conversation")) "The ENA algorithm constructs a network model for each unit of analysis, showing how the codes in all lines associated with that unit—within a given set of lines called a conversation—are connected to one another. The resulting networks are aggregated for all conversations for each unit of analysis in the model." else if(is.numeric(wb)) paste0("The ENA algorithm uses a moving window to construct a network model for each line in the data, showing how codes in the current line are connected to codes that occur previously (Ruis et al., 2019; Siebert-Evenstone et al., 2017), defined as ", (if(!is.infinite(wb)){ paste0(wb," lines (each line plus the ",(wb-1)," previous lines)")} else {("all lines preceding the current line")}))," within a given conversation. [Provide justification for the window length.] The resulting networks are aggregated for all lines for each unit of analysis in the model. In this model, we aggregated networks using a ",if(wgt %in% c('binary',"'binary'")){("binary")}else{("weighted")},(" summation in which the networks for a given"),(if(identical(wb, "Conversation")){(" conversation")}else{(" line")}),(" reflect"),(if(wgt %in% c('binary',"'binary'")){(" the presence or absence of the co-occurrence")} else if(identical(wgt, 'product')){(" the product of the value")} else if(identical(wgt,'sqrt')){(" square root of the product")} else{ (" log of the product") }),(" of each pair of codes."))

r if(methods == T) paste0("Our ENA model included the following codes: ", concatenate::cc_and(codes), ". We defined conversations as all lines of data associated with a single value of ", cvb[1], if(length(cvb) > 1) paste0(" subsetted by ", concatenate::cc_and(cvb[2:length(cvb)], oxford=T)), ". For example, one conversation consisted of all the lines associated with ", concatenate::cc_and(apply(matrix(c(cvb, cvEx), nrow=2), 1, paste, collapse=" "), oxford=T),".")

r if(methods ==T) paste0(if(normed) paste0("The ENA model normalized the networks for all units of analysis before they were subjected to a dimensional reduction, which accounts for the fact that different units of analysis may have different numbers of coded lines in the data. "),"For the dimensional reduction, we used a ", if(identical(enaset$function.params$rotation.by, rENA::ena.rotate.by.mean)) paste0("means rotation based on two groups in our data: ", concatenate::cc_and(names(enaset$function.params$rotation.params)),". This projection highlights the differences between these groups (if any) by constructing a dimensional reduction that places the means of the groups as close as possible to the x-axis of the projected space. Subsequent dimensions were projected using")," singular value decomposition, which produces orthogonal dimensions that maximize the variance explained by each dimension. (See Bowman et al., 2021, and Shaffer et al., 2016 for a more detailed explanation of the mathematics.)")

r if(methods ==T) paste0("Networks were visualized using network graphs where nodes correspond to the codes, and edges reflect the", if(normed) " relative"," frequency of co-occurrence, or connection, between two codes. The result is two coordinated representations for each unit of analysis: (1) a plotted point, which represents the location of that unit’s network in the low-dimensional projected space, and (2) a weighted network graph. The positions of the network graph nodes are fixed, and those positions are determined by an optimization routine that minimizes the difference between the plotted points and their corresponding network centroids. Because of this co-registration of network graphs and projected space, the positions of the network graph nodes—and the connections they define—can be used to interpret the dimensions of the projected space and explain the positions of plotted points in the space. Our model had co-registration correlations of ", corrRnd[1,1]," (Pearson) and ",corrRnd[1,2]," (Spearman) for the first dimension and co-registration correlations of ", corrRnd[2,1]," (Pearson) and ",corrRnd[2,2]," (Spearman) for the second.")``r if(methods == T && all(corrRnd > 0.9)) paste0(" These measures indicate that there is a strong goodness of fit between the visualization and the original model.")

r if(methods == T && !is.null(comparison)) paste0("ENA can be used to compare units of analysis in terms of their plotted point positions, individual networks, mean plotted point positions, and mean networks, which average the connection weights across individual networks. Networks may also be compared using network difference graphs. These graphs are calculated by subtracting the weight of each connection in one network from the corresponding connections in another.")

r if(methods == T && !is.null(comparison) && !is.null(comparison.groups)) paste0("To test for differences we applied a ", if(comparison == "parametric") "two-sample t-test assuming unequal variance" else "Mann-Whitney test", " to the location of points in the projected ENA space for units in ", comparison.groups[1], " and ", comparison.groups[2], ".")

References

r if(theory == T) paste0("Andrist, S., Collier, W., Gleicher, M., Mutlu, B., & Shaffer, D. (2015). Look together: Analyzing gaze coordination with epistemic network analysis. Frontiers in Psychology, 6(1016).")

r if(theory == T) paste0("Arastoopour, G., Shaffer, D. W., Swiecki, Z., Ruis, A. R., & Chesler, N. C. (2016). Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. International Journal of Engineering Education, 32(3B), 1492–1501.")

Bowman, D., Swiecki, Z., Cai, Z., Wang, Y., Eagan, B., Linderoth, J., & Williamson Shaffer, D. (2021). The mathematical foundations of epistemic network analysis. In A. Ruis & S. B. Lee (Eds.), Advances in Quantitative Ethnography: Second International Conference, ICQE 2020, Malibu, CA, USA, February 1–3, 2021, Proceedings (pp. 91–105). Springer.

r if(theory == T) paste0("Brückner, S., Schneider, J., Zlatkin-Troitschanskaia, O., & Drachsler, H. (2020). Epistemic network analyses of economics students’ graph understanding: An eye-tracking study. Sensors, 20, 6908.")

r if(theory == T) paste0("D’Angelo, A.-L., Ruis, A., Pugh, C., Collier, W., & Williamson Shaffer, D. (2020). Evaluating how residents talk and what it means for surgical performance in the simulation lab. The American Journal of Surgery, 220(1), 37–43.")

r if(theory == T) paste0("Dubovi, I., & Tabak, I. (2021). Interactions between emotional and cognitive engagement with science on YouTube. Public Understanding of Science. In press.")

r if(theory == T) paste0("Eagan, B., & Hamilton, E. (2018). Epistemic network analysis of an international digital makerspace in Africa, Europe, and the US. Presented at the Annual Meeting of the American Education Research Association, New York, NY.")

r if(methods == T && tool=="webENA") paste0("Marquart, C. L., Hinojosa, C., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2021). Epistemic network analysis (Version ", tool.version, ") [Software]. Available from http://app.epistemicnetwork.org")``r if(methods == T && tool=="rENA") paste0("Marquart, C. L., Swiecki, Z., Collier, W., Eagan, B., Woodward, R., & Shaffer, D. W. (2021). rENA: epistemic network analysis (Version ", tool.version, "). Retrieved from https://cran.r-project.org/web/packages/rENA/index.html")

r if(theory == T) paste0("Ruis, A. R. (2018). \"Trois empreintes d’un même Cachet\": Toward a historical definition of nutrition. In E. T. Ewing & K. Randall (eds.), Viral networks: Connecting digital humanities and medical history (pp. 179–212). Blacksburg: VT Publishing.")

r if(theory == T) paste0("Ruis, A. R., Rosser, A. A., Quandt-Walle, C., Nathwani, J. N., Shaffer, D. W., & Pugh, C. M. (2018). The hands and head of a surgeon: Modeling operative competency with multimodal epistemic network analysis. American Journal of Surgery, 216(5), 835-840.")

r if(methods == T) paste("Ruis, A. R., Siebert-Evenstone, A. L., Pozen, R., Eagan, B. R., & Shaffer, D. W. (2019). Finding common ground: A method for measuring recent temporal context in analyses of complex, collaborative thinking. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A wide lens: Combining embodied, enactive, extended, and embedded learning in collaborative settings: 13^th^ International Conference on Computer-Supported Collaborative Learning (CSCL) 2019: Vol. I (pp. 136–143).")

r if(theory == T) paste0("Schnaider, K., Schiavetto, S., Meier, F., Wasson, B., Allsopp, B., & Spikol, D. (2021). Governmental response to the COVID-19 pandemic—A quantitative ethnographic comparison of public health authorities’ communication in Denmark, Norway, and Sweden. In A. Ruis & S. Lee (Eds.), Advances in quantitative ethnography: Second International Conference, ICQE 2020, Malibu, CA, USA, February 1–3, 2021, Proceedings (pp. 406–421). Springer.")

Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI: Cathcart Press.

Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.

r if(theory == T) paste0("Shaffer, D. W., Hatfield, D. L., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E. A., … Frank, K. (2009). Epistemic network analysis: A prototype for 21^st^ century assessment of learning. International Journal of Learning and Media, 1(1), 1–21.")

Shaffer, D. W., & Ruis, A. R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. F. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (pp. 175–187). Society for Learning Analytics Research.

r if(methods == T) paste0("Siebert-Evenstone, A., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A. R., & Williamson Shaffer, D. (2017). In search of conversational grain size: Modelling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139. https://doi.org/10.18608/jla.2017.43.7")

Sullivan, S. A., Warner-Hillard, C., Eagan, B. R., Thompson, R., Ruis, A. R., Haines, K., … Jung, H. S. (2017). Using epistemic network analysis to identify targets for educational interventions in trauma team communication. Surgery, 163, 938–43.

r if(theory == T) paste0("Wooldridge, A. R., Carayon, P., Eagan, B. R., & Shaffer, D. W. (2018). Quantifying the qualitative with epistemic network analysis: A human factors case study of task-allocation communication in a primary care team. IIE Transactions on Healthcare Systems Engineering, 8, 72–82.")

r if(theory == T) paste0("Zörgő, S., & Peters, G.-J. (2019). Epistemic network analysis for semi-structured interviews and other continuous narratives: Challenges and insights. In B. Eagan, M. Misfeldt, & A. Siebert-Evenstone (Eds.), Advances in quantitative ethnography: First International Conference, ICQE 2019, Madison, WI, USA, October 20–22, 2019, Proceedings (pp. 267–277). Springer.")