In garyfeng/pdata: Process Data Analysis and Reporting for Large Scale Educatoinal Assessment

Conducting statistical significant tests

Adjusting the p-value for multiple comparisons

In the following example, we have a data frame dfSceneDur with three variables,

• sceneId indicates the scenes
• blockId marks the two conditions in this study
• dur is the duration of each scene

We wish to test if dur differs by condition in each scene. Given the number of multiple comparisons, we need to adjust the family significance level, e.g., using the Bonferroni method.

``` {r eval=FALSE}

do Wilcoxon for each scene

wilcoxRes <- dfSceneDur %>% group_by(sceneId) %>% do(w = wilcox.test(as.numeric(.$dur) ~ .$blockId, data=.))

find out which scene is significant, with p.adjust()

wilcoxRes.sig <- NULL for (i in 1:length(wilcoxRes$sceneId)) { # loop through all scenes scene<- wilcoxRes$sceneId[i] p<- wilcoxRes$w[i][[1]]$p.value if (p.adjust(p, n=length(wilcoxRes$sceneId), method="bonferroni")<0.05) { append(wilcoxRes.sig, paste("Test is significant (with Bonferroni adjustment) for Scence='", scene, "', with p=", p)) } }

In generating the report, we can do something like 

There are r '\x60r length(wilcoxRes.sig) \x60' scenes showing significant difference after the Bonferroni adjustment: r '\x60r print (wilcoxRes.sig) \x60' ```

garyfeng/pdata documentation built on May 16, 2019, 5:42 p.m.