Rho is used to test the generalization of inter rater reliability (IRR) statistics. Calculating rho starts by generating a large number of simulated, fullycoded data sets: a sizable collection of hypothetical populations, all of which have a kappa value below a given threshold  which indicates unacceptable agreement. Then kappa is calculated on a sample from each of those sets in the collection to see if it is equal to or higher than the kappa in then real sample. If less than five percent of the distribution of samples from the simulated data sets is greater than actual observed kappa, the null hypothesis is rejected and one can conclude that if the two raters had coded the rest of the data, we would have acceptable agreement (kappa above the threshold).
Package details 


Author  Brendan Eagan [aut], Brad Rogers [aut], Rebecca Pozen [aut], Cody L Marquart [cre, aut], David Williamson Shaffer [aut] 
Date of publication  20170216 22:38:21 
Maintainer  Cody L Marquart <cody.marquart@wisc.edu> 
License  GPL3  file LICENSE 
Version  1.1.0.0 
Package repository  View on CRAN 
Installation 
Install the latest version of this package by entering the following in R:

Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.