item.ICC: Item Intra-class Correlations

Description Usage Arguments Details Value Author(s) References See Also Examples

Description

Calculates the item (or rater) intra-class correlations for a single item and a composite of the items, following Shrout and Fleiss (1979), for the corresponding columns in each set provided to the function.

ICC1: Each target is rated by a different judge and the judges are selected at random. (This is a one-way ANOVA fixed effects model and is found by (MSB- MSW)/(MSB+ (nr-1)*MSW))

ICC2: A random sample of k judges rate each target. The measure is one of absolute agreement in the ratings. Found as (MSB- MSE)/(MSB + (nr-1)*MSE + nr*(MSJ-MSE)/nc)

ICC3: A fixed set of k judges rate each target. There is no generalization to a larger population of judges. (MSB - MSE)/(MSB+ (nr-1)*MSE)

Then, for each of these cases, is reliability to be estimated for a single rating or for the average of k ratings? (The 1 rating case is equivalent to the average intercorrelation, the k rating case to the Spearman Brown adjusted reliability.)

ICC1 is sensitive to differences in means between raters and is a measure of absolute agreement.

ICC2 and ICC3 remove mean differences between judges, but are sensitive to interactions of raters by judges. The difference between ICC2 and ICC3 is whether raters are seen as fixed or random effects.

ICC1k, ICC2k, ICC3K reflect the means of k raters.

Usage

1
item.ICC(set1, set2, ..., omit = TRUE)

Arguments

set1

A data.frame or matrix with corresponding columns in set2 and any additional sets passed to the function.

set2

A data.frame or matrix with correpsonding columns in set1 and any additional sets passed to the function.

...

Additional matrices or data.frames with corresponding columns to set1 and set2 passed to the function.

omit

omit A logical indicating whether incomplete cases should be omitted from analysis. If set to FALSE and data are missing, warning(s) will result.

Details

This function returns the ICCs for the corresponding columns from set1, set2, and each additional set to passed to the function where single item ICCs (ICC[x,1]) composite ICCs (ICC[x,k]) are computed for each group of corresponding columns in the sets. Follow Shrout and Fleiss (1979) for interpretation of the different ICCs.

Value

A data.frame containing the above described ICCs.

Author(s)

Ryne A. Sherman

References

Shrout, P. E. & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428

See Also

get.ICC Profile.ICC ICC

Examples

1
2
3
4
5
6
7
8
9
data(acq1)
data(acq2)
   #lets look at the item ICC between two aquaintance ratings of subjects' personality
   #on 100 personality traits. Notice the columns of each data.frame are corresponding.
names(acq1) 
names(acq2)
item.ICC(acq1, acq2)
   # We can get the descriptives for these using describe() from the 'psych' package   
describe(item.ICC(acq1, acq2))

Example output

Loading required package: psych
Loading required package: abind
Loading required package: foreach
  [1] "acq1CAQ001" "acq1CAQ002" "acq1CAQ003" "acq1CAQ004" "acq1CAQ005"
  [6] "acq1CAQ006" "acq1CAQ007" "acq1CAQ008" "acq1CAQ009" "acq1CAQ010"
 [11] "acq1CAQ011" "acq1CAQ012" "acq1CAQ013" "acq1CAQ014" "acq1CAQ015"
 [16] "acq1CAQ016" "acq1CAQ017" "acq1CAQ018" "acq1CAQ019" "acq1CAQ020"
 [21] "acq1CAQ021" "acq1CAQ022" "acq1CAQ023" "acq1CAQ024" "acq1CAQ025"
 [26] "acq1CAQ026" "acq1CAQ027" "acq1CAQ028" "acq1CAQ029" "acq1CAQ030"
 [31] "acq1CAQ031" "acq1CAQ032" "acq1CAQ033" "acq1CAQ034" "acq1CAQ035"
 [36] "acq1CAQ036" "acq1CAQ037" "acq1CAQ038" "acq1CAQ039" "acq1CAQ040"
 [41] "acq1CAQ041" "acq1CAQ042" "acq1CAQ043" "acq1CAQ044" "acq1CAQ045"
 [46] "acq1CAQ046" "acq1CAQ047" "acq1CAQ048" "acq1CAQ049" "acq1CAQ050"
 [51] "acq1CAQ051" "acq1CAQ052" "acq1CAQ053" "acq1CAQ054" "acq1CAQ055"
 [56] "acq1CAQ056" "acq1CAQ057" "acq1CAQ058" "acq1CAQ059" "acq1CAQ060"
 [61] "acq1CAQ061" "acq1CAQ062" "acq1CAQ063" "acq1CAQ064" "acq1CAQ065"
 [66] "acq1CAQ066" "acq1CAQ067" "acq1CAQ068" "acq1CAQ069" "acq1CAQ070"
 [71] "acq1CAQ071" "acq1CAQ072" "acq1CAQ073" "acq1CAQ074" "acq1CAQ075"
 [76] "acq1CAQ076" "acq1CAQ077" "acq1CAQ078" "acq1CAQ079" "acq1CAQ080"
 [81] "acq1CAQ081" "acq1CAQ082" "acq1CAQ083" "acq1CAQ084" "acq1CAQ085"
 [86] "acq1CAQ086" "acq1CAQ087" "acq1CAQ088" "acq1CAQ089" "acq1CAQ090"
 [91] "acq1CAQ091" "acq1CAQ092" "acq1CAQ093" "acq1CAQ094" "acq1CAQ095"
 [96] "acq1CAQ096" "acq1CAQ097" "acq1CAQ098" "acq1CAQ099" "acq1CAQ100"
  [1] "acq2CAQ001" "acq2CAQ002" "acq2CAQ003" "acq2CAQ004" "acq2CAQ005"
  [6] "acq2CAQ006" "acq2CAQ007" "acq2CAQ008" "acq2CAQ009" "acq2CAQ010"
 [11] "acq2CAQ011" "acq2CAQ012" "acq2CAQ013" "acq2CAQ014" "acq2CAQ015"
 [16] "acq2CAQ016" "acq2CAQ017" "acq2CAQ018" "acq2CAQ019" "acq2CAQ020"
 [21] "acq2CAQ021" "acq2CAQ022" "acq2CAQ023" "acq2CAQ024" "acq2CAQ025"
 [26] "acq2CAQ026" "acq2CAQ027" "acq2CAQ028" "acq2CAQ029" "acq2CAQ030"
 [31] "acq2CAQ031" "acq2CAQ032" "acq2CAQ033" "acq2CAQ034" "acq2CAQ035"
 [36] "acq2CAQ036" "acq2CAQ037" "acq2CAQ038" "acq2CAQ039" "acq2CAQ040"
 [41] "acq2CAQ041" "acq2CAQ042" "acq2CAQ043" "acq2CAQ044" "acq2CAQ045"
 [46] "acq2CAQ046" "acq2CAQ047" "acq2CAQ048" "acq2CAQ049" "acq2CAQ050"
 [51] "acq2CAQ051" "acq2CAQ052" "acq2CAQ053" "acq2CAQ054" "acq2CAQ055"
 [56] "acq2CAQ056" "acq2CAQ057" "acq2CAQ058" "acq2CAQ059" "acq2CAQ060"
 [61] "acq2CAQ061" "acq2CAQ062" "acq2CAQ063" "acq2CAQ064" "acq2CAQ065"
 [66] "acq2CAQ066" "acq2CAQ067" "acq2CAQ068" "acq2CAQ069" "acq2CAQ070"
 [71] "acq2CAQ071" "acq2CAQ072" "acq2CAQ073" "acq2CAQ074" "acq2CAQ075"
 [76] "acq2CAQ076" "acq2CAQ077" "acq2CAQ078" "acq2CAQ079" "acq2CAQ080"
 [81] "acq2CAQ081" "acq2CAQ082" "acq2CAQ083" "acq2CAQ084" "acq2CAQ085"
 [86] "acq2CAQ086" "acq2CAQ087" "acq2CAQ088" "acq2CAQ089" "acq2CAQ090"
 [91] "acq2CAQ091" "acq2CAQ092" "acq2CAQ093" "acq2CAQ094" "acq2CAQ095"
 [96] "acq2CAQ096" "acq2CAQ097" "acq2CAQ098" "acq2CAQ099" "acq2CAQ100"
                    ICC1         ICC1k         ICC2        ICC2k         ICC3
acq2CAQ001  0.2047021860  0.3398386562  0.203468552  0.338136883  0.202839280
acq2CAQ002  0.0159369848  0.0313739632  0.033854654  0.065492095  0.035134086
acq2CAQ003  0.0208405537  0.0408301838  0.018542916  0.036410672  0.018456299
acq2CAQ004  0.3187194247  0.4833771593  0.317758557  0.482271286  0.316864755
acq2CAQ005  0.1014524648  0.1842157842  0.099996110  0.181811752  0.099673013
acq2CAQ006  0.1907374203  0.3203685667  0.194635734  0.325849510  0.196529142
acq2CAQ007  0.0675414334  0.1265364160  0.065734634  0.123360228  0.065480873
acq2CAQ008 -0.0118186629 -0.0239200286  0.001944689  0.003881829  0.001999074
acq2CAQ009  0.0415289943  0.0797462088  0.039469658  0.075941915  0.039300778
acq2CAQ010  0.0912427897  0.1672272946  0.092022334  0.168535626  0.092180481
acq2CAQ011  0.0714111200  0.1333029286  0.073679690  0.137247060  0.074041461
acq2CAQ012  0.0648235369  0.1217545155  0.074812144  0.139209711  0.076445164
acq2CAQ013  0.0657538834  0.1233941239  0.079278609  0.146910369  0.081642422
acq2CAQ014  0.1969989449  0.3291547512  0.195794307  0.327471549  0.195208615
acq2CAQ015  0.2127770606  0.3508922910  0.212582418  0.350627577  0.212477346
acq2CAQ016  0.0201767368  0.0395553752  0.021525503  0.042143839  0.021584928
acq2CAQ017  0.2334456718  0.3785260708  0.237589294  0.383954993  0.240185953
acq2CAQ018  0.2686570137  0.4235297812  0.274039293  0.430189704  0.278133105
acq2CAQ019  0.1653783014  0.2838190847  0.164472597  0.282484273  0.164116410
acq2CAQ020  0.0717817775  0.1339484940  0.070702245  0.132067053  0.070538171
acq2CAQ021 -0.0713811109 -0.1537360734 -0.072634047 -0.156645922 -0.072464559
acq2CAQ022  0.0200903302  0.0393893161  0.033441626  0.064718946  0.034378441
acq2CAQ023  0.1885669819  0.3173013970  0.186960955  0.315024608  0.186223789
acq2CAQ024  0.1093277989  0.1971063901  0.107626668  0.194337445  0.107217112
acq2CAQ025  0.1365999685  0.2403659551  0.135539746  0.238723033  0.135207687
acq2CAQ026  0.2813407674  0.4391349664  0.280076637  0.437593545  0.279094774
acq2CAQ027  0.1875945276  0.3159235299  0.192867386  0.323367691  0.195403896
acq2CAQ028  0.0630134082  0.1185561871  0.072440789  0.135095176  0.073928433
acq2CAQ029  0.0296002824  0.0574985903  0.041667312  0.080001190  0.042730017
acq2CAQ030  0.0931272791  0.1703868907  0.091296787  0.167317980  0.090929710
acq2CAQ031  0.3595983891  0.5289773686  0.363144002  0.532803580  0.367210154
acq2CAQ032  0.0380297661  0.0732729779  0.035806849  0.069138082  0.035642126
acq2CAQ033  0.2599428607  0.4126264274  0.275343651  0.431795228  0.287301287
acq2CAQ034  0.0719258120  0.1341992350  0.070513381  0.131737506  0.070299405
acq2CAQ035  0.2361501149  0.3820735233  0.234913058  0.380452789  0.234154630
acq2CAQ036  0.1059994259  0.1916807973  0.108558344  0.195854994  0.109183380
acq2CAQ037  0.0927758162  0.1697984433  0.090755835  0.166409075  0.090353481
acq2CAQ038  0.0829309640  0.1531602046  0.084538326  0.155897350  0.084835711
acq2CAQ039  0.2288389039  0.3724473618  0.227383863  0.370517928  0.226529025
acq2CAQ040  0.2217092469  0.3629492819  0.220239158  0.360977037  0.219410283
acq2CAQ041  0.0608067429  0.1146424518  0.058716155  0.110919540  0.058455916
acq2CAQ042  0.1327106994  0.2343240855  0.132429380  0.233885455  0.132343525
acq2CAQ043 -0.0435755231 -0.0911217230 -0.013228851 -0.026812400 -0.014045738
acq2CAQ044 -0.0648248502 -0.1386368109 -0.058794823 -0.124935188 -0.059468353
acq2CAQ045  0.0002013654  0.0004026497 -0.002174706 -0.004358891 -0.002164418
acq2CAQ046  0.0727059850  0.1355562214  0.070707690  0.132076553  0.070404252
acq2CAQ047  0.0243529812  0.0475480263  0.022014332  0.043080281  0.021909298
acq2CAQ048  0.0730813288  0.1362083689  0.071682843  0.133776226  0.071467192
acq2CAQ049  0.1490929379  0.2594967439  0.147322181  0.256810482  0.146711561
acq2CAQ050  0.0872153863  0.1604381016  0.088077522  0.161895674  0.088244217
acq2CAQ051  0.2038611450  0.3386788349  0.203636903  0.338369324  0.203522254
acq2CAQ052  0.1260534766  0.2238854179  0.125135236  0.222435903  0.124872833
acq2CAQ053  0.0949415798  0.1734185304  0.099614317  0.181180466  0.100653649
acq2CAQ054  0.2356795631  0.3814574104  0.234698700  0.380171616  0.234097858
acq2CAQ055  0.1773928191  0.3013315798  0.175757576  0.298969072  0.175061573
acq2CAQ056  0.0501045072  0.0954276586  0.067817575  0.127020901  0.070444798
acq2CAQ057 -0.0211565291 -0.0432276042 -0.021620943 -0.044197478 -0.021601295
acq2CAQ058  0.0499626286  0.0951702990  0.058468692  0.110477886  0.059534769
acq2CAQ059  0.0985693092  0.1794503240  0.097040009  0.176912433  0.096711860
acq2CAQ060  0.1007048451  0.1829824690  0.098718281  0.179697166  0.098284057
acq2CAQ061  0.0853390088  0.1572577934  0.084028223  0.155029586  0.083788073
acq2CAQ062  0.1507541588  0.2620093225  0.153470041  0.266101477  0.154457952
acq2CAQ063  0.1715599442  0.2928743767  0.194367918  0.325474111  0.205693932
acq2CAQ064  0.0297469153  0.0577751967  0.027880240  0.054248033  0.027773374
acq2CAQ065  0.1892787102  0.3183084143  0.187725367  0.316109047  0.187008749
acq2CAQ066  0.2153333006  0.3543608992  0.214530635  0.353273320  0.214092628
acq2CAQ067  0.0569453033  0.1077544942  0.061948649  0.116669764  0.062613032
acq2CAQ068  0.2121005887  0.3499719259  0.215816435  0.355014833  0.217871460
acq2CAQ069  0.0194870094  0.0382290489  0.017126989  0.033677189  0.017044937
acq2CAQ070  0.0531451528  0.1009265488  0.053023166  0.100706552  0.053009507
acq2CAQ071 -0.0192336873 -0.0392217535 -0.020083118 -0.040989431 -0.020049699
acq2CAQ072  0.0825141706  0.1524491278  0.080739403  0.149415118  0.080428245
acq2CAQ073  0.1861139525  0.3138213695  0.191267901  0.321116519  0.193721389
acq2CAQ074  0.1274155710  0.2260312421  0.125554948  0.223098745  0.125021777
acq2CAQ075  0.0585951686  0.1107036389  0.069525071  0.130011110  0.071177850
acq2CAQ076  0.1406347671  0.2465903567  0.146374386  0.255369254  0.148356098
acq2CAQ077  0.0923807501  0.1691365398  0.099478410  0.180955641  0.101058990
acq2CAQ078  0.0308438871  0.0598420138  0.030782164  0.059725838  0.030778244
acq2CAQ079  0.0796970790  0.1476285905  0.077825643  0.144412306  0.077510409
acq2CAQ080  0.1940971511  0.3250944044  0.192776807  0.323240368  0.192147202
acq2CAQ081  0.1699365038  0.2905055159  0.171878838  0.293338923  0.172687007
acq2CAQ082  0.0962923760  0.1756691520  0.095361432  0.174118659  0.095165365
acq2CAQ083  0.1212371381  0.2162560158  0.119739067  0.213869589  0.119332204
acq2CAQ084  0.2314463837  0.3758935618  0.231819696  0.376385759  0.232045120
acq2CAQ085  0.1000545341  0.1819083163  0.100513157  0.182665979  0.100615707
acq2CAQ086 -0.1426673126 -0.3328166877 -0.130766702 -0.300878262 -0.133548453
acq2CAQ087  0.0204447740  0.0400703194  0.019923179  0.039067999  0.019901984
acq2CAQ088  0.1747821630  0.2975567190  0.177303970  0.301203384  0.178394290
acq2CAQ089  0.1111133243  0.2000035854  0.109281437  0.197031039  0.108832855
acq2CAQ090  0.1422242801  0.2490303920  0.140465606  0.246330280  0.139891973
acq2CAQ091  0.0524064663  0.0995935848  0.050211471  0.095621639  0.049979925
acq2CAQ092  0.1627855328  0.2799923602  0.162696430  0.279860547  0.162661806
acq2CAQ093  0.0688499944  0.1288300412  0.070438999  0.131607685  0.070680230
acq2CAQ094  0.1009522730  0.1833908253  0.110190648  0.198507614  0.112502745
acq2CAQ095  0.1052732200  0.1904926640  0.104316304  0.188924683  0.104093646
acq2CAQ096 -0.0498896064 -0.1050185468 -0.051153523 -0.107822549 -0.051030656
acq2CAQ097 -0.0621564533 -0.1325518601 -0.056400762 -0.119543891 -0.057018717
acq2CAQ098  0.1055091535  0.1908788419  0.105331032  0.190587307  0.105289100
acq2CAQ099  0.1235423689  0.2199158169  0.121694241  0.216982911  0.121183179
acq2CAQ100  0.0755988706  0.1405707511  0.079549706  0.147375717  0.080235550
                  ICC3k
acq2CAQ001  0.337267470
acq2CAQ002  0.067883159
acq2CAQ003  0.036243674
acq2CAQ004  0.481241151
acq2CAQ005  0.181277546
acq2CAQ006  0.328498714
acq2CAQ007  0.122913277
acq2CAQ008  0.003990172
acq2CAQ009  0.075629267
acq2CAQ010  0.168800822
acq2CAQ011  0.137874494
acq2CAQ012  0.142032620
acq2CAQ013  0.150960096
acq2CAQ014  0.326651955
acq2CAQ015  0.350484645
acq2CAQ016  0.042257726
acq2CAQ017  0.387338612
acq2CAQ018  0.435217746
acq2CAQ019  0.281958760
acq2CAQ020  0.131780768
acq2CAQ021 -0.156251838
acq2CAQ022  0.066471688
acq2CAQ023  0.313977498
acq2CAQ024  0.193669535
acq2CAQ025  0.238207842
acq2CAQ026  0.436394206
acq2CAQ027  0.326925312
acq2CAQ028  0.137678510
acq2CAQ029  0.081957969
acq2CAQ030  0.166701317
acq2CAQ031  0.537167096
acq2CAQ032  0.068830970
acq2CAQ033  0.446362153
acq2CAQ034  0.131363999
acq2CAQ035  0.379457524
acq2CAQ036  0.196871648
acq2CAQ037  0.165732458
acq2CAQ038  0.156402874
acq2CAQ039  0.369382249
acq2CAQ040  0.359862937
acq2CAQ041  0.110455079
acq2CAQ042  0.233751546
acq2CAQ043 -0.028491662
acq2CAQ044 -0.126456889
acq2CAQ045 -0.004338226
acq2CAQ046  0.131547033
acq2CAQ047  0.042879144
acq2CAQ048  0.133400616
acq2CAQ049  0.255882240
acq2CAQ050  0.162177232
acq2CAQ051  0.338211035
acq2CAQ052  0.222021244
acq2CAQ053  0.182897952
acq2CAQ054  0.379382975
acq2CAQ055  0.297961532
acq2CAQ056  0.131617806
acq2CAQ057 -0.044156426
acq2CAQ058  0.112379075
acq2CAQ059  0.176366945
acq2CAQ060  0.178977481
acq2CAQ061  0.154620769
acq2CAQ062  0.267585236
acq2CAQ063  0.341204225
acq2CAQ064  0.054045716
acq2CAQ065  0.315092452
acq2CAQ066  0.352679232
acq2CAQ067  0.117847288
acq2CAQ068  0.357790567
acq2CAQ069  0.033518553
acq2CAQ070  0.100681915
acq2CAQ071 -0.040919828
acq2CAQ072  0.148882159
acq2CAQ073  0.324567175
acq2CAQ074  0.222256635
acq2CAQ075  0.132896418
acq2CAQ076  0.258379955
acq2CAQ077  0.183566895
acq2CAQ078  0.059718458
acq2CAQ079  0.143869438
acq2CAQ080  0.322354826
acq2CAQ081  0.294515086
acq2CAQ082  0.173791772
acq2CAQ083  0.213220353
acq2CAQ084  0.376682828
acq2CAQ085  0.182835310
acq2CAQ086 -0.308265255
acq2CAQ087  0.039027249
acq2CAQ088  0.302775211
acq2CAQ089  0.196301641
acq2CAQ090  0.245447773
acq2CAQ091  0.095201678
acq2CAQ092  0.279809322
acq2CAQ093  0.132028645
acq2CAQ094  0.202251626
acq2CAQ095  0.188559451
acq2CAQ096 -0.107549640
acq2CAQ097 -0.120932870
acq2CAQ098  0.190518661
acq2CAQ099  0.216170170
acq2CAQ100  0.148551952
      vars   n mean   sd median trimmed  mad   min  max range  skew kurtosis
ICC1     1 100 0.11 0.09   0.10    0.11 0.08 -0.14 0.36  0.50  0.14     0.11
ICC1k    2 100 0.18 0.15   0.17    0.19 0.15 -0.33 0.53  0.86 -0.41     0.60
ICC2     3 100 0.11 0.09   0.10    0.11 0.08 -0.13 0.36  0.49  0.19     0.11
ICC2k    4 100 0.18 0.15   0.18    0.19 0.15 -0.30 0.53  0.83 -0.34     0.48
ICC3     5 100 0.11 0.09   0.10    0.11 0.09 -0.13 0.37  0.50  0.20     0.15
ICC3k    6 100 0.18 0.15   0.18    0.19 0.15 -0.31 0.54  0.85 -0.34     0.53
        se
ICC1  0.01
ICC1k 0.01
ICC2  0.01
ICC2k 0.01
ICC3  0.01
ICC3k 0.01

multicon documentation built on May 2, 2019, 3:18 a.m.