Assessing Agreement with Intraclass Correlation Coefficient and Concordance Correlation Coefficient

Abstract

Accurate and precise measurement serves as a basis for studies in bioscience research. Agreement studies are often concerned with assessing whether different observers (e.g. machines, raters, methods, instruments, laboratories, assays, devices, etc.) for measuring responses on the same subject or sample can produce similar results. The intraclass correlation coefficient (ICC) and the concordance correlation coefficient (CCC) are two popular scaled indices (with values between -1 and 1) for assessing agreement (closeness) for continuous measurements, where these two indices may take the systematic shifts into account when assessing reliability between multiple observers. We conducted systematic and in-depth comparisons of these two indices under a general model since ICC depends on specific ANOVA models while CCC does not. Usually, the ICC and CCC are used for data without and with replications based on subject and observer effects only. However, we can not use the methodology if repeated measurements rather than replications are collected. There exist some ICC and CCC type indices for assessing agreement with repeated measurements. However, there is no CCC for random observers and random time points, we consider a new CCC for repeated measures where both observers and time are treated as random effects and also summarize other remaining combinations of random or fixed factors for observers and time. Finally, we compare ICCs and CCCs for data with repeated measurements.