Probability Models for Rater Agreement in Medical Diagnosis.

Abstract

<P>COHEN'S KAPPA IS AN AGREEMENT INDEX FOR THE CORRELATION BETWEEN CATEGORICAL VARIABLES OFTEN USED AS A RELIABILITY OR VALIDITY MEASURE. ALTHOUGH KAPPA HAS MEANING AS PERCENTAGE AGREEMENT CORRECTED FOR CHANCE, ITS DEFINITION DOES NOT GIVE A DIRECT PROBABILISTIC SUPPORT. IN THE THESIS, WE PROPOSE A NEW INDEX OF AGREEMENT BASED ON A PROBABILITY MODEL, WHICH CARRIES SOME USEFUL INTERPRETATIONS ALONG WITH PROVIDING INFORMATION ABOUT INTRINSIC AGREEMENT NOT DUE TO CHANCE. THE NEW INDEX RESULTS IN THE SAME FORM AS COHEN'S KAPPA IN PRACTICE AND THUS PROVIDES A NOVEL MEANS TO UNDERSTAND KAPPA. ESTIMATION BASED ON MAXIMUM LIKELIHOOD AND STATISTICAL TESTS FOR HYPOTHESIS RELATED TO THIS NEW INDEX ARE DEVELOPED FOR INFERENCES. THE MODEL IS ALSO ADJUSTED FOR COVARIATES. NUMERICAL ANALYSIS OF DATA FROM MEDICAL STUDIES AND MONTE CARLO SIMULATIONS ARE USED TO CONFIRM THE THEORETICAL FINDINGS AND DEMONSTRATE THE PRACTICAL APPLICATIONS. </P>