The impact of inconsistent human annotations on AI driven clinical decision making

Sylolypavan, A., Sleeman, D., Wu, H. and Sim, M. (2023) The impact of inconsistent human annotations on AI driven clinical decision making. npj Digital Medicine, 6(1), 26. (doi: 10.1038/s41746-023-00773-3) (PMID:36810915) (PMCID:PMC9944930)

Abstract

In supervised learning model development, domain experts are often used to provide the class labels (annotations). Annotation inconsistencies commonly occur when even highly experienced clinical experts annotate the same phenomenon (e.g., medical image, diagnostics, or prognostic status), due to inherent expert bias, judgments, and slips, among other factors. While their existence is relatively well-known, the implications of such inconsistencies are largely understudied in real-world settings, when supervised learning is applied on such ‘noisy’ labelled data. To shed light on these issues, we conducted extensive experiments and analyses on three real-world Intensive Care Unit (ICU) datasets. Specifically, individual models were built from a common dataset, annotated independently by 11 Glasgow Queen Elizabeth University Hospital ICU consultants, and model performance estimates were compared through internal validation (Fleiss’ κ = 0.383 i.e., fair agreement). Further, broad external validation (on both static and time series datasets) of these 11 classifiers was carried out on a HiRID external dataset, where the models’ classifications were found to have low pairwise agreements (average Cohen’s κ = 0.255 i.e., minimal agreement). Moreover, they tend to disagree more on making discharge decisions (Fleiss’ κ = 0.174) than predicting mortality (Fleiss’ κ = 0.267). Given these inconsistencies, further analyses were conducted to evaluate the current best practices in obtaining gold-standard models and determining consensus. The results suggest that: (a) there may not always be a “super expert” in acute clinical settings (using internal and external validation model performances as a proxy); and (b) standard consensus seeking (such as majority vote) consistently leads to suboptimal models. Further analysis, however, suggests that assessing annotation learnability and using only ‘learnable’ annotated datasets for determining consensus achieves optimal models in most cases.

• Item Type: Articles
• Additional Information: H.W. is supported by Medical Research Council (MR/S004149/1, MR/S004149/2); National Institute for Health Research (NIHR202639); British Council (UCL-NMU-SEU International Collaboration On Artificial Intelligence In Medicine: Tackling Challenges Of Low Generalisability And Health Inequality); Welcome Trust ITPA (PIII0054/005); The Alan Turing Institute, London, UK.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Sim, Malcolm
• Authors: Sylolypavan, A., Sleeman, D., Wu, H., and Sim, M.
• College/School: College of Medical Veterinary and Life Sciences > School of Medicine, Dentistry & Nursing
• Journal Name: npj Digital Medicine
• Publisher: Nature Research
• ISSN: 2398-6352
• ISSN (Online): 2398-6352
• Published Online: 21 February 2023
• Copyright Holders: Copyright © The Author(s) 2023
• First Published: First published in npj Digital Medicine 6(1):26
• Publisher Policy: Reproduced under a Creative Commons license