Optical diagnosis of early colorectal carcinoma: performance of an artificial intelligence algorithm vs endoscopists

Background and aims Early colorectal carcinomas (CRCs) are poorly recognized during colonoscopy. In this study, we compared early CRC diagnosis of a computer-aided diagnosis (CADx) based on artificial intelligence (AI) with the optical diagnosis of endoscopists. Methods A large training dataset with images and videos of colorectal lesions (≥10 mm or <10 mm with a suspicion of CRC) for CADx system development and a testing dataset of 50 videos from thirteen centers were collected. Colonoscopists were invited to also diagnose these 50 videos online. Primary outcomes were diagnostic performance of AI and endoscopists to predict presence of CRC. Endoscopist characteristics such as sex and endoscopy experience were collected and tested for association with diagnostic performance. Results Seventy-eight international endoscopists participated. AI and endoscopists reached sensitivities of 78.6% (95%CI 48.8-94.3%) and 89.2% (95%CI 87.2-90.9%), specificities of 83.3% (95%CI 66.5-93.0%) and 68.2% (95%CI 66.4-69.9%), and diagnostic accuracies of 82.0% (95%CI 68.1-91.0%) and 74.1% (95%CI 72.7-75.4%), respectively. Out of all endoscopists characteristics, only the number of annual colonoscopies, hospital type, and number of yearly T1 CRCs seen, showed statistically significant differences in one or more of the diagnostic performance measures. Conclusions In conclusion, the COMET CADx system for early CRC diagnosis showed higher diagnostic accuracy and specificity but lower sensitivity than the mean of endoscopists. Endoscopists with various characteristics could benefit from using AI. To further guide CADx system development, clear optical diagnosis thresholds for early CRC recognition could be helpful and could be informed by endoscopists with high exposure to early CRCs. Writing assistance Not applicable.