Optical diagnosis of early colorectal carcinoma: performance of a newly developed artificial intelligence algorithm vs international endoscopists

Aims Colorectal carcinomas (CRCs) are more often diagnosed at an early stage [1], enabling local resection strategies. As early CRC is poorly recognized by endoscopists, we have developed the first European artificial intelligence (AI) algorithm aiming to improve the optical diagnosis of early CRC. In this study, diagnosis by AI was compared to optical diagnosis by endoscopists. Additionally, to determine which endoscopists could benefit most from AI, we assessed endoscopist characteristics associated with optical diagnosis performance.

Methods We collected a large training dataset of endoscopic images and videos of colorectal lesions (≥ 10 mm or<10 mm but with a suspicion of CRC) in nine Dutch hospitals. A test set of 50 videos, each containing both high-definition white light and narrow band imaging, was selected from data collected in four different hospitals for external testing. In an online module containing the test set videos, endoscopists were invited to participate in the study and report their 50 optical diagnoses. Primary outcomes were the diagnostic performance of AI and the endoscopist to predict the presence of CRC. Additionally, endoscopist characteristics including years of endoscopy experience, participation in the OPTICAL training [2], and CRC screening program certification were collected and tested for association with diagnostic accuracy using Pearson correlation and an independent samples T-test.

Results Since September 2024, 50 international endoscopists participated in the online test. AI and the endoscopists reached a mean sensitivity of 80.0% (95%CI 44.2-96.5) and 85.8% (95%CI 83.7-87.9%), specificity of 80.0% (95%CI 58.7-92.4) and 64.5% (95%CI 61.1-67.8%), negative predictive value (NPV) of 90.9% (95%CI 69.4-98.4) and 92.1% (95%CI 91.1-93.1%), positive predictive value (PPV) of 61.5% (95%CI 32.3-84.9) and 50.5 (95%CI 48.2-52.9%), and diagnostic accuracy of 80.0% (95%CI 62.5-90.9) and 70.6% (95%CI 68.4-72.8%), respectively. The years of endoscopy experience (mean 8.6) showed a very weak and not statistically significant correlation with diagnostic accuracy. (Pearson correlation 0.114, p=0.431). The mean diagnostic accuracy was significantly lower for endoscopists that did not receive OPTICAL training (n=32/50) than for endoscopists who did (68.9% [95%CI 66.3-71.6] vs 73.5% [95%CI 69.6-77.4], p=0.045), and for endoscopists that were not certified for CRC screening programs (= 24/50) vs certified endoscopists (68.2% [95%CI 65.1-71.3] vs 72.7% [95%CI 69.6-75.8], p=0.038).

Conclusions AI outperformed a group of 50 endoscopists regarding diagnostic accuracy, specificity, and PPV for the optical diagnosis of early CRC. Sensitivity and NPV are similar and relatively high. These results indicate that AI could potentially improve the optical diagnosis of colorectal lesions≥10 mm or with a suspicion of CRC, particularly for endoscopists without OPTICAL training or who are not certified for CRC screening programs. Future research should focus on prospective validation of this AI algorithm in clinical practice including evaluation of the joint performance of AI and the endoscopists.

Publication History

Article published online:
27 March 2025

• References

• 1 Breekveldt ECH, Lansdorp-Vogelaar I, Toes-Zoutendijk E. et al. Colorectal cancer incidence, mortality, tumour characteristics, and treatment before and after introduction of the faecal immunochemical testing-based screening programme in the Netherlands: a population-based study. Lancet Gastroenterol Hepatol 2022; 7: 60-68
• 2 Backes Y, Schwartz MP, Ter Borg F. et al. Multicentre prospective evaluation of real-time optical diagnosis of T1 colorectal cancer in large non-pedunculated colorectal polyps using narrow band imaging (the OPTICAL study). Gut 2019; 68: 271-279