Subscribe to RSS
DOI: 10.1055/a-2542-0943
Enhancing diagnostics: ChatGPT-4 performance in ulcerative colitis endoscopic assessment
Abstract
Background and study aims
The Mayo Endoscopic Subscore (MES) is widely utilized for assessing mucosal activity in ulcerative colitis (UC). Artificial intelligence has emerged as a promising tool for enhancing diagnostic precision and addressing interobserver variability. This study evaluated the diagnostic accuracy of ChatGPT-4, a multimodal large language model, in identifying and grading endoscopic images of UC patients using the MES.
Patients and methods
Real-world endoscopic images of UC patients were reviewed by an expert consensus board. Each image was graded based on the MES. Only images that were uniformly graded were subsequently provided to three inflammatory bowel disease (IBD) specialists and ChatGPT-4. Severity gradings of the IBD specialists and ChatGPT-4 were compared with assessments made by the expert consensus board.
Results
Thirty of 50 images were graded with complete agreement among the experts. Compared with the consensus board, ChatGPT-4 gradings had a mean accuracy rate of 78.9% whereas the mean accuracy rate for the IBD specialists was 81.1%. Between the two groups, there was no statistically significant difference in mean accuracy rates (P = 0.71) and a high degree of reliability was found.
Conclusions
ChatGPT-4 has the potential to assess mucosal inflammation severity from endoscopic images of UC patients, without prior configuration or fine-tuning. Performance rates were comparable to those of IBD specialists.
Keywords
Endoscopy Lower GI Tract - Inflammatory bowel disease - Diagnosis and imaging (inc chromoendoscopy, NBI, iSCAN, FICE, CLE...) - Quality and logistical aspects - Image and data processing, documentatitonPublication History
Received: 13 September 2024
Accepted after revision: 14 February 2025
Accepted Manuscript online:
18 February 2025
Article published online:
14 March 2025
© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/).
Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany
Asaf Levartovsky, Ahmad Albshesh, Ana Grinman, Eyal Shachar, Adi Lahat, Rami Eliakim, Uri Kopylov. Enhancing diagnostics: ChatGPT-4 performance in ulcerative colitis endoscopic assessment. Endosc Int Open 2025; 13: a25420943.
DOI: 10.1055/a-2542-0943
-
References
- 1 Colombel JF, Rutgeerts P, Reinisch W. et al. Early mucosal healing with infliximab is associated with improved long- term clinical outcomes in ulcerative colitis. Gastroenterology 2011; 141: 1194-1201
- 2 Turner D, Ricciuto A, Lewis A. et al. STRIDE-II: An update on the Selecting Therapeutic Targets in Inflammatory Bowel Disease (STRIDE) Initiative of the International Organization for the Study of IBD (IOIBD): determining therapeutic goals for treat-to-target strategies in IBD. Gastroenterology 2021; 160: 1570-1583
- 3 Schroeder K, Tremanie W, Ilstrup D. Coated Oral 5-aminosalicylic acid therapy for mildly to moderately active ulcerative colitis. A randomized study. N Engl J Med 1987; 317: 1625-1629
- 4 Osada T, Ohkusa T, Yokoyama T. et al. Comparison of several activity indices for the evaluation of endoscopic activity in UC: Inter- and intraobserver consistency. Inflammatory Bowel Diseases 2010; 16: 192-197
- 5 de Lange T, Larsen S, Aabakken L. Inter-observer agreement in the assessment of endoscopic findings in ulcerative colitis. BMC Gastroenterology 2004; 4: 9
- 6 Jahagirdar V, Bapaye J, Chandan S. et al. Diagnostic accuracy of convolutional neural network - based machine learning algorithms in endoscopic severity prediction of ulcerative colitis: a systematic review and meta-analysis Search strategy. Gastrointest Endosc 2023; 98: 145-154.e8
- 7 Takabayashi K, Kobayashi T, Matsuoka K. et al. Artificial intelligence quantifying endoscopic severity of ulcerative colitis in gradation scale. Dig Endosc 2024; 36: 582-590
- 8 Lo B, Liu ZY, Bendtsen F. et al. High accuracy in classifying endoscopic severity in ulcerative colitis using convolutional neural network. Am J Gastroenterol 2022; 117: 1648-1654
- 9 Lee TC, Staller K, Botoman V. et al. ChatGPT answers common patient questions about colonoscopy. Gastroenterology 2023; 165: 509-511.e7
- 10 Ghersin I, Weishoff R, Koifman E. et al. Comparative evaluation of a language model and human specialists in the application of European guidelines for the management of inflammatory bowel diseases and malignancies. Endoscopy 2024; 56: 706-709
- 11 Mukherjee S, Durkin C, PeBenito AM. et al. Assessing ChatGPT’s ability to reply to queries regarding colon cancer screening based on multisociety guidelines. Gastro Hep Advances 2023; 2: 1040-1043
- 12 Levartovsky A, Ben-Horin S, Kopylov U. et al. Towards AI-augmented clinical decision-making: an examination of chatgpt’s utility in acute ulcerative colitis presentations. Am J Gastroenterol 2023; 118: 2283-2289
- 13 Ogata N, Maeda Y, Misawa M. et al. Artificial intelligence-assisted video colonoscopy for disease monitoring of ulcerative colitis: A prospective study. J Crohns Colitis 2025; 19
- 14 Wei MT, Fay S, Yung D. et al. Artificial intelligence-assisted colonoscopy in real-world clinical practice: A systematic review and meta-analysis. Clin Transl Gastroenterol 2024; 15