Subscribe to RSS
Deep learning and colon capsule endoscopy: automatic detection of blood and colonic mucosal lesions using a convolutional neural network
Background and study aims Colon capsule endoscopy (CCE) is a minimally invasive alternative to conventional colonoscopy. However, CCE produces long videos, making its analysis time-consuming and prone to errors. Convolutional neural networks (CNN) are artificial intelligence (AI) algorithms with high performance levels in image analysis. We aimed to develop a deep learning model for automatic identification and differentiation of significant colonic mucosal lesions and blood in CCE images.
Patients and methods A retrospective multicenter study including 124 CCE examinations was conducted for development of a CNN model, using a database of CCE images including anonymized images of patients with normal colon mucosa, several mucosal lesions (erosions, ulcers, vascular lesions and protruding lesions) and luminal blood. For CNN development, 9005 images (3,075 normal mucosa, 3,115 blood and 2,815 mucosal lesions) were ultimately extracted. Two image datasets were created and used for CNN training and validation.
Results The mean (standard deviation) sensitivity and specificity of the CNN were 96.3 % (3.9 %) and 98.2 % (1.8 %) Mucosal lesions were detected with a sensitivity of 92.0 % and a specificity of 98.5 %. Blood was detected with a sensitivity and specificity of 97.2 % and 99.9 %, respectively. The algorithm was 99.2 % sensitive and 99.6 % specific in distinguishing blood from mucosal lesions. The CNN processed 65 frames per second.
Conclusions This is the first CNN-based algorithm to accurately detect and distinguish colonic mucosal lesions and luminal blood in CCE images. AI may improve diagnostic and time efficiency of CCE exams, thus facilitating CCE adoption to routine clinical practice.
Received: 15 April 2021
Accepted: 21 September 2021
Article published online:
16 February 2022
© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
- 1 Eliakim R, Fireman Z, Gralnek IM. et al. Evaluation of the PillCam Colon capsule in the detection of colonic pathology: results of the first multicenter, prospective, comparative study. Endoscopy 2006; 38: 963-970
- 2 Niikura R, Yasunaga H, Yamada A. et al. Factors predicting adverse events associated with therapeutic colonoscopy for colorectal neoplasia: a retrospective nationwide study in Japan. Gastrointest Endosc 2016; 84: 971-982.e976
- 3 Eliakim R, Yassin K, Niv Y. et al. Prospective multicenter performance evaluation of the second-generation colon capsule compared with colonoscopy. Endoscopy 2009; 41: 1026-1031
- 4 Yasaka K, Akai H, Abe O. et al. Deep Learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: A preliminary study. Radiology 2018; 286: 887-896
- 5 Esteva A, Kuprel B, Novoa RA. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115-118
- 6 Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 2017; 124: 962-969
- 7 Soffer S, Klang E, Shimon O. et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92: 831-839.e838
- 8 Aoki T, Yamada A, Aoyama K. et al. Clinical usefulness of a deep learning-based system as the first screening on small-bowel capsule endoscopy reading. Dig Endosc 2020; 32: 585-591
- 9 Hosoe N, Limpias Kamiya KJL, Hayashi Y. et al. Current status of colon capsule endoscopy. Dig Endosc 2020; DOI: 10.1111/den.13769.
- 10 Herrerías-Gutiérrez JM, Argüelles-Arias F, Caunedo-Álvarez A. et al. PillCamColon Capsule for the study of colonic pathology in clinical practice. Study of agreement with colonoscopy. Rev Esp Enferm Dig 2011; 103: 69-75
- 11 Shi HY, Chan FKL, Higashimori A. et al. A prospective study on second-generation colon capsule endoscopy to detect mucosal lesions and disease activity in ulcerative colitis (with video). Gastrointest Endosc 2017; 86: 1139-1146.e1136
- 12 Spada C, Hassan C, Galmiche JP. et al. Colon capsule endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2012; 44: 527-536
- 13 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python. J Machine Learning Res 2011; 12: 2825-2830
- 14 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078.e1068
- 15 Wang K, Lu X, Zhou H. et al. Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study. Gut 2019; 68: 729-741
- 16 Marya NB, Powers PD, Chari ST. et al. Utilisation of artificial intelligence for the development of an EUS-convolutional neural network model trained to enhance the diagnosis of autoimmune pancreatitis. Gut 2020; DOI: 10.1136/gutjnl-2020-322821.
- 17 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363.e352
- 18 Aoki T, Yamada A, Kato Y. et al. Automatic detection of blood content in capsule endoscopy images based on a deep convolutional neural network. J Gastroenterol Hepatol 2020; 35: 1196-1200
- 19 Leenhardt R, Vasseur P, Li C. et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2019; 89: 189-194
- 20 Klang E, Barash Y, Margalit RY. et al. Deep learning algorithms for automated detection of Crohnʼs disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020; 91: 606-613.e602
- 21 Wang X, Qian H, Ciaccio EJ. et al. Celiac disease diagnosis from videocapsule endoscopy images with residual learning and deep feature extraction. Comput Methods Programs Biomed 2020; 187: 105236
- 22 Saito H, Aoki T, Aoyama K. et al. Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2020; 92: 144-151.e141
- 23 Spada C, Barbaro F, Andrisani G. et al. Colon capsule endoscopy: What we know and what we would like to know. World J Gastroenterol 2014; 20: 16948-16955
- 24 Cash BD, Fleisher MR, Fern S. et al. Multicentre, prospective, randomised study comparing the diagnostic yield of colon capsule endoscopy versus CT colonography in a screening population (the TOPAZ study). Gut 2020; DOI: 10.1136/gutjnl-2020-322578.
- 25 Groth S, Krause H, Behrendt R. et al. Capsule colonoscopy increases uptake of colorectal cancer screening. BMC Gastroenterol 2012; 12: 80
- 26 Holleran G, Leen R, O'Morain C. et al. Colon capsule endoscopy as possible filter test for colonoscopy selection in a screening population with positive fecal immunology. Endoscopy 2014; 46: 473-478
- 27 Watabe H, Nakamura T, Yamada A. et al. Assessment of an electronic learning system for colon capsule endoscopy: a pilot study. J Gastroenterol 2016; 51: 579-585
- 28 Blanes-Vidal V, Baatrup G, Nadimi ES. Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning. Acta Oncol 2019; 58: S29-s36
- 29 Yamada A, Niikura R, Otani K. et al. Automatic detection of colorectal neoplasia in wireless colon capsule endoscopic images using a deep convolutional neural network. Endoscopy 2020; DOI: 10.1055/a-1266-1066.
- 30 Sung J, Ho KY, Chiu HM. et al. The use of Pillcam Colon in assessing mucosal inflammation in ulcerative colitis: a multicenter study. Endoscopy 2012; 44: 754-758
- 31 Hosoe N, Matsuoka K, Naganuma M. et al. Applicability of second-generation colon capsule endoscope to ulcerative colitis: a clinical feasibility study. J Gastroenterol Hepatol 2013; 28: 1174-1179
- 32 Ye CA, Gao YJ, Ge ZZ. et al. PillCam colon capsule endoscopy versus conventional colonoscopy for the detection of severity and extent of ulcerative colitis. J Dig Dis 2013; 14: 117-124
- 33 San Juan-Acosta M, Caunedo-Álvarez A, Argüelles-Arias F. et al. Colon capsule endoscopy is a safe and useful tool to assess disease parameters in patients with ulcerative colitis. Eur J Gastroenterol Hepatol 2014; 26: 894-901
- 34 D’Haens G, Löwenberg M, Samaan MA. et al. Safety and feasibility of using the second-generation pillcam colon capsule to assess active colonic Crohn's disease. Clin Gastroenterol Hepatol 2015; 13: 1480-1486.e1483