Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2022; 10(02): E171-E177
DOI: 10.1055/a-1675-1941
Original article

Deep learning and colon capsule endoscopy: automatic detection of blood and colonic mucosal lesions using a convolutional neural network

Miguel Mascarenhas
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
3   Faculty of Medicine of the University of Porto Porto, Portugal
,
Tiago Ribeiro
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
,
João Afonso
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
,
João P.S. Ferreira
4   Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal
5   INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.
,
Hélder Cardoso
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
3   Faculty of Medicine of the University of Porto Porto, Portugal
,
Patrícia Andrade
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
3   Faculty of Medicine of the University of Porto Porto, Portugal
,
Marco P.L. Parente
4   Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal
5   INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.
,
Renato N. Jorge
4   Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Porto, Portugal
5   INEGI – Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal.
,
Miguel Mascarenhas Saraiva
6   ManopH Gastroenterology Clinic, Porto, Portugal
,
Guilherme Macedo
1   Department of Gastroenterology, São João University Hospital, Porto, Portugal
2   WGO Gastroenterology and Hepatology Training Center, Porto, Portugal
3   Faculty of Medicine of the University of Porto Porto, Portugal
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Abstract

Background and study aims Colon capsule endoscopy (CCE) is a minimally invasive alternative to conventional colonoscopy. However, CCE produces long videos, making its analysis time-consuming and prone to errors. Convolutional neural networks (CNN) are artificial intelligence (AI) algorithms with high performance levels in image analysis. We aimed to develop a deep learning model for automatic identification and differentiation of significant colonic mucosal lesions and blood in CCE images.

Patients and methods A retrospective multicenter study including 124 CCE examinations was conducted for development of a CNN model, using a database of CCE images including anonymized images of patients with normal colon mucosa, several mucosal lesions (erosions, ulcers, vascular lesions and protruding lesions) and luminal blood. For CNN development, 9005 images (3,075 normal mucosa, 3,115 blood and 2,815 mucosal lesions) were ultimately extracted. Two image datasets were created and used for CNN training and validation.

Results The mean (standard deviation) sensitivity and specificity of the CNN were 96.3 % (3.9 %) and 98.2 % (1.8 %) Mucosal lesions were detected with a sensitivity of 92.0 % and a specificity of 98.5 %. Blood was detected with a sensitivity and specificity of 97.2 % and 99.9 %, respectively. The algorithm was 99.2 % sensitive and 99.6 % specific in distinguishing blood from mucosal lesions. The CNN processed 65 frames per second.

Conclusions This is the first CNN-based algorithm to accurately detect and distinguish colonic mucosal lesions and luminal blood in CCE images. AI may improve diagnostic and time efficiency of CCE exams, thus facilitating CCE adoption to routine clinical practice.



Publication History

Received: 15 April 2021

Accepted: 21 September 2021

Article published online:
16 February 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

Comment to this article:

Endoscopy International Open – recently publishedEndoscopy 2022; 54(03): 340-340
DOI: 10.1055/a-1683-0942

 

  • References

  • 1 Eliakim R, Fireman Z, Gralnek IM. et al. Evaluation of the PillCam Colon capsule in the detection of colonic pathology: results of the first multicenter, prospective, comparative study. Endoscopy 2006; 38: 963-970
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Niikura R, Yasunaga H, Yamada A. et al. Factors predicting adverse events associated with therapeutic colonoscopy for colorectal neoplasia: a retrospective nationwide study in Japan. Gastrointest Endosc 2016; 84: 971-982.e976
    CrossrefPubMedGoogle Scholar
  • 3 Eliakim R, Yassin K, Niv Y. et al. Prospective multicenter performance evaluation of the second-generation colon capsule compared with colonoscopy. Endoscopy 2009; 41: 1026-1031
    Article in Thieme ConnectPubMedGoogle Scholar
  • 4 Yasaka K, Akai H, Abe O. et al. Deep Learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: A preliminary study. Radiology 2018; 286: 887-896
    CrossrefPubMedGoogle Scholar
  • 5 Esteva A, Kuprel B, Novoa RA. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115-118
    CrossrefPubMedGoogle Scholar
  • 6 Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology 2017; 124: 962-969
    CrossrefPubMedGoogle Scholar
  • 7 Soffer S, Klang E, Shimon O. et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92: 831-839.e838
    CrossrefPubMedGoogle Scholar
  • 8 Aoki T, Yamada A, Aoyama K. et al. Clinical usefulness of a deep learning-based system as the first screening on small-bowel capsule endoscopy reading. Dig Endosc 2020; 32: 585-591
    CrossrefPubMedGoogle Scholar
  • 9 Hosoe N, Limpias Kamiya KJL, Hayashi Y. et al. Current status of colon capsule endoscopy. Dig Endosc 2020; DOI: 10.1111/den.13769.
    CrossrefPubMedGoogle Scholar
  • 10 Herrerías-Gutiérrez JM, Argüelles-Arias F, Caunedo-Álvarez A. et al. PillCamColon Capsule for the study of colonic pathology in clinical practice. Study of agreement with colonoscopy. Rev Esp Enferm Dig 2011; 103: 69-75
    CrossrefPubMedGoogle Scholar
  • 11 Shi HY, Chan FKL, Higashimori A. et al. A prospective study on second-generation colon capsule endoscopy to detect mucosal lesions and disease activity in ulcerative colitis (with video). Gastrointest Endosc 2017; 86: 1139-1146.e1136
    CrossrefPubMedGoogle Scholar
  • 12 Spada C, Hassan C, Galmiche JP. et al. Colon capsule endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2012; 44: 527-536
    Article in Thieme ConnectPubMedGoogle Scholar
  • 13 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python. J Machine Learning Res 2011; 12: 2825-2830
    PubMedGoogle Scholar
  • 14 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078.e1068
    CrossrefPubMedGoogle Scholar
  • 15 Wang K, Lu X, Zhou H. et al. Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study. Gut 2019; 68: 729-741
    CrossrefPubMedGoogle Scholar
  • 16 Marya NB, Powers PD, Chari ST. et al. Utilisation of artificial intelligence for the development of an EUS-convolutional neural network model trained to enhance the diagnosis of autoimmune pancreatitis. Gut 2020; DOI: 10.1136/gutjnl-2020-322821.
    CrossrefPubMedGoogle Scholar
  • 17 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363.e352
    CrossrefPubMedGoogle Scholar
  • 18 Aoki T, Yamada A, Kato Y. et al. Automatic detection of blood content in capsule endoscopy images based on a deep convolutional neural network. J Gastroenterol Hepatol 2020; 35: 1196-1200
    CrossrefPubMedGoogle Scholar
  • 19 Leenhardt R, Vasseur P, Li C. et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 2019; 89: 189-194
    CrossrefPubMedGoogle Scholar
  • 20 Klang E, Barash Y, Margalit RY. et al. Deep learning algorithms for automated detection of Crohnʼs disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020; 91: 606-613.e602
    CrossrefPubMedGoogle Scholar
  • 21 Wang X, Qian H, Ciaccio EJ. et al. Celiac disease diagnosis from videocapsule endoscopy images with residual learning and deep feature extraction. Comput Methods Programs Biomed 2020; 187: 105236
    CrossrefPubMedGoogle Scholar
  • 22 Saito H, Aoki T, Aoyama K. et al. Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2020; 92: 144-151.e141
    CrossrefPubMedGoogle Scholar
  • 23 Spada C, Barbaro F, Andrisani G. et al. Colon capsule endoscopy: What we know and what we would like to know. World J Gastroenterol 2014; 20: 16948-16955
    CrossrefPubMedGoogle Scholar
  • 24 Cash BD, Fleisher MR, Fern S. et al. Multicentre, prospective, randomised study comparing the diagnostic yield of colon capsule endoscopy versus CT colonography in a screening population (the TOPAZ study). Gut 2020; DOI: 10.1136/gutjnl-2020-322578.
    CrossrefPubMedGoogle Scholar
  • 25 Groth S, Krause H, Behrendt R. et al. Capsule colonoscopy increases uptake of colorectal cancer screening. BMC Gastroenterol 2012; 12: 80
    CrossrefPubMedGoogle Scholar
  • 26 Holleran G, Leen R, O'Morain C. et al. Colon capsule endoscopy as possible filter test for colonoscopy selection in a screening population with positive fecal immunology. Endoscopy 2014; 46: 473-478
    Article in Thieme ConnectPubMedGoogle Scholar
  • 27 Watabe H, Nakamura T, Yamada A. et al. Assessment of an electronic learning system for colon capsule endoscopy: a pilot study. J Gastroenterol 2016; 51: 579-585
    CrossrefPubMedGoogle Scholar
  • 28 Blanes-Vidal V, Baatrup G, Nadimi ES. Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning. Acta Oncol 2019; 58: S29-s36
    CrossrefPubMedGoogle Scholar
  • 29 Yamada A, Niikura R, Otani K. et al. Automatic detection of colorectal neoplasia in wireless colon capsule endoscopic images using a deep convolutional neural network. Endoscopy 2020; DOI: 10.1055/a-1266-1066.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 30 Sung J, Ho KY, Chiu HM. et al. The use of Pillcam Colon in assessing mucosal inflammation in ulcerative colitis: a multicenter study. Endoscopy 2012; 44: 754-758
    Article in Thieme ConnectPubMedGoogle Scholar
  • 31 Hosoe N, Matsuoka K, Naganuma M. et al. Applicability of second-generation colon capsule endoscope to ulcerative colitis: a clinical feasibility study. J Gastroenterol Hepatol 2013; 28: 1174-1179
    CrossrefPubMedGoogle Scholar
  • 32 Ye CA, Gao YJ, Ge ZZ. et al. PillCam colon capsule endoscopy versus conventional colonoscopy for the detection of severity and extent of ulcerative colitis. J Dig Dis 2013; 14: 117-124
    CrossrefPubMedGoogle Scholar
  • 33 San Juan-Acosta M, Caunedo-Álvarez A, Argüelles-Arias F. et al. Colon capsule endoscopy is a safe and useful tool to assess disease parameters in patients with ulcerative colitis. Eur J Gastroenterol Hepatol 2014; 26: 894-901
    CrossrefPubMedGoogle Scholar
  • 34 D’Haens G, Löwenberg M, Samaan MA. et al. Safety and feasibility of using the second-generation pillcam colon capsule to assess active colonic Crohn's disease. Clin Gastroenterol Hepatol 2015; 13: 1480-1486.e1483
    CrossrefPubMedGoogle Scholar