Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2021; 09(05): E741-E748
DOI: 10.1055/a-1388-6735
Original article

Real-time deep learning-based colorectal polyp localization on clinical video footage achievable with a wide array of hardware configurations

Jeremi Podlasek‡
1   Department of Technology, moretho Ltd., Manchester, United Kingdom
,
Mateusz Heesch‡
1   Department of Technology, moretho Ltd., Manchester, United Kingdom
2   Department of Robotics and Mechatronics, AGH University of Science and Technology, Kraków, Poland
,
Robert Podlasek
3   Department of Surgery with the Trauma and Orthopedic Division, District Hospital in Strzyżów, Strzyżów, Poland
,
Wojciech Kilisiński
4   Department of Gastroenterology with IBD Unit, Voivodship Hospital No 2 in Rzeszow, Rzeszów, Poland
,
Rafał Filip
4   Department of Gastroenterology with IBD Unit, Voivodship Hospital No 2 in Rzeszow, Rzeszów, Poland
5   Faculty of Medicine, University of Rzeszów, Rzeszów, Poland
› Author Affiliations
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background and study aims Several computer-assisted polyp detection systems have been proposed, but they have various limitations, from utilizing outdated neural network architectures to a requirement for multi-graphics processing unit (GPU) processing, to validating on small or non-robust datasets. To address these problems, we developed a system based on a state-of-the-art convolutional neural network architecture able to detect polyps in real time on a single GPU and tested on both public datasets and full clinical examination recordings.

Methods The study comprised 165 colonoscopy procedure recordings and 2678 still photos gathered retrospectively. The system was trained on 81,962 polyp frames in total and then tested on footage from 42 colonoscopies and CVC-ClinicDB, CVC-ColonDB, Hyper-Kvasir, and ETIS-Larib public datasets. Clinical videos were evaluated for polyp detection and false-positive rates whereas the public datasets were assessed for F1 score. The system was tested for runtime performance on a wide array of hardware.

Results The performance on public datasets varied from an F1 score of 0.727 to 0.942. On full examination videos, it detected 94 % of the polyps found by the endoscopist with a 3 % false-positive rate and identified additional polyps that were missed during initial video assessment. The system’s runtime fits within the real-time constraints on all but one of the hardware configurations.

Conclusions We have created a polyp detection system with a post-processing pipeline that works in real time on a wide array of hardware. The system does not require extensive computational power, which could help broaden the adaptation of new commercially available systems.



Publication History

Received: 01 October 2020

Accepted: 30 December 2020

Article published online:
22 April 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Dahms S, Nowicki A. Epidemiology and results of treatment of colorectal cancer in Poland. Polish J Surgery 2015; 87: 598-605
    CrossrefPubMedGoogle Scholar
  • 2 Siegel R, DeSantis C, Jemal A. Colorectal cancer statistics, 2014. CA Cancer J Clin 2014; 64: 104-117
    CrossrefPubMedGoogle Scholar
  • 3 Brenner H, Chang-Claude J, Jansen L. et al. Reduced risk of colorectal cancer up to 10 years after screening, surveillance, or diagnostic colonoscopy. Gastroenterology 2014; 146: 709-717
    CrossrefPubMedGoogle Scholar
  • 4 Winawer SJ, Zauber AG, Ho MN. et al. Prevention of colorectal cancer by colonoscopic polypectomy. N Engl J Med 1993; 329: 1977-1981
    CrossrefPubMedGoogle Scholar
  • 5 Săftoiu A, Hassan C, Areia M. et al. Role of gastrointestinal endoscopy in the screening of digestive tract cancers in Europe: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2020; 52: 293-304
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Kaminski MF, Regula J, Kraszewska E. et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010; 362: 1795-1803
    CrossrefPubMedGoogle Scholar
  • 7 Corley DA, Jensen CD, Marks AR. et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014; 370: 1298-1306
    CrossrefPubMedGoogle Scholar
  • 8 Buchner AM, Shahid MW, Heckman MG. et al. Trainee participation is associated with increased small adenoma detection. Gastrointest Endosc 2011; 73: 1223-1231
    CrossrefPubMedGoogle Scholar
  • 9 Wang P, Berzin TM, Brown JRG. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
    CrossrefPubMedGoogle Scholar
  • 10 Yang YJ, Bang CS. Application of artificial intelligence in gastroenterology. World J Gastroenterol 2019; 25: 1666
    CrossrefPubMedGoogle Scholar
  • 11 Angermann Q, Aymeric H, Olivier R. Active learning for real time detection of polyps in videocolonoscopy. Medical Image Understanding and Analysis Conference 2016; 90: 182-187
    PubMedGoogle Scholar
  • 12 Otani K, Nakada A, Kurose Y. et al. Automatic detection of different types of small-bowel lesions on capsule endoscopy images using a newly developed deep convolutional neural network. Endoscopy 2020; 52: 786-791
    Article in Thieme ConnectPubMedGoogle Scholar
  • 13 Cho B-J, Bang CS, Park SW. et al. Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network. Endoscopy 2019; 51: 1121-1129
    Article in Thieme ConnectPubMedGoogle Scholar
  • 14 Pogorelov K, Ostroukhova O, Jeppsson M. et al. Deep learning and hand-crafted feature based approaches for polyp detection in medical videos. IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS) 2018; 381-386
    CrossrefPubMedGoogle Scholar
  • 15 Wang P, Xiao X, Brown JRG. et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nature Biomed Engineer 2018; 2: 741-748
    CrossrefPubMedGoogle Scholar
  • 16 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078
    CrossrefPubMedGoogle Scholar
  • 17 Lee JY, Jeong J, Song EM. et al. Real-time detection of colon polyps during colonoscopy using deep learning: systematic validation with four independent datasets. Sci Rep 2020; 10: 1-9
    CrossrefPubMedGoogle Scholar
  • 18 Guo Z, Nemoto D, Zhu X. et al. A polyp detection algorithm can detect small polyps: An ex vivo reading test compared with endoscopists. Digest Endosc 2020; DOI: 10.1111/den.13670.
    CrossrefPubMedGoogle Scholar
  • 19 Angermann Q, Bernal J, Sánchez-Montes C. et al. Towards real-time polyp detection in colonoscopy videos: Adapting still frame-based methodologies for video sequences analysis. In: Computer Assisted and Robotic Endoscopy and Clinical Image-Based Procedures. Springer; 2017: 29-41
    PubMedGoogle Scholar
  • 20 Borgli H, Thambawita V, Smedsrud PH. et al. Hyper-kvasir: A comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Dec 2019. [Online]. Available from: osf.io/mkzcq, Accessed 2020 Jun 3
    PubMedGoogle Scholar
  • 21 Bernal J, Sánchez FJ, Fernández-Esparrach G. et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graphics 2015; 43: 99-111
    CrossrefPubMedGoogle Scholar
  • 22 Bernal J, Sánchez J, Vilarino F. Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 2012; 45: 3166-3182
    CrossrefPubMedGoogle Scholar
  • 23 Silva J, Histace A, Romain O. et al. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comput Assist Radiology Surgery 2014; 9: 283-293
    CrossrefPubMedGoogle Scholar
  • 24 Lin T-Y, Goyal P, Girshick R. et al. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision 2017; 2980-2988
    PubMedGoogle Scholar
  • 25 Tan M, Le Q. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:190511946 2019.
    PubMedGoogle Scholar
  • 26 Wilkins AJ. Visual stress. Oxford University Press; 1995
    CrossrefGoogle Scholar
  • 27 Yu L, Chen H, Dou Q. et al. Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Informat 2016; 21: 65-75
    CrossrefPubMedGoogle Scholar
  • 28 Mohammed A, Yildirim S, Farup I. et al. Y-net: A deep convolutional neural network for polyp detection. arXiv preprint arXiv:180601907 2018.
    PubMedGoogle Scholar
  • 29 Fan D-P, Ji G-P, Zhou T. et al. Pranet: Parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2020: 263-273
    PubMedGoogle Scholar
  • 30 Guo YB, Matuszewski B. Giana polyp segmentation with fully convolutional dilation neural networks. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications: SCITEPRESS-Science and Technology Publications 2019; 632-641
    CrossrefPubMedGoogle Scholar
  • 31 Sornapudi S, Meng F, Yi S. Region-based automated localization of colonoscopy and wireless capsule endoscopy polyps. Applied Sci 2019; 9: 2404
    CrossrefPubMedGoogle Scholar
  • 32 Qadir HA, Balasingham I, Solhusvik J. et al. Improving automatic polyp detection using cnn by exploiting temporal dependency in colonoscopy video. IEEE J Biomed Health Informat 2019; 24: 180-193
    CrossrefPubMedGoogle Scholar
  • 33 Dijkstra W, Sobiecki A, Bernal J. et al. Towards a single solution for polyp detection, localization and segmentation in colonoscopy images. In: VISIGRAPP (4: VISAPP). 2019: 616-625
    PubMedGoogle Scholar
  • 34 Bernal J, Tajkbaksh N, Sánchez FJ. et al. Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Transact Med Imaging 2017; 36: 1231-1249
    CrossrefPubMedGoogle Scholar
  • 35 Gurudu SR, Boroff ES, Crowell MD. et al. Impact of feedback on adenoma detection rates: Outcomes of quality improvement program. J Gastroenterol Hepatol 2018; 33: 645-649
    CrossrefPubMedGoogle Scholar