Künstliche Intelligenz in der Endoskopie: Neuronale Netze und maschinelles Sehen – Techniken und Perspektiven

Zusammenfassung

Künstliche neuronale Netze als Methoden der künstlichen Intelligenz (KI) können der Endoskopie neue Möglichkeiten eröffnen, etwa im Sinne einer automatischen Polypenerkennung oder der präzisen Vorhersage des histopathologischen Befunds einer Läsion anhand ihres endoskopischen Bildes. Während erste Versuche tatsächlich ein weitreichendes Potenzial erahnen lassen, leiten sich öffentliche und medial transportierte Erwartungen häufig mehr von einer abstrakten Faszination als von der detaillierten Funktionsweise der Methoden ab. Dieser Artikel soll anhand einer selektiven Literaturübersicht ein intuitives Verständnis der Methoden vermitteln und helfen, die Lücke zwischen Funktion und Faszination zu schließen, um Potenzial und Grenzen dieser Techniken im Bereich der Endoskopie realistisch abschätzen zu können.

Mit ihrem Erfolg bei der maschinellen Klassifikation von Bildern haben insbesondere „tiefe neuronale Netze“ der KI nach jahrzehntelanger Forschung zu rasant anwachsendem Interesse verholfen. Wir umreißen kurz die diesbezüglichen Entwicklungen und die Gründe für ihre Bedeutung weit über die Informatik hinaus. Durch den Vergleich von maschinellem und menschlichem Sehen wird ein Verständnis der detaillierten Funktionsweise dieser Methoden und ihrer Erfolge bei Seh-Aufgaben vermittelt. Darauf aufbauend analysieren wir die Funktionsweise jüngst demonstrierter Anwendungen in Hinblick auf methodische Perspektiven und Grenzen, die Aussagekraft bisher erbrachter Leistungsnachweise und die Notwendigkeit weiterer Tests. Zudem geben wir einen Eindruck von weiteren, konkret absehbaren Einzelanwendungen und besprechen, welchen Charakter diese dem Einsatz der künstlichen Intelligenz in der Endoskopie insgesamt geben könnten.

Abstract

Artificial neural networks, as a specific approach towards artificial intelligence (AI), can open up a variety of new perspectives for endoscopy, such as automated lesion detection and the precise prediction of a lesion’s histology by its endoscopic appearance. Whilst early experiments do suggest an enormous potential for these methods, public expectations on their application in various fields of medicine sometimes appear to be grounded on general fascination rather than detailed understanding of their inner workings. Based on a selective review of the literature, this article shall convey an intuitive understanding of the underlying methods in order to help close the gap between functioning and fascination and allow for a realistic discussion of their perspectives and limitations in endoscopy.

After decades of research, the success of deep neuronal networks in image classification has provoked rising interest for AI during recent years. We quickly touch upon the developments surrounding this breakthrough and the reasons for their impact on various disciplines much beyond computer science. Through a comparison with the functioning of the human vision system, we aim to understand the mechanisms of these techniques and their success in computer vision tasks in detail. Based on these considerations, we analyse the functioning of some important AI applications in endoscopy, deduce specific limitations and perspectives, discuss the current state of their evaluation in practical endoscopy and make a plea for the need for additional and realistic tests. Moreover, we seek to give an impression of some further specific applications that can currently be foreseen and how these can shape the role that AI might finally acquire in the routine clinical practice of GI endoscopy.

Schlüsselwörter

Endoskopie - künstliche Intelligenz - deep learning - tiefe neuronale Netze - computer-aided detection - computer-aided diagnosis - maschinelles Sehen

Key words

endoscopy - artificial intelligence - deep learning - computer-aided detection - computer-aided diagnosis - computer vision

• Literatur

• 1 Misawa M, Kudo S, Mori Y. et al. Artificial Intelligence-Assisted Polyp Detection for Colonoscopy: Initial Experience. Gastroenterology 2018; 154: 2027-2029.e3
• 2 Russakovsky O, Deng J, Su H. et al. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 2015; 115: 211-252
• 3 Rosenblatt F. The Perceptron. A Perceiving and Recocgnizing Automaton. Cornell Aeronautical Laboratory; 1957 Technical Report 85-460-1
• 4 Hubel DH, Wiesel TN. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology 1962; 160: 106-154
• 5 Desimone R, Albright T, Gross C. et al. Stimulus-selective properties of inferior temporal neurons in the macaque. The Journal of Neuroscience 1984; 4: 2051-2062
• 6 Zeiler MD, Fergus R. Visualizing and Understanding Convolutional Networks. arXiv:13112901 [cs] 2013
• 7 Gross CG. Genealogy of the “Grandmother Cell”. The Neuroscientist 2002; 8: 512-518
• 8 Photographs by Maximilian Groß (Grand mother, private photo, by courtesy of the photographer), Arne J. de Vries (tree, private photo, by courtesy of the photographer), Rüdiger Schmitz (Zurich Opera House) and Thomas Rösch (endoscopy images).
• 9 Minsky M, Papert S. Perceptrons: An Introduction to Computational Geometry. Cambridge, MA, USA: MIT Press; 1969
• 10 LeCun Y. Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 1998; 86: 47
• 11 LeCun Y, Fu Jie H, Bottou L. Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. Washington, DC, USA: IEEE; 2004: 97-104
• 12 Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira F, Burges CJC, Bottou L. et al. (Hrsg) Advances in Neural Information Processing Systems 25. Curran Associates, Inc; 2012: 1097-1105
• 13 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444
• 14 Ciresan D, Giusti A, Gambardella LM. et al. Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images. In: Pereira F, Burges CJC, Bottou L. et al. (Hrsg) Advances in Neural Information Processing Systems 25. Curran Associates, Inc; 2012: 2843-2851
• 15 Redmon J, Divvala S, Girshick R. et al. You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Las Vegas, NV, USA: IEEE; 2016: 779-788
• 16 Liu W, Anguelov D, Erhan D. et al. SSD: Single Shot MultiBox Detector. arXiv:151202325 [cs] 2016; 9905: 21–37
• 17 Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 3431-3440
• 18 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM. et al. (Hrsg) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015: 234-241
• 19 Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2017; 39: 2481-2495
• 20 Hassan C, Senore C, Radaelli F. et al. Full-spectrum (FUSE) versus standard forward-viewing colonoscopy in an organised colorectal cancer screening programme. Gut 2017; 66: 1949-1955
• 21 Urban G, Tripathi P, Alkayali T. et al. Deep Learning Localizes and Identifies Polyps in Real Time with 96% Accuracy in Screening Colonoscopy. Gastroenterology 2018; 155: 1069-1078.e8
• 22 Wang P, Xiao X, Glissen Brown JR. et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nature Biomedical Engineering 2018; 2: 741-748
• 23 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; DOI: gutjnl-2018-317500.
• 24 Hirasawa T, Aoyama K, Tanimoto T. et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 2018; 21: 653-660
• 25 Shichijo S, Nomura S, Aoyama K. et al. Application of Convolutional Neural Networks in the Diagnosis of Helicobacter pylori Infection Based on Endoscopic Images. EBioMedicine 2017; 25: 106-111
• 26 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2017; DOI: gutjnl-2017-314547.
• 27 Mori Y, Kudo S, Misawa M. et al. Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy: A Prospective Study. Annals of Internal Medicine 2018; 169: 357
• 28 Mori Y, Kudo S, Misawa M. et al. Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy. VideoGIE 2019; 4: 7-10
• 29 Horie Y, Yoshio T, Aoyama K. et al. The diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointestinal Endoscopy 2019; 89: 25-32
• 30 Săftoiu A, Vilmann P, Gorunescu F. et al. Efficacy of an Artificial Neural Network–Based Approach to Endoscopic Ultrasound Elastography in Diagnosis of Focal Pancreatic Masses. Clinical Gastroenterology and Hepatology 2012; 10: 84-90.e1
• 31 Shibata J, Ozawa T, Ishihara S. et al. Novel computer-aided diagnosis system using convolutional neural networks for endoscopic disease activity in patiens with ulcerative colitis. Oral presentation at the United European Gastroenterology Week (UEGW). Wien: 2018
• 32 Mendel R, Ebigbo A, Probst A. et al. Barrett’s Esophagus Analysis Using Convolutional Neural Networks. In: Maier-Hein, geb. Fritzsche KH, Deserno, geb. Lehmann TM, Handels H, Tolxdorff T (Hrsg) Bildverarbeitung für die Medizin 2017. Berlin, Heidelberg: Springer Berlin Heidelberg; 2017: 80-85
• 33 de Groof J, van der Putten J, van der Sommen F. et al. Tu2011 – The Argos Project: Evaluation of Results of a Clinicallyinspired Algorithm vs. a Deep Learning Algorithm for the Detection and Delineation of Barrett’s Neoplasia. Gastroenterology 2018; 154: S-1368
• 34 de Groof J, van der Putten J, Struyvenberg MR. et al. The Argos Project II: The Development Of An Endoscopic Deep Learning Computer Aided Detection System For Barrett’s Neoplasia. Oral presentation at the United European Gastroenterology Week (UEGW). Wien: 2018
• 35 Ebigbo A, Mendel R, Probst A. et al. Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 2018; DOI: gutjnl-2018-317573.
• 36 Miao S, Wang ZJ, Liao R. A CNN Regression Approach for Real-Time 2D/3D Registration. IEEE Transactions on Medical Imaging 2016; 35: 1352-1363
• 37 de Vos BD, Berendsen FF, Viergever MA. et al. End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network. In: Cardoso MJ, Arbel T, Carneiro G. et al. (Hrsg) Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA 2017, ML-CDS 2017. Cham: Springer International Publishing; 2017: 204-212
• 38 Rohé MM, Datar M, Heimann T. et al. SVF-Net: Learning Deformable Image Registration Using Shape Matching. In: Descoteaux M, Maier-Hein L, Franz A. et al. (Hrsg) Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. Cham: Springer International Publishing; 2017: 266-274
• 39 Sentker T, Madesta F, Werner R. GDL-FIRE4D: Deep Learning-Based Fast 4D CT Image Registration. In: Frangi AF, Schnabel JA, Davatzikos C. et al. (Hrsg) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Cham: Springer International Publishing; 2018: 765-773
• 40 Turan M, Almalioglu Y, Araujo H. et al. Deep EndoVO: A recurrent convolutional neural network (RCNN) based visual odometry approach for endoscopic capsule robots. Neurocomputing 2018; 275: 1861-1870
• 41 Chen H, Ni D, Qin J. et al. Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks. IEEE Journal of Biomedical and Health Informatics 2015; 19: 1627-1636
• 42 Schachschal G, Mayr M, Treszl A. et al. Endoscopic versus histological characterisation of polyps during screening colonoscopy. Gut 2014; 63: 458-465
• 43 Ladabaum U, Fioritto A, Mitani A. et al. Real-Time Optical Biopsy of Colon Polyps With Narrow Band Imaging in Community Practice Does Not Yet Meet Key Thresholds for Clinical Decisions. Gastroenterology 2013; 144: 81-91
• 44 Chen PJ, Lin MC, Lai MJ. et al. Accurate Classification of Diminutive Colorectal Polyps Using Computer-Aided Analysis. Gastroenterology 2018; 154: 568-575
• 45 Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Computation 1997; 9: 1735-1780
• 46 Fernández S, Graves A, Schmidhuber J. An Application of Recurrent Neural Networks to Discriminative Keyword Spotting. In: de Sá JM, Alexandre LA, Duch W. et al. (Hrsg) Artificial Neural Networks – ICANN 2007. Berlin, Heidelberg: Springer Berlin Heidelberg; 2007: 220-229
• 47 Schmitz R, Krause J, Krech T. et al. Virtual Endoscopy Based on 3-Dimensional Reconstruction of Histopathology Features of Endoscopic Resection Specimens. Gastroenterology 2018; 154: 1234-1236.e4
• 48 Nguyen A, Yosinski J, Clune J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Boston, MA, USA: IEEE; 2015: 427-436
• 49 Szegedy C, Zaremba W, Sutskever I. et al. Intriguing properties of neural networks. arXiv:13126199 [cs] 2013
• 50 Selvaraju RR, Cogswell M, Das A. et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In: 2017 IEEE International Conference on Computer Vision (ICCV) Venice: IEEE; 2017: 618-626
• 51 Vinsard DG, Mori Y, Misawa M. et al. Quality Assurance of Computer-Aided Detection and Diagnosis in Colonoscopy. Gastrointestinal Endoscopy 2019; DOI: 10.1016/j.gie.2019.03.019.