Artificial intelligence and the future of endoscopy: should we be quietly excited?

It is quite difficult these days to open a newspaper or webpage without reading about the advent, and possible threat, of artificial intelligence (AI) in our lives, whether that be self-driving cars, ever-improving facial recognition, automation of many hitherto manual jobs, and a litany of other applications. (In general literature, terms such as AI, deep learning, machine learning, and convolutional neural networks are used essentially synonymously.) Unsurprisingly, the field of medicine has not been immune from the “potential” that AI can bring, whether that be in image interpretation, drug discovery, “personalized medicine,” genomics, or hospital-wide data analysis, to name just a few opportunities.

“We need to start thinking less of ‘modular solutions’, such as ‘detection’ or ‘optical biopsy’, and move towards more comprehensive AI tools to help us in our practice, such as the ability to ‘track’ lesions in real time and apply unique identifiers to such lesions.”

In our own discipline of gastroenterology, there has been a significant amount of effort recently in relation to AI, in particular in the field of endoscopy. The challenge to improve human operator performance during endoscopy has been taken on by a number of groups and, after somewhat slow progress initially, the last couple of years has seen significant advances, with some developments already being in clinical use or at least in clinical trials in human subjects. For example, our own group and others have published on optical biopsy of colonic polyps in real time during colonoscopy [1], and others have shown great advances in AI-assisted colonic polyp detection [2].

The applications of AI to gastrointestinal endoscopy are not limited to the colon, however. There is promising work in optical biopsy of Barrett’s esophagus [3], assessment of mucosal healing in inflammatory bowel disease [4], and assessment of early gastric cancer, although it is fair to say that, at the current time of writing, AI “solutions” are at a more advanced stage of development for colonic disease than for other parts of the gastrointestinal tract [5].

It is therefore of great interest that, in this edition of Endoscopy, Wu et al. describe their application of deep neural networks to the endoscopic detection of early gastric cancer [6]. Gastric cancer is of course a common and often fatal malignancy, and the literature supports that the miss rate for early, potentially curable, lesions is significant. With the advent of advanced techniques such as endoscopic mucosal resection (EMR) and endoscopic submucosal dissection (ESD), we now have tools to be able to significantly improve outcomes from early gastric cancer, but we undoubtedly need to improve detection before we can see the full benefit of advanced endoscopic resection options. In their paper, Wu et al. describe using deep learning techniques to detect early gastric cancers, and also to “map” the stomach to address the issue of “blind spots” during gastroscopy. Using thousands of images in the training phase, the authors describe an impressive 92.5 % accuracy of their AI model to differentiate early gastric cancer from non-malignancy, and accuracies of over 90 % and 63 % to classify the gastric location of images for 10-part and 26-part gastric maps, respectively. Limitations of the study are addressed by the authors, and real-time clinical trials are planned.

I believe that we are now reaching a tipping point when it comes to AI and gastrointestinal endoscopy. There has been much promise, but we are now moving quickly towards clinical applicability of these AI models. The number of abstracts and presentations at international meetings has dramatically increased in the last couple of years. I personally review several articles a month that are put forward for consideration of publication in various journals.

So, what is the current state of AI in endoscopy, and what is the future? Well, as mentioned, we already have some AI products that have gained some regulatory approval, and will be available for clinical use in the near future (for optical biopsy of colonic polyps with endocytoscopy in Japan – www.cybernet.co.jp/English/documents/pdf/news/press/2018/20181210.pdf – and for volumetric laser endomicroscopy in the esophagus in the USA – www.octnews.org/articles/8323398/ninepoint-medical-announces-fda-clearance-of-an-ar).

How are we looking to initially embrace these and other contemporary AI solutions? It is very likely that such models will be used, and regarded, as forms of clinical decision support or “second readers” initially, at least until improvements are made, safety is confirmed, and consumer (patient and physician) confidence is gained. “Standalone” AI models in endoscopy making actionable decisions without human input are unlikely to be in our hands in the next few years and, depending on the actual problem being solved by the AI model, having a “human (physician) in the loop” (HITL) is the likely path for adoption of AI into our endoscopic practice for the foreseeable future [7].

If AI is helping us with grading the bowel preparation, or helping in (semi)automated report generation for an endoscopic procedure, we are more likely to accept some or full autonomy of proven AI models than in other domains such as optical biopsy, where human experience is invaluable. For example, I am thinking about how experts decide if a bowel lesion is or is not amenable to ESD based on human eye assessment of the surface features to gauge degree of submucosal invasion. Would having some AI assistance here be desirable? In my opinion, undoubtedly. However, at present there is no substitute for expert physician experience in assessment of such lesions “globally,” taking into account various factors such as lesion location, heterogeneity, and patient demographics. In such situations, we want to move towards AI being part of a “man and machine in perfect harmony” reality.

Despite the excitement, which I share, about the potential of AI in endoscopy, we do however need to encourage quality work, less copycat work, and more uniformity in model training and reporting of results. It is such a new and unfamiliar field for most physicians that it is easy to be bamboozled by various claims of what AI can do in our own field. Dedicated committees for AI in the various endoscopy organizations are necessary at this point to help us guide endoscopic research in this AI era, to set standards, to promote safety, but also to encourage thinking outside the box. We need to start thinking less of “modular solutions,” such as “detection” or “optical biopsy,” and move towards more comprehensive AI tools to help us in our practice, such as the ability to “track” lesions in real time and apply unique identifiers to such lesions [8]. In these ways, we will come to regard AI as an ally rather than a threat to our practice.

We should consider that disease is often a continuum in endoscopy, and that our human mind categorizes disease into distinct groupings, often based on what the human eye (and mind) can appreciate. AI can “see” detail in medical images that the human eye cannot appreciate (or sometimes actually cannot even see, such as in hyperspectral imaging), so if we can harness this power of AI, our understanding, assessment, and treatment of luminal disease will evolve. As I said earlier, “man and machine in perfect harmony.”

• References

• 1 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019; 68: 94-100
• 2 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078
• 3 Sehgal V, Rosenfeld A, Graham DG. et al. Machine learning creates a simple endoscopic classification system that improves dysplasia detection in Barrett’s oesophagus amongst non-expert endoscopists. Gastroenterol Res Pract 2018; 29: 1872437
• 4 Maeda Y, Kudo SE, Mori Y. et al. Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video). Gastrointest Endosc 2019; 89: 408-415
• 5 Byrne MF, Shahidi N, Rex DK. Will computer-aided detection and diagnosis revolutionize colonoscopy?. Gastroenterology 2017; 153: 1460-1464
• 6 Wu L, Zhou W, Wan X. et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy 2019; 51: 522-531
• 7 Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the-loop?. Brain Inf 2016; 3: 119-131
• 8 Bewley A, Ge Z, Ott L. et al. Simple online and realtime tracking. 2016 IEEE International Conference on Image Processing 2016; 3464-3468