Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi methodSupported by: Wellcome Trust 203145Z/16/Z
Supported by: Engineering and Physical Sciences Research Council 203145Z/16/Z EP/P027938/1
Background Artificial intelligence (AI) research in colonoscopy is progressing rapidly but widespread clinical implementation is not yet a reality. We aimed to identify the top implementation research priorities.
Methods An established modified Delphi approach for research priority setting was used. Fifteen international experts, including endoscopists and translational computer scientists/engineers, from nine countries participated in an online survey over 9 months. Questions related to AI implementation in colonoscopy were generated as a long-list in the first round, and then scored in two subsequent rounds to identify the top 10 research questions.
Results The top 10 ranked questions were categorized into five themes. Theme 1: clinical trial design/end points (4 questions), related to optimum trial designs for polyp detection and characterization, determining the optimal end points for evaluation of AI, and demonstrating impact on interval cancer rates. Theme 2: technological developments (3 questions), including improving detection of more challenging and advanced lesions, reduction of false-positive rates, and minimizing latency. Theme 3: clinical adoption/integration (1 question), concerning the effective combination of detection and characterization into one workflow. Theme 4: data access/annotation (1 question), concerning more efficient or automated data annotation methods to reduce the burden on human experts. Theme 5: regulatory approval (1 question), related to making regulatory approval processes more efficient.
Conclusions This is the first reported international research priority setting exercise for AI in colonoscopy. The study findings should be used as a framework to guide future research with key stakeholders to accelerate the clinical implementation of AI in endoscopy.
Received: 24 June 2020
Accepted: 09 November 2020
09 November 2020 (online)
© 2020. © 2020. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
- 1 Kudo S, Mori Y, Misawa M. et al. Artificial intelligence and colonoscopy: Current status and future perspectives. Dig Endosc 2019; 31: 363-371
- 2 Ahmad OF, Soares AS, Mazomenos E. et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019; 4: 71-80
- 3 Mori Y, Kudo S, Berzin TM. et al. Computer-aided diagnosis for colonoscopy. Endoscopy 2017; 49: 813-819
- 4 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 2019; 68: 1813-1819
- 5 Wang P, Liu X, Berzin TM. et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020; 5: 343-351
- 6 Su J-R, Li Z, Shao X-J. et al. Impact of a real-time automatic quality control system on colorectal polyp and adenoma detection: a prospective randomized controlled study (with videos). Gastrointest Endosc 2020; 91: 415-424.e4
- 7 Liu W-N, Zhang Y-Y, Bian X-Q. et al. Study on detection rate of polyps and adenomas in artificial-intelligence-aided colonoscopy. Saudi J Gastroenterol 2020; 26: 13-19
- 8 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520.e7
- 9 He J, Baxter SL, Xu J. et al. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019; 25: 30-36
- 10 Langlotz CP, Allen B, Erickson BJ. et al. A roadmap for foundational research on artificial intelligence in medical imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791
- 11 Vinsard DG, Mori Y, Misawa M. et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc 2019; 90: 55-63
- 12 Rees CJ, Ngu WS, Regula J. et al. European Society of Gastrointestinal Endoscopy – Establishing the key unanswered research questions within gastrointestinal endoscopy. Endoscopy 2016; 48: 884-891
- 13 Francis N, Kazaryan AM, Pietrabissa A. et al. A research agenda for the European Association for Endoscopic Surgeons (EAES). Surg Endosc 2017; 31: 2042-2049
- 14 Burt CG, Cima RR, Koltun WA. et al. Developing a research agenda for the American Society of Colon and Rectal Surgeons: Results of a Delphi approach. Dis Colon Rectum 2009; 52: 898-905
- 15 Hart AL, Lomer M, Verjee A. et al. What are the top 10 research questions in the treatment of inflammatory bowel disease? A priority setting partnership with the James Lind Alliance. J Crohns Colitis 2017; 11: 204-211
- 16 Liu X, Cruz Rivera S, Moher D. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med 2020; 26: 1364-1374
- 17 Wang P, Liu P, Glissen Brown JR. et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study. Gastroenterology 2020; 159: 1252-1261.e5
- 18 Mori Y, Kudo S, Misawa M. et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: A prospective study. Ann Intern Med 2018; 169: 357-366
- 19 Hassan C, Repici A. The FUSE enigma: Wide-angle or wide-minded?. Gastrointest Endosc 2018; 88: 865-867
- 20 East JE, Rees CJ. Making optical biopsy a clinical reality in colonoscopy. Lancet Gastroenterol Hepatol 2018; 3: 10-12
- 21 Bisschops R, East JE, Hassan C. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline – Update 2019. Endoscopy 2019; 51: 1155-1179
- 22 Barua I, Vinsard D, Jodal H. et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy 2020; DOI: 10.1055/a-1201-7165.
- 23 Hassan C, Bhandari P, Antonelli G. et al. Artificial intelligence for non-polypoid colorectal neoplasms. Dig Endosc 2020; DOI: 10.1111/den.13807.
- 24 Bernal J, Tajkbaksh N, Sanchez FJ. et al. Comparative validation of polyp detection methods in video colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge. IEEE Trans Med Imaging 2017; 36: 1231-1249
- 25 Xu S, Perez M, Yang K. et al. Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer simulator. Surg Endosc 2014; 28: 2569-2576
- 26 Byrne MF, Chapados N, Soudan F. et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019; 68: 94-100
- 27 Mori Y, Kudo S-E, Misawa M. et al. Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy. VideoGIE 2019; 4: 7-10
- 28 Guizard N, Ghalehjegh SH, Henkel M. et al. Artificial intelligence for real-time multiple polyp detection with identification, tracking, and optical biopsy during colonoscopy. Gastroenterology 2019; 156: S48-S49
- 29 Sánchez-Montes C, Sánchez FJ, Bernal J. et al. Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy 2019; 51: 261-265
- 30 Lutnick B, Ginley B, Govind D. et al. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nat Mach Intell 2019; 1: 112-119
- 31 Borgli H, Thambawita V, Smedsrud PH. et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 2020; 7: 283
- 32 Ahmad OF, Stoyanov D, Lovat LB. Barriers and pitfalls for artificial intelligence in gastroenterology: Ethical and regulatory issues. Tech Innov Gastrointest Endosc 2020; 22: 80-84
- 33 Mori Y, Kudo S, East JE. et al. Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video). Gastrointest Endosc 2020; 92: 905-911.e1