Endoscopy 2021; 53(05): 469-477
DOI: 10.1055/a-1229-0920
Original article

A deep learning-based system for identifying differentiation status and delineating the margins of early gastric cancer in magnifying narrow-band imaging endoscopy

Tingsheng Ling*
1   Department of Gastroenterology, Nanjing Drum Tower Hospital of Nanjing University, Nanjing, China
2   Department of Gastroenterology, Nanjing Gaochun People’s Hospital, Nanjing, China
,
Lianlian Wu*
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Yiwei Fu
6   Department of Gastroenterology, Taizhou Peopleʼs Hospital, Taizhou, China
,
Qinwei Xu
7   Endoscopy Center, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, China
,
Ping An
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Jun Zhang
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Shan Hu
8   Technology Department, Wuhan EndoAngel Medical Technology Company, Wuhan, China
,
Yiyun Chen
9   School of Resources and Environmental Sciences of Wuhan University, Wuhan University, Wuhan, China
,
Xinqi He
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Jing Wang
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Xi Chen
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Jie Zhou
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Youming Xu
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Xiaoping Zou
1   Department of Gastroenterology, Nanjing Drum Tower Hospital of Nanjing University, Nanjing, China
2   Department of Gastroenterology, Nanjing Gaochun People’s Hospital, Nanjing, China
,
Honggang Yu
3   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
4   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
5   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
› Author Affiliations

Abstract

Background Accurate identification of the differentiation status and margins for early gastric cancer (EGC) is critical for determining the surgical strategy and achieving curative resection in EGC patients. The aim of this study was to develop a real-time system to accurately identify differentiation status and delineate the margins of EGC on magnifying narrow-band imaging (ME-NBI) endoscopy.

Methods 2217 images from 145 EGC patients and 1870 images from 139 EGC patients were retrospectively collected to train and test the first convolutional neural network (CNN1) to identify EGC differentiation status. The performance of CNN1 was then compared with that of experts using 882 images from 58 EGC patients. Finally, 928 images from 132 EGC patients and 742 images from 87 EGC patients were used to train and test CNN2 to delineate the EGC margins.

Results The system correctly predicted the differentiation status of EGCs with an accuracy of 83.3 % (95 % confidence interval [CI] 81.5 % – 84.9 %) in the testing dataset. In the man – machine contest, CNN1 performed significantly better than the five experts (86.2 %, 95 %CI 75.1 % – 92.8 % vs. 69.7 %, 95 %CI 64.1 % – 74.7 %). For delineating EGC margins, the system achieved an accuracy of 82.7 % (95 %CI 78.6 % – 86.1 %) in differentiated EGC and 88.1 % (95 %CI 84.2 % – 91.1 %) in undifferentiated EGC under an overlap ratio of 0.80. In unprocessed EGC videos, the system achieved real-time diagnosis of EGC differentiation status and EGC margin delineation in ME-NBI endoscopy.

Conclusion We developed a deep learning-based system to accurately identify differentiation status and delineate the margins of EGC in ME-NBI endoscopy. This system achieved superior performance when compared with experts and was successfully tested in real EGC videos.

* Joint first authors


Supplementary material



Publication History

Received: 05 March 2020

Accepted: 28 July 2020

Accepted Manuscript online:
28 July 2020

Article published online:
29 September 2020

© 2020. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Bray F, Ferlay J, Soerjomataram I. et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2018; 68: 394-424
  • 2 Veisani Y, Delpisheh A. Survival rate of gastric cancer in Iran; a systematic review and meta-analysis. Gastroenterol Hepatol Bed Bench 2016; 9: 78
  • 3 Katai H, Ishikawa T, Akazawa K. et al. Five-year survival analysis of surgically resected gastric cancer cases in Japan: a retrospective analysis of more than 100,000 patients from the nationwide registry of the Japanese Gastric Cancer Association (2001–2007). Gastric Cancer 2018; 21: 144-154
  • 4 Hirao M, Masuda K, Asanuma T. et al. Endoscopic resection of early gastric cancer and other tumors with local injection of hypertonic saline-epinephrine. Gastrointest Endosc 1988; 34: 264-269
  • 5 Bourke MJ, Neuhaus H, Bergman JJ. Endoscopic submucosal dissection: indications and application in Western endoscopy practice. Gastroenterology 2018; 154: 1887-1900 e5
  • 6 Japanese Gastric Cancer Association. Japanese gastric cancer treatment guidelines 2014 (ver. 4). Gastric Cancer 2017; 20: 1-19
  • 7 Huh CW, Jung DH, Kim JH. et al. Signet ring cell mixed histology may show more aggressive behavior than other histologies in early gastric cancer. J Surg Oncol 2013; 107: 124-129
  • 8 Nakamura R, Omori T, Mayanagi S. et al. Risk of lymph node metastasis in undifferentiated-type mucosal gastric carcinoma. World J Surg Oncol 2019; 17: 32
  • 9 Lee CK, Chung IK, Lee SH. et al. Is endoscopic forceps biopsy enough for a definitive diagnosis of gastric epithelial neoplasia?. J Gastroenterol Hepatol 2010; 25: 1507-1513
  • 10 Shim CN, Kim H, Kim DW. et al. Clinicopathologic factors and outcomes of histologic discrepancy between differentiated and undifferentiated types after endoscopic resection of early gastric cancer. Surg Endosc 2014; 28: 2097-2105
  • 11 Takao M, Kakushima N, Takizawa K. et al. Discrepancies in histologic diagnoses of early gastric cancer between biopsy and endoscopic mucosal resection specimens. Gastric Cancer 2012; 15: 91-96
  • 12 Lee JH, Kim JH, Rhee K. et al. Undifferentiated early gastric cancer diagnosed as differentiated histology based on forceps biopsy. Pathol Res Pract 2013; 209: 314-318
  • 13 Association JGC. Japanese gastric cancer treatment guidelines 2014 (ver. 4). Gastric Cancer 2017; 20: 1-19
  • 14 Iizuka H, Kakizaki S, Sohara N. et al. Stricture after endoscopic submucosal dissection for early gastric cancers and adenomas. Dig Endosc 2010; 22: 282-288
  • 15 Imagawa A, Okada H, Kawahara Y. et al. Endoscopic submucosal dissection for early gastric cancer: results and degrees of technical difficulty as well as success. Endoscopy 2006; 38: 987-990
  • 16 Mannen K, Tsunada S, Hara M. et al. Risk factors for complications of endoscopic submucosal dissection in gastric tumors: analysis of 478 lesions. J Gastroenterol 2010; 45: 30-36
  • 17 Abe S, Oda I, Suzuki H. et al. Short- and long-term outcomes of endoscopic submucosal dissection for undifferentiated early gastric cancer. Endoscopy 2013; 45: 703-707
  • 18 Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019; 25: 44-56
  • 19 Bi WL, Hosny A, Schabath MB. et al. Artificial intelligence in cancer imaging: Clinical challenges and applications. CA Cancer J Clin 2019; 69: 127-157
  • 20 Mori Y, Kudo SE, Misawa M. et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med 2018; 169: 357-366
  • 21 Ebigbo A, Mendel R, Probst A. et al. Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma. Gut 2019; 68: 1143-1145
  • 22 Wu L, Zhang J, Zhou W. et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut 2019; 68: 2161-2169
  • 23 Wu L, Zhou W, Wan X. et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy 2019; 51: 522-531
  • 24 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014 https://arxiv.org/abs/1409.1556
  • 25 Akiba T, Suzuki S, Fukuda K. Extremely large minibatch SGD: training resnet-50 on imagenet in 15 minutes. 2017 https://arxiv.org/abs/1711.04325
  • 26 Deng J, Dong W, Socher R. et al. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009: 248-255
  • 27 Shao L, Zhu F, Li X. Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst 2015; 26: 1019-1034
  • 28 Abadi M, Agarwal A, Barham P. et al. Tensorflow: A system for largescale machine learning. Proceedings of the 12th Symposium on Operating Systems Design and Implementation; 2016: 265-283 Available at (Accessed 13 August 2020): https: //www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf
  • 29 Baldi P, Sadowski P. The dropout learning algorithm. Artif Intell 2014; 210: 78-122
  • 30 Schulze M, Grobbel M, Riesenbeck A. et al. Dose rates of antimicrobial substances in boar semen preservation time to establish new protocols. Reprod Domest Anim 2017; 52: 397-402
  • 31 Dong J, Cheng YP, Jin K. et al. Effects of diffusion and suction negative pressure on coalbed methane extraction and a new measure to increase the methane utilization rate. Fuel 2017; 197: 70-81
  • 32 Zhou Z, Siddiquee MMR, Tajbakhsh N. et al. Unet++: A nested u-net architecture for medical image segmentation. In: Stoyanov D, Taylor Z, Carneiro G. et al. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Switzerland: Springer; 2018: 3-11
  • 33 Isomoto H, Shikuwa S, Yamaguchi N. et al. Endoscopic submucosal dissection for early gastric cancer: a large-scale feasibility study. Gut 2009; 58: 331-336
  • 34 Lee S, Choi KD, Han M. et al. Long-term outcomes of endoscopic submucosal dissection versus surgery in early gastric cancer meeting expanded indication including undifferentiated-type tumors: a criteria-based analysis. Gastric Cancer 2018; 21: 490-499
  • 35 Nakayoshi T, Tajiri H, Matsuda K. et al. Magnifying endoscopy combined with narrow band imaging system for early gastric cancer: correlation of vascular pattern with histopathology (including video). Endoscopy 2004; 36: 1080-1084
  • 36 Shibagaki K, Amano Y, Ishimura N. et al. Diagnostic accuracy of magnification endoscopy with acetic acid enhancement and narrow-band imaging in gastric mucosal neoplasms. Endoscopy 2016; 48: 16-25
  • 37 Feng F, Liu J, Wang F. et al. Prognostic value of differentiation status in gastric cancer. BMC Cancer 2018; 18: 865
  • 38 Li L, Chen Y, Shen Z. et al. Convolutional neural network for the diagnosis of early gastric cancer based on magnifying narrow band imaging. Gastric Cancer 2020; 23: 126-132
  • 39 Liu X, Wang C, Hu Y. et al. Transfer learning with convolutional neural network for early gastric cancer classification on magnifiying narrow-band imaging images. Proceedings of 2018 25th IEEE International Conference on Image Processing (ICIP); 2018: 1388-1392
  • 40 Yao K, Uedo N, Muto M. et al. Development of an e-learning system for teaching endoscopists how to diagnose early gastric cancer: basic principles for improving early detection. Gastric Cancer 2017; 20: 28-38
  • 41 Nakanishi H, Doyama H, Ishikawa H. et al. Evaluation of an e-learning system for diagnosis of gastric lesions using magnifying narrow-band imaging: a multicenter randomized controlled study. Endoscopy 2017; 49: 957-967