Deep Learning and Automatic Capsule Endoscopy Segmentation – A Crucial Step Towards AI-assisted Capsule Panendoscopy

Aims Capsule endoscopy (CE) is a minimally invasive exam for evaluating the entire gastrointestinal tract. Nevertheless, CE time-consuming nature, with reading times up to 120 minutes, limit widespread use. Artificial intelligence (AI) models have demonstrated high diagnostic accuracy for identification of pleomorphic lesions in CE videos, however automated segmentation of organ topographies is fundamental for its clinical implementation. Our group aimed to develop the first deep learning model for automatic segmentation of CE videos, categorizing each frame to esophageal, gastric, enteric or colonic topography.

Methods A total 481859 images (810 esophageal, 1553 gastric, 33608 enteric, 12023 colonic) from 1841 patients and 8 different CE and colon CE devices were included. Frames were split in training and validation dataset with a 90/10% ratio. Model prediction was compared to three CE experts’ classification. The model’s performance was evaluated with the validation dataset by its sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and area under the precision-recall curve (AUCPR).

Results Esophageal images were identified with 97.4% sensitivity, 99.9% specificity and 99.9% accuracy. Gastric images were detected with 93.4% sensitivity, 99.9% specificity and 99.7% accuracy. The model identified small bowel images with 99.6% sensitivity, 97.3% specificity and 98.8% accuracy. Colonic lesions were detected with 97.3% sensitivity, 99.6% specificity and 99.0% accuracy. The AUCPR was 0.97 for stomach images and 1.00 for the remaining topographies.

Conclusions Our group developed the first AI model for panendoscopic segmentation of CE images, with images from multiple CE devices, solving an interoperability challenge. Automatic CE video segmentation is essential for implementing AI-assisted minimally invasive capsule panendoscopy.

Publication History

Article published online:
27 March 2025