Endoscopy 2019; 51(04): S115-S116
DOI: 10.1055/s-0039-1681509
ESGE Days 2019 oral presentations
Saturday, April 6, 2019 11:00 – 13:00: Best abstract awards Congress Hall
Georg Thieme Verlag KG Stuttgart · New York

INCORPORATION OF TEMPORAL INFORMATION IN A DEEP NEURAL NETWORK IMPROVES PERFORMANCE LEVEL FOR AUTOMATED POLYP DETECTION AND DELINEATION

T Eelbode
1   Medical Imaging Research Center, PSI, KU Leuven, Leuven, Belgium
,
I Demedts
2   Department of Gastroenterology and Hepatology, KU Leuven, Leuven, Belgium
,
P Roelandt
2   Department of Gastroenterology and Hepatology, KU Leuven, Leuven, Belgium
,
C Hassan
3   Gastroenterology, Nuovo Regina Margherita Hospital, Rome, Italy
,
E Coron
4   Hepatogastroenterology, Centre Hospitalier Universitaire Hotel Dieu, Nantes, France
,
P Bhandari
5   Solent Centre for Digestive Diseases, Portsmouth University Hospital, Portsmouth, United Kingdom
,
H Neumann
6   First Medical Department, University Medical Center Mainz, Mainz, Germany
,
O Pech
7   Department of Gastroenterology and Interventional Endoscopy, Krankenhaus Barmherzige Brüder Regensburg, Regensburg, Germany
,
A Repici
8   Digestive Endoscopy Unit, Humanitas University, Milan, Italy
,
F Maes
1   Medical Imaging Research Center, PSI, KU Leuven, Leuven, Belgium
,
R Bisschops
2   Department of Gastroenterology and Hepatology, KU Leuven, Leuven, Belgium
› Author Affiliations
Further Information

Publication History

Publication Date:
18 March 2019 (online)

Also available at
 

Aims:

As opposed to current automated polyp detection techniques, endoscopists will use information from previous video-frames to indicate the presence of a polyp. We aim to exploit this type of temporal information by introducing memory cells into an artificial intelligence (AI) system.

Methods:

Colonoscopy videos from 104 patients are included with 258 polyps. Shorter video-clips of each polyp are extracted and only a few frames were annotated by experts. These manual annotations are automatically propagated over the entire clip. The resulting, much larger annotated dataset is then used to train a convolutional neural network (CNN). This network is extended with a recurrent module, resulting in an AI system that uses knowledge from previous timesteps.

Frame-level sensitivity and specificity describe detection power. For delineation accuracy, the soft Dice score quantifies the amount of overlap between a delineation map and its ground truth considering the confidence of the network (a number between 0 and 1 where the latter means perfect overlap with 100% confidence).

Results:

Two different networks are trained for evaluation. A first CNN is trained solely on the expert annotated frames and a second CNN includes the temporal module and is trained on all the auto-generated annotations (called EXP and REC respectively). The results are shown in table 1. The incorporation of temporal information improves the network for each metric and especially increases specificity since it makes the network less sensitive to confusing frames. Pairwise t-tests show that all differences are significant with p < 0,00001 (significance level of 0,05).

Tab. 1:

Sensitivity, specificity and soft Dice score for both networks evaluated on an independent test set. N = number of images used for training.

N

Sensitivity

Specificity

Soft Dice score

CNN1 – EXP

758

0,83

0,54

0,38

CNN2 – REC

40887

0,91

0,74

0,56

Conclusions:

The inclusion of temporal information provides more accurate and confident results for polyp detection and delineation on endoscopic videos.