CC BY-NC-ND 4.0 · Endosc Int Open 2021; 09(07): E1136-E1144
DOI: 10.1055/a-1468-3964
Original article

Multi-expert annotation of Crohn’s disease images of the small bowel for automatic detection using a convolutional recurrent attention neural network

Astrid de Maissin
1  CHD La Roche Sur Yon, department of gastroenterology, La Roche Sur Yon, France
,
Remi Vallée
2  Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Mathurin Flamant
3  Clinique Jules Verne, department of gastroenterology, Nantes, France
,
Marie Fondain-Bossiere
4  CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Catherine Le Berre
4  CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Antoine Coutrot
2  Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Nicolas Normand
2  Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Harold Mouchère
2  Nantes University, CNRS, LS2N UMR 6004, Nantes, France
,
Sandrine Coudol
5  CHU de Nantes, INSERM CIC 1413, Pôle Hospitalo-Universitaire 11: Santé Publique, Clinique des données, Nantes, France
,
Caroline Trang
4  CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
,
Arnaud Bourreille
4  CHU Nantes, Institut des Maladies de l’Appareil Digestif, CIC Inserm 1413, Nantes University, Nantes, France
› Author Affiliations

Abstract

Background and study aims Computer-aided diagnostic tools using deep neural networks are efficient for detection of lesions in endoscopy but require a huge number of images. The impact of the quality of annotation has not been tested yet. Here we describe a multi-expert annotated dataset of images extracted from capsules from Crohn’s disease patients and the impact of the quality of annotations on the accuracy of a recurrent attention neural network.

Methods Images of capsule were annotated by a reader first and then reviewed by three experts in inflammatory bowel disease. Concordance analysis between experts was evaluated by Fleiss’ kappa and all the discordant images were, again, read by all the endoscopists to obtain a consensus annotation. A recurrent attention neural network developed for the study was tested before and after the consensus annotation. Available neural networks (ResNet and VGGNet) were also tested under the same conditions.

Results The final dataset included 3498 images with 2124 non-pathological (60.7 %), 1360 pathological (38.9 %), and 14 (0.4 %) inconclusive. Agreement of the experts was good for distinguishing pathological and non-pathological images with a kappa of 0.79 (P < 0.0001). The accuracy of our classifier and the available neural networks increased after the consensus annotation with a precision of 93.7 %, sensitivity of 93 %, and specificity of 95 %.

Conclusions The accuracy of the neural network increased with improved annotations, suggesting that the number of images needed for the development of these systems could be diminished using a well-designed dataset.

Supplementary material



Publication History

Received: 25 November 2020

Accepted: 04 March 2021

Publication Date:
21 June 2021 (online)

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany