A comparative study benchmarking colon polyp detection with CADe software

Aims Some studies have shown that Computer-Aided Detection software (CADe) for real-time colonic polyp detection improves polyp detection and adenoma detection rate (ADR), but no head-to-head comparison of different commercially available CADe systems has been performed. This study aims to compare the performance of three different polyp detection systems using a novel standardized methodology.

Methods Three hundred colonoscopies were video recorded and short video clips (25–40 seconds) representing normal segments or segments containing a polyp were randomly selected. These videos were then streamed through each CADe system from the manufacturers Medtronic, Olympus, and Augere Medical. Each system had various configurations or versions, resulting in a total of six distinct CADe software settings that were compared to each other with respect to sensitivity, specificity and the false positive rate (FPR). We also assess five endoscopists performance regarding both polyp detection and reaction time and compared it to the CADe systems.

Results All the CADe system had a high sensitivity for polyp detection ranging from 90%–100%, but a more than four-fold difference in the false positive rates (FPR) per frame ranging from 1.2% to 5.6%. Overall, the CADe systems achieving the highest sensitivity also had the highest FPR, or most false alerts potentially distracting the endoscopists. The differences between the systems were statistically significant (p-value<0.01). We also observed significant differences in alert delay between the different endoscopists and CADe systems.

Conclusions Our study shows that the CADe software performs better than the endoscopists, but there are significant differences between the CADe software. Although all of them have a very high sensitivity, some of them are hampered by an important FPR. A high FPR poses a risk of distracting the endoscopist. The most sensitive software also has the highest FPR.

 

MissingLinkText

Conflicts of interest

Kim Ånonsen, Håvard Espeland, Andreas Petlund, Pål Hålvorsen and Michael Riegler have a financial interest as shareholders in Augere Medical ASPia Smedsrup, Tor Jan Berstad and Thomas de Lange are employed by Augere Medical AS.

Publikationsverlauf

Artikel online veröffentlicht:
15. April 2024