Using computerized assessment in simulated colonoscopy: a validation study

Abstract

Background and study aims Patient safety during a colonoscopy highly depends on endoscopist competence. Endoscopic societies have been calling for an objective and regular assessment of the endoscopists, but existing assessment tools are time-consuming and prone to bias. We aimed to develop and gather evidence of validity for a computerized assessment tool delivering automatic and unbiased assessment of colonoscopy based on 3 dimensional coordinates from the colonoscope.

Methods Twenty-four participants were recruited and divided into two groups based on experience: 12 experienced and 12 novices. Participants performed twice on a physical phantom model with a standardized alpha loop in the sigmoid colon. Data was gathered directly from the Olympus ScopeGuide system providing XYZ-coordinates along the length of the colonoscope. Five different motor skill measures were developed based on the data, named: Travel Length, Tip Progression, Chase Efficiency, Shaft movement without tip progression, and Looping.

Results The experinced had a lower travel length (P < 0.001), tip progression (P < 0.001), chase efficiency (P = 0.001) and looping (P = 0.006), and a higher shaft movement without tip progression (P < 0.001) reaching the cecum compared with the novices. A composite score was developed based on the five measurements to create a combined score of progression, the 3D-Colonoscopy-Progression-Score (3D-CoPS). The 3D-CoPS revealed a significant difference between groups (experienced: 0.495 (SD 0.303) and novices –0.454 (SD 0.707), P < 0.001).

Conclusion This study presents a novel, real-time computerized assessment tool for colonoscopy, and strong evidence of validity was gathered in a simulation-based setting. The system shows promising opportunities for automatic, unbiased and continuous assessment of colonoscopy performance.

Publication History

Received: 29 April 2019

Accepted: 27 January 2020

Article published online:
25 May 2020

• References

• 1 Benson VS, Patnick J, Davies AK. et al. Colorectal cancer screening: A comparison of 35 initiatives in 17 countries. Int J Cancer 2008; 122: 1357-1367
• 2 Rembacken B, Hassan C, Riemann JF. et al. Quality in screening colonoscopy: position statement of the European Society of Gastrointestinal Endoscopy (ESGE). Endoscopy 2012; 44: 957-968
• 3 Kaminski MF, Thomas-Gibson S, Bugajski M. et al. Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy ( ESGE ) Quality Improvement Initiative. Endoscopy 2017; 49: 1175-1189
• 4 Day LW, Cohen J, Greenwald D. et al. Quality indicators for gastrointestinal endoscopy units. VideoGIE 2017; 2: 119-140
• 5 Valori R, Senore C, Thomas de Lange I. et al. Performance measures for endoscopy services: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative ESGE Quality Improvement Committee (QIC) Endoscopy Service Working Group. Endoscopy 2018; 50: 1186-1204
• 6 Rex DK, Boland CR, Dominitz JA. et al. Colorectal Cancer Screening: Recommendations for Physicians and Patients from the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2017; 112: 1016-1030
• 7 Chak A, Cooper GS, Blades EW. et al. Prospective assessment of colonoscopic intubation skills in trainees. Gastrointest Endosc 1996; 44: 54-57
• 8 Sedlack RE. Training to competency in colonoscopy: Assessing and defining competency standards. Gastrointest Endosc 2011; 74: 355-366.e2
• 9 Baruk JH, Cohen ER, Feinglass J. et al. Residents’ procedural experience does not ensure competence: a research synthesis. J Grad Med Educ 2017; 9: 201-208
• 10 Sedlack RE, Coyle WJ, Obstein KL. et al. ASGE’s assessment of competency in endoscopy evaluation tools for colonoscopy and EGD. Gastrointest Endosc 2014; 79: 1-7
• 11 Walsh CM, Ling SC, Khanna N. et al. Gastrointestinal Endoscopy Competency Assessment Tool: reliability and validity evidence. Gastrointest Endosc 2015; 81: 1417-1424.e2
• 12 Vassiliou MC, Kaneva PA, Poulose BK. et al. Global assessment of gastrointestinal endoscopic skills (GAGES): A valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 2010; 24: 1834-1841
• 13 Siau K, Dunckley P, Anderson J. et al. Changes in scoring of direct observation of procedural skills (DOPS) forms in endoscopy training and their impact on competence assessment. Trainees’ Sect Symp #FOAMed Film Fest Educ Dragons Den 2017; A11.1-A11
• 14 Sedlack RE, Coyle WJ. Assessment of competency in endoscopy: Establishing and validating generalizable competency benchmarks for colonoscopy. Gastrointest Endosc 2016; 83: 516-523.e1
• 15 Coe SG, Panjala C, Heckman MG. et al. Quality in colonoscopy reporting: An assessment of compliance and performance improvement. Dig Liver Dis 2012; 44: 660-664
• 16 Konge L, Vilmann P, Clementsen P. et al. Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. Endoscopy 2012; 44: 928-933
• 17 Nerup N, Preisler L, Svendsen MBS. et al. Assessment of colonoscopy by use of magnetic endoscopic imaging: Design and validation of an automated tool. Gastrointest Endosc 2015; 81: 548-554
• 18 Mark-Christensen A, Brandsborg S, Iversen LH. Magnetic endoscopic imaging as an adjuvant to elective colonoscopy: a systematic review and meta-analysis of randomized controlled trials. Endoscopy 2015; 47: 251-261
• 19 Rogen P, Fain B. Automatic classification of protein structure by using Gauss integrals. Proc Natl Acad Sci U S A 2003; 100: 119-124
• 20 Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: Theory and application. Am J Med 2006; 119: 166.e7-16
• 21 Plooy AM, Hill A, Horswill MS. et al. Construct validation of a physical model colonoscopy simulator. Gastrointest Endosc 2012; 76: 144-150
• 22 Downing S, Yudkowsky R. Assessment in Health Professions Education. New York: Routledge; 2009
• 23 Shah SG, Thomas-Gibson S, Brooker JC. et al. Use of video and magnetic endoscope imaging for rating competence at colonoscopy: Validation of a measurement tool. Gastrointest Endosc 2002; 56: 568-573
• 24 Teshima CW, Zepeda-Gómez S, AlShankiti SH. et al. Magnetic imaging-assisted colonoscopy vs conventional colonoscopy: A randomized controlled trial. World J Gastroenterol 2014; 20: 13178-13184
• 25 Duncan JE, McNally MP, Sweeney WB. et al. CT colonography predictably overestimates colonic length and distance to polyps compared with optical colonoscopy. Am J Roentgenol 2009; 193: 1291-1295
• 26 Barton JR, Corbett S, Van Der Vleuten CP. The validity and reliability of a Direct Observation of Procedural Skills assessment tool: Assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc 2012; 75: 591-597
• 27 Sedlack RE. The Mayo Colonoscopy Skills Assessment Tool: Validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 2010; 72: 1125-1133.e3
• 28 Shah SG, Brooker JC, Thapar C. et al. Patient pain during colonoscopy: An analysis using real-time magnetic endoscope imaging. Endoscopy 2002; 34: 435-440
• 29 Haycock A, Cohen J, Saunders BP. et al. Cotton and Williamsʼ Practical Gastrointestinal Endoscopy: The Fundamentals. Oxford: Wiley Blackwell; 2014
• 30 Ritter EM, Cox TC, Trinca KD. et al. Simulated Colonoscopy Objective Performance Evaluation (SCOPE): A non-computer-based tool for assessment of endoscopic skills. Surg Endosc Other Interv Tech 2013; 27: 4073-4080
• 31 Cadoni S, Gallittu P, Sanna S. et al. A two-center randomized controlled trial of water-aided colonoscopy versus air insufflation colonoscopy. Endoscopy 2014; 46: 212-218
• 32 Asai S, Fujimoto N, Tanoue K. et al. Water immersion colonoscopy facilitates straight passage of the colonoscope through the sigmoid colon without loop formation: Randomized controlled trial. Dig Endosc 2015; 27: 345-353
• 33 Haycock A, Koch AD, Familiari P. et al. Training and transfer of colonoscopy skills: a multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest Endosc 2010; 71: 298-307
• 34 Sedlack RE, Shami VM, Adler DG. et al. Colonoscopy core curriculum. Gastrointest Endosc 2012; 76: 482-490
• 35 Kaminski MF, Thomas-Gibson S, Bugajski M. et al. Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy ( ESGE ) Quality Improvement Initiative. Endoscopy 2015; 49: 1175-1189
• 36 Preisler L, Bulut M, Svendsen MS. et al. An automatic measure of progression during colonoscopy correlates to patient experienced pain. Scand J Gastroenterol 2018; 53: 345-349
• 37 Hatala R, Cook DA, Zendejas B. et al. Feedback for simulation-based procedural skills training: A meta-analysis and critical narrative synthesis. Adv Heal Sci Educ 2014; 19: 251-272
• 38 Chung JIl, Kim N, Um MS. et al. Learning curves for colonoscopy: A prospective evaluation of gastroenterology fellows at a single center. Gut Liver 2010; 4: 31-35
• 39 Hitchins CR, Metzner M, Edworthy J. et al. Non-technical skills and gastrointestinal endoscopy: a review of the literature. Frontline Gastroenterol 2018; 9: 129-134
• 40 Brydges R, Hatala R, Zendejas B. et al. Linking simulation-based educational assessments and patient-related outcomes. Acad Med 2015; 90: 246-256
• 41 Stefanidis D, Sevdalis N, Paige J. et al. Simulation in surgery: what’s needed next?. Ann Surg 2015; 261: 846-853
• 42 Zupanc CM, Burgess-Limerick R, Hill A. et al. A competency framework for colonoscopy training derived from cognitive task analysis techniques and expert review. BMC Med Educ 2015; 15: 1-11