Face validity of a virtual reality simulation platform to improve competency in endoscopy: a prospective observational cohort study

Abstract

Background and study aims Virtual reality endoscopic simulation training has the potential to expedite competency development in novice trainees. However, simulation platforms must be realistic and confer face validity. This study aimed to determine the face validity of high-fidelity virtual reality simulation (EndoSim, Surgical Science, Gothenburg), and establish benchmark metrics to guide the development of a Simulation Pathway to Improve Competency in Endoscopy (SPICE).

Methods A pilot cohort of four experts rated simulated exercises (Likert scale score 1–5) and following iterative development, 10 experts completed 13 simulator-based endoscopy exercises amounting to 859 total metric values.

Results Expert metric performance demonstrated equivalence (P = 0.992). In contrast, face validity of each exercise varied among experts (median 4 (interquartile range [IQR] 3–5), P < 0.003) with Mucosal Examination receiving the highest scores (median 5 [IQR 4.5–5], P = 1.000) and Loop Management and Intubation exercises receiving the lowest scores (median 3 [IQR 1–3], P < 0.001, P = 0.004), respectively. The provisional validated SPICE comprised 13 exercises with pass marks and allowance buffers defined by median and IQR expert performance.

Conclusions EndoSim Face Validity was very good related to early scope handling skills, but more advanced competencies and translation of acquired clinical skills require further research within an established training program. The existing training deficit with superadded adverse effects of the COVID pandemic make this initiative an urgent priority.

Publication History

Received: 20 November 2021

Accepted after revision: 08 June 2022

Article published online:
14 September 2022

• References

• 1 Siau K, Anderson J, Valori R. et al. Certification of UK gastrointestinal endoscopists and variations between trainee specialties: results from the JETS e-portfolio. Endosc Int Open 2019; 7: E551-E560
• 2 Clough J, Fitzpatrick M, Harvey P. et al. Shape of training review: An impact assessment for UK gastroenterology trainees. Frontline Gastroenterol 2019; 10: 356-363
• 3 Rutter MD, Brookes M, Lee TJ. et al. Impact of the COVID-19 pandemic on UK endoscopic activity and cancer detection: A National Endoscopy Database Analysis. Gut 2021; 70: 537-543
• 4 British Society of Gastroenterology. BSG multi-society guidance on further recovery of endoscopy services during the post-pandemic phase of COVID-19. https://www.bsg.org.uk/covid-19-advice/bsg-multi-society-guidance-on-further-recovery-of-endoscopy-services-during-the-post-pandemic-phase-of-covid-19/
• 5 Pawlak KM, Kral J, Khan R. et al. Impact of COVID-19 on endoscopy trainees: an international survey. Gastrointest Endosc 2020; 92: 925-935
• 6 Fitzpatrick M, Clough J, Harvey P. et al. How can gastroenterology training thrive in a post-COVID world?. Frontline Gastroenterol 2021; 12: 338-341
• 7 Brown C, Robinson D, Egan R. et al. Prospective cohort study of haptic virtual reality laparoscopic appendicectomy learning curve trajectory. J Laparoendosc Adv Surg Tech 2019; 29: 1128-1134
• 8 Larsen CR, Soerensen JL, Grantcharov TP. et al. Effect of virtual reality training on laparoscopic surgery: Randomised controlled trial. BMJ 2009; 338: b1802
• 9 Wang F, Zhang C, Guo F. et al. The application of virtual reality training for anastomosis during robot-assisted radical prostatectomy. Asian J Urol 2021; 8: 204-208
• 10 Waterman BR, Martin KD, Cameron KL. et al. Simulation training improves surgical proficiency and safety during diagnostic shoulder arthroscopy performed by residents. Orthopedics 2016; 39: 479-485
• 11 Ward ST, Hancox A, Mohammed MA. et al. The learning curve to achieve satisfactory completion rates in upper GI endoscopy: An analysis of a national training database. Gut 2017; 66: 1022-1033
• 12 Siau K, Crossley J, Dunckley P. et al. Direct observation of procedural skills (DOPS) assessment in diagnostic gastroscopy: nationwide evidence of validity and competency development during training. Surg Endosc 2020; 34: 105-114
• 13 Siau K, Iacucci M, Dunckley P. et al. The impact of COVID-19 on gastrointestinal endoscopy training in the United Kingdom. Gastroenterology 2020; 159: 1582-1585.e3
• 14 han R, Plahouras J, Johnston BC. et al. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev 2018; 8: CD008237
• 15 Edwards K, Siau K, Neville P. et al. PTH-129 Evaluation of simulation-based induction programme to enhance acquisition of handling skills for upper GI endoscopy. Gut 2018;
• 16 Dawe SR, Windsor JA, Broeders JAJL. et al. A systematic review of surgical skills transfer after simulation-based training: Laparoscopic cholecystectomy and endoscopy. Ann Surg 2014; 259: 236-248
• 17 Joint Advisory Group on GI Endoscopy. Formative DOPS: Diagnostic upper gastrointestinal endoscopy (OGD). https://www.thejag.org.uk/AboutUs/DownloadCentre.aspx
• 18 Jensen K, Bjerrum F, Hansen HJ. et al. Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS lobectomy. Surg Endosc 2017; 31: 2520-2528
• 19 Ahlberg G, Enochsson L, Gallagher AG. et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007; 193: 797-804
• 20 Larsen CR, Grantcharov T, Schouenborg L. et al. Objective assessment of surgical competence in gynaecological laparoscopy: Development and validation of a procedure-specific rating scale. BJOG 2008; 115: 908-916
• 21 Siau K, Hodson J, Anderson JT. et al. Impact of a national basic skills in colonoscopy course on trainee performance: An interrupted time series analysis. World J Gastroenterol 2020; 26: 3283-3292
• 22 Cooper JB, Taqueti VR. A brief history of the development of mannequin simulators for clinical education and training. Qual Saf Health Care 2004; 13: 11-18
• 23 Gordon MS, Ewy GA, Felner JM. et al. Teaching bedside cardiologic examination skills using “Harvey”, the cardiology patient simulator. Med Clin North Am 1980; 64: 303-315
• 24 Ewy GA, Felner JM, Juul D. et al. Test of a cardiology patient simulator with students in fourth-year electives. J Med Educ 1987; 62: 738-743
• 25 Grover SC, Garg A, Scaffidi MA. et al. Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: a randomized trial. Gastrointest Endosc 2015; 82: 1072-1079
• 26 Grover SC, Scaffidi MA, Khan R. et al. Progressive learning in endoscopy simulation training improves clinical performance: a blinded randomized trial. Gastrointest Endosc 2017; 86: 881-889
• 27 Walsh CM, Scaffidi MA, Khan R. et al. Non-technical skills curriculum incorporating simulation-based training improves performance in colonoscopy among novice endoscopists: Randomized controlled trial. Digest Endosc 2020; 32: 940-948
• 28 Sedlack RE. Validation of computer simulation training for esophagogastroduodenoscopy: Pilot study. J Gastroenterol Hepatol 2007; 22: 1214-1219
• 29 Ferlitsch A, Schoefl R, Puespoek A. et al. Effect of virtual endoscopy simulator training on performance of upper gastrointestinal endoscopy in patients: A randomized controlled trial. Endoscopy 2010; 42: 1049-1056
• 30 Hawkes N, Turner J, Hurley J. et al. UEG Week 2015 Poster Presentations. United Europ Gastroenterol J 2015;
• 31 Choy MC, Matharoo M, Thomas-Gibson S. Diagnostic ileocolonoscopy: Getting the basics right. Frontline Gastroenterol 2020; 11: 484-490
• 32 Dyke C, Franklin BR, Sweeney WB. et al. Early implementation of Fundamentals of Endoscopic Surgery training using a simulation-based mastery learning curriculum. Surgery 2021; 169: 1228-1233
• 33 Issenberg SB, McGaghie WC, Petrusa ER. et al. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach 2005; 27: 10-28