How Consistent is Trainer Assessment of Trainee Performance during Colonoscopy?

Aims Conscious competence is a fundamental concept in colonoscopy training¹. A trainer should be able to recognise and anticipate the challenges their trainee is facing during a colonoscopy procedure, and offer useful suggestions to overcome or prevent them. This is largely based on careful observation and interpretation of the visual information available during a training episode. Furthermore, direct observation of procedural skill is a commonly used method of colonoscopy competency assessment. However, there may be differences in what trainers attend to while watching colonoscopy, and how they interpret what they observe. We aimed to explore qualitative differences in the thought processes of several different trainers while they viewed a video recording of the same colonoscopy procedure.

Methods Eleven trainers, all of whom are faculty on basic skills colonoscopy training courses, were shown an 8-minute video recording (without audio) of a colonoscopy insertion. A room view of the endoscopist was included, picture-in-picture, but his identity was hidden. Participants were told the endoscopist was a trainee, whereas, in fact, he was an experienced consultant gastroenterologist with 20 years’ experience who meets all relevant key performance indicators. Prior to watching the video, participants received instruction in the Think Aloud (TA) method, and undertook a brief training exercise. TA is a technique used widely in sports science to gain insight into the cognition and behaviours of athletes and coaches². Participants were asked to verbalise anything that came to mind whilst watching the video and say what feedback they might give to the ‘trainee’. TA audio was recorded and transcribed, and the content analysed according to a predetermined thematic classification [1] [2].

Results Considerable variation was observed in how positively trainers perceived the performance of the ‘trainee’. Eight participants expressed thoughts suggesting they judged the endoscopist to be either highly skilled (3) or at least competent (5), whereas 3 were predominantly critical of his technique. Only one participant expressed doubt that the colonoscopist was a genuine trainee. One trainer verbalised that they would have probably taken over the scope during passage of the sigmoid, and one suggested that the technique displayed was not safe. Trainers showed congruence of thought on themes related to scope handling and ergonomics, but a wide range of often contradictory thoughts were expressed about aspects of performance relating to tip control, luminal visualisation, use of water insufflation, and proactive position change. There was fairly good agreement in selecting the main areas for discussion in feedback.

Conclusions The use of TA has highlighted surprising differences in how trainers perceive aspects of trainee performance. Reflecting on their own TA transcripts, and comparing them with others, might provide a useful exercise for trainer development and benchmarking.

Publication History

Article published online:
27 March 2025

• References

• 1 Waschke KA, Anderson J, Macintosh D, Valori RM.. Training the gastrointestinal endoscopy trainer. Best Practice ' Research Clinical Gastroenterology 2016; 30 (03): 409-19
• 2 Whitehead AE, Cropley B, Huntley T, Miles A, Quayle L, Knowles Z.. ‘Think Aloud’: Toward a framework to facilitate reflective practice amongst rugby league coaches. International Sport Coaching Journal. 2016; 3 (03): 269-286