Abstract
Background The endoscopic retrograde cholangiopancreatography (ERCP) direct observation of procedural
skills (DOPS) is a 27-item competency assessment tool that was developed to support
UK ERCP training. We evaluated validity of ERCP DOPS and competency development during
training.
Methods This prospective study analyzed ERCP DOPS performed in the UK between July 2016 and
October 2018. Reliability was measured using Cronbach’s alpha, and DOPS scores were
benchmarked using the contrasting groups method. The percentage of competent scores
was averaged for each item, domain, and overall rating, and stratified by lifetime
procedure count to evaluate learning curves. Multivariable analyses were performed
to identify predictors of DOPS competence.
Results 818 DOPS (109 trainees, 80 UK centers) were analyzed. Overall Cronbach’s alpha was
0.961. Attaining competency in 87 % of assessed DOPS items provided the optimal competency
benchmark. This was achieved in the domain sequence of: pre-procedure, post-procedure
management, endoscopic non-technical skills, cannulation & imaging, and execution
of selected therapy, and across all items after 200 – 249 procedures (89 %). After
300 procedures, the benchmark was reached for selective cannulation (89 %), but not
for stenting (plastic 73 %; metal 70 %), sphincterotomy (80 %), and sphincteroplasty
(56 %). On multivariable analysis, lifetime procedure count (P = 0.002), easier case difficulty (P < 0.001), trainee grade (P = 0.03), and higher lifetime DOPS count (P = 0.01) were predictors of DOPS competence.
Conclusion This study provides novel validity, reliability, and learning curve data for ERCP
DOPS. Trainees should have a minimum of 300 hands-on ERCP procedures before undertaking
summative assessment for independent practice.
