Introduction
Over the past decade, increasing emphasis has been placed on quality metrics and competency
assessment in health care. The goals of high-quality endoscopy – appropriate patient
selection (indicated procedure), accurate diagnosis, and appropriate implementation
– should be achieved with minimal patient risk and performed by properly trained and
competent endoscopists [1]. The ultimate goals should be improvement in patient outcome and satisfaction [2]
[3]. Competency, an important element of quality endoscopy, should be defined as the
consistent ability to meet technical goals of the intended procedure and to correctly
perform cognitive aspects of the procedure. In a constantly changing environment and
with the introduction of new techniques and technologies, competency is crucial to
endoscopic practice.
Endoscopic retrograde cholangiopancreatography (ERCP) is one of the most technically
demanding and high-risk procedures performed by gastrointestinal endoscopists, requiring
significant focused training and experience to maximize success and patient outcome.
It has a steep learning curve in both physical skills and judgment/interpretation,
and increasingly more sophisticated diagnostic and therapeutic techniques are employed.
Goals of an endoscopy training program include ensuring that core motor and cognitive
skills necessary to successfully and competently perform ERCP are acquired. Ensuring
competence in ERCP has recently emerged as an area of intense scrutiny as training
programs and hospital credentialing committees attempt to produce and approve, respectively,
adequately trained endoscopists. Despite this, universally accepted standards for
competence in ERCP have not been established. It is commonly assumed that competence
is achieved when a minimum number of ERCP procedures have been performed. For most
training programs this may only be in the range of 100 to 200 cases [4]. With the appreciation that individual trainees develop endoscopic skills at different
rates, [5] there has been a shift towards competency-based training and certification. An assessment
of individual performance is probably more robust than the use of minimum numbers
for defining competence. Accurate and validated ERCP performance measures are necessary
in order to effectively train practitioners using a competency-based curriculum. The
endoscopic trainee must be able to achieve a standard rate of technical success, but
must also be able to recognize abnormal pathology, identify it correctly, and decide
upon the appropriate course of action.
Recognizing the limitations of the current models of training and assessment of competence,
the Accreditation Council for Graduate Medical Education (ACGME) replaced the reporting
system with the Next Accreditation System focusing on competency-based medical education
(CBME). Ideally, ERCP training programs will move toward the adoption of CBME and
demonstrate that trainees have achieved competence and attained the technical and
cognitive skills required for safe and effective unsupervised practice in advanced
endoscopy [6]
[7]
[8].
Quality metrics
The demand for quality assessment in endoscopic procedures is increasing, but to date,
there is still no gold standard to assess ERCP quality. The American Society for Gastrointestinal
Endoscopy (ASGE) Committee on Outcomes Research has proposed several quality metrics
to establish competence and help define areas of continuous quality improvement [9]
[10]. Many of these quality metrics are not yet validated, and include assessment of
pre-, intra-, and post-procedural periods. Pre-procedural considerations include documentation
of appropriate indication(s), obtaining complete informed consent, appropriate assessment
of procedural difficulty and the appropriate use of prophylactic antibiotics and rectally
administered nonsteroidal agents for prevention of post-ERCP pancreatitis. Peri-procedural
factors, relative to endoscopist technical performance include selective cannulation
success rates of the duct of interest, and technical success rates of subsequent interventions,
such as extraction of bile duct stones and biliary stent placement, as well as documentation
of fluoroscopic time and radiation dose. ASGE guidelines suggest competency in ERCP
is demonstrated by the ability to consistently perform (without assistance) cannulation
and opacification of the desired duct, sphincterotomy, stone extraction and stent
placement [11]. Post-procedural factors to be monitored consist of procedure-related adverse events
(AEs), including those related to sedation (e. g. cardiopulmonary depression) and
local events (e. g. pancreatitis, bleeding and perforation).
Several publications have shown that AEs are markers of quality endoscopic, with identification
of predisposing factors and approaches to minimize them [12]. However, procedural success or patient-related outcomes are less often described
[13]. The ASGE recommends quality indicators as appropriate ERCP indication, cannulation
rate, stone extraction success rate, stent insertion success rate and frequency of
post-ERCP pancreatitis [9]. For each of these indicators, reaching the recommended performance target is strongly
associated with important clinical outcomes. It is seldom possible for a trainee to
be as good as an experienced practitioner at the completion of formal training. The
ASGE suggests trainees should demonstrate a minimum deep cannulation rate of the duct
of interest of 80 % to 85 % by the end of ERCP training. Expert endoscopists have
been shown to successfully cannulate the common bile duct (CBD) at a rate of > 95 %.
[9] The ASGE also recommends deep cannulation of the duct of interest in patients with
native papillae without surgically altered anatomy, clearance of CBD stones < 1 cm
and stent placement for biliary obstruction in patients with normal anatomy and with
obstruction below the bifurcation should be achieved in > 90 % of cases. [9] The British Society of Gastroenterology (BSG) recommends a cannulation rate of ≥ 90 %
for native papillae, CBD stone clearance for ≥ 80 % of those patients who are ERCP-naïve,
and for patients with an extrahepatic stricture, successful stent placement with tissue
sampling, as appropriate in ≥ 85 % of ERCP-naïve patients. Both success rates and
AEs should be routinely assessed [14]
[15]. The current rate of post-ERCP pancreatitis in clinical practice ranges from 1 %
to 7 %, varies by patient selection, operator skill and experience as well as the
type of ERCP performed, and, for that reason, it is difficult to set a single performance
target for all ERCPs for this indicator. Perforation and clinically significant hemorrhage
after sphincterotomy or sphincteroplasty rates should be ≤ 0.2 % and ≤ 1 %, respectively.
[9]
Assessment of competence
To ensure that future endoscopists practice high-quality endoscopy, the issue of quality
needs to be introduced and practiced from the onset of training. This may require
a change in the culture of training programs and their sponsor institutions. A constraint
to trainee’learning is the difficulty in allowing trainees enough independent time
(observation without instruction), especially in a busy practice with high case volumes
and time constraints that may be set by anesthesia. Even in a relaxed learning environment
that may allow for the additional time necessary for trainees to perform procedures,
different trainers may have different teaching methodologies, being more or less proactive.
To assess and document competence in ERCP, a number of assessment tools have been
developed in the past 3 years. They are intended to facilitate training programs to
meet the new ACGME reporting requirements and, more importantly, to help program directors
identify specific skill deficiencies early in training, thus allowing for the development
of tailored, individualized remediation. To meet these endpoints, it must be stressed
that assessment using these tools must be done in a continuous fashion to allow differentiation
of learning curve progression from premature plateauing of skills. Ideally, it should
also allow a balanced evaluation of both cognitive and motor skills. Self-assessment
of procedural performance by the trainee should also be considered in these tools,
once they are a successful device to provide insight into quality of ERCPs, and might
enhance quality by stimulating active reflection of one’s actions [16]. Furthermore, the comparison of subjective scores given by trainees and trainers
may provide additional insight on the value of self-reflection. The concept of comprehensive
competency must be highlighted, since once assessment becomes more robust and refined,
it has been suggested that procedural thresholds will quickly rise above traditional
estimates [17].
In 2013 Ekkelenkamp et al. [4] proposed a self-assessment evaluation tool (Rotterdam Assessment Form) comprising
the proposed quality indicators for ERCP. It consists of 3 parts, with the first part
covering objective parameters such as procedural indication, degree of technical difficulty,
presence of a native papilla or previous sphincterotomy and previous ERCP failure;
the second part encompasses success or failure options for different parts of the
procedure such as CBD cannulation, sphincterotomy, stone extraction and stent placement
(scored as successfully completed, partial success, or failure) and the third part
consisting of an improvement plan proposal after every 10 procedures (identification
of the deficiencies, proposed solution(s) and improvement strategies). The form also
invites the endoscopist to rate his/her own performance using a 10-point visual analogue
scale for each technical skill. The value of the subjective assessment is the creation
of self-awareness to enable reflection on performance, rather than providing evidence
for quality measurements. The time required for participating endoscopists to complete
the form was approximately 1 minute, with adherence to completion of the self-assessment
ranging from 82.9 % to 89.6 % [4]
[14]. The main reasons cited for lack of form completion were lack of time due to practice
demands.
Recently, the BSG ERCP working party proposed the Direct Observation of Procedural
Skills (DOPS) used by the Joint Advisory Group (JAG) for Gastrointestinal Endoscopy
in the United Kingdom [18]. It consists of pre-, intra-, and post-procedural components. The first part covered
the grade of difficulty, as well as the presence of a native papilla. A 4-point scale
was used to grade the individual endpoints (1 = trainer undertakes the majority of
the tasks/decisions and delivers constant verbal prompts; 2 = trainee undertakes tasks
requiring frequent supervisor input and verbal prompts; 3 = trainee undertakes tasks
requiring occasional supervisor input and verbal prompts; 4 = no supervision required).
The pre-procedural component comprises indication, risk, preparation, equipment check,
consent and sedation and monitoring. The intra-procedural component covers performance
of basic maneuvers (intubation, visualization and position relative to ampulla, and
patient comfort) as well as technical aspects (cannulation and imaging, selective
cannulation, wire management, fluoroscopy quality and interpretation, execution of
selected therapy, decisions on appropriate therapy, sphincterotomy, sphincteroplasty,
stone therapy, tissue sampling, stent placement). It also includes trainees’ actions
to minimize pancreatitis, and documents procedural-related AEs. The post-procedural
component covers report writing and management plan. Non-technical skills are also
evaluated (communication and teamwork, situation awareness, leadership, judgment and
decision). There is also a comment section for each of the components, as well as
learning objectives for future cases, to be added to the trainee’s personal development
plan once the DOPS is completed.
In 2016 Wani et al. [19] proposed an assessment form, which evaluates individual technical and cognitive
aspects of ERCP, distinguishing biliary from pancreatic procedures. A 4-point scoring
system was used to grade the individual endpoints (1 = achieves without instruction;
2 = achieves with minimal verbal cues; 3 = achieves with multiple verbal cues or hands-on
assistance; 4 = unable to complete and requires trainer to take over). The first part
covers the indication for ERCP and the grade of difficulty, as well as performance
of basic maneuvers (intubation, achieving the short position and identification of
the papilla) and the presence or absence of a previous sphincterotomy. The second
part evaluates technical aspects such as performance of deep cannulation of the desired
duct, sphincterotomy, stone clearance, and stent placement (when applicable). The
time allowed to attempt cannulation, whether cannulation was achieved by the trainer
when the trainee was unsuccessful, as well as the need for advanced cannulation techniques
(placement of a pancreatic duct stent, use of double-wire technique, or precut sphincterotomy)
are also considered. The third part grades cognitive aspects of ERCP that focus on
a clear understanding of the indication and appropriate use of fluoroscopy, proficient
use of real-time cholangiography/pancreatography interpretation, and the ability to
formulate a logical plan based on the procedural findings. The fourth part documents
procedural AEs (pain requiring hospitalization, pancreatitis and severity, immediate
or delayed post-sphincterotomy bleeding, perforation, mortality and cardiopulmonary
AE). In addition to grading specific endpoints, an overall assessment was provided
by trainers using a 10-point scoring system.
An important aspect of these type of forms, intended to be completed after every procedure,
is the balance between evaluation of the most important procedural quality parameters
and completion of the forms in an acceptably brief length of time. A major consideration
is the inverse correlation between complexity of the forms and their completion. Previous
studies, however, state that once familiar with the forms, staff typically require
less than 1 minute to complete the assessment tool form [20]. Periodic spot-checking at specific steps of training may be used instead of completion
of forms for all procedures. The ASGE recommends a minimum assessment with each of
these tools on a periodic basis so that approximately 10 % of the total procedures
performed by a trainee have completed evaluation forms. [19] Understandably, the more forms completed, the more precise the performance profile
of a specific trainee, and which allows training directors to quickly identify those
who are meeting or surpassing the expected milestones as well as those who are in
need of remediation [19]. Minimal competency thresholds or endpoints that need to be achieved must be defined.
The only competency threshold data currently available are based on the Mayo Colonoscopy
Skills Assessment Tool (MCSAT), which suggest that achieving average scores of 3.5
or higher for each specific core skill correlates with achievement of minimal competence
criteria [21]. The form by Ekkelenkamp et al. [4] focuses primarily on a limited number of motor skills, with minimal inclusion procedure-related
cognitive skill assessment. The forms created by Wani et al. [19] and the BSG DOPS [18] assess cognitive and motor skills in a balanced manner, however, only the latter
considers endoscopic non-technical skills. A strategy plan based on procedural findings
as well as when technical difficulty arises is also another important issue that should
be assessed by these forms. It can be argued that procedural outcome and AE may be
difficult to determine at the time of the procedure, as many AEs are delayed. In our
opinion, such kind of form should be completed independently by both the trainee and
the trainer, with the purpose of comparing different subjective scores. It is anticipated
that this will allow insight for the trainee and promote self-reflection, and increase
comprehensive competency of the trainer. Deconstructing ERCP and its techniques and
maneuvers in may allow address various deficiencies in ERCP training and teaching
to be addressed in a more granular fashion.
Conclusion
Pathways to achieve competence and measurement of competence continue to evolve with
the goal of increasing quality delivery of health care by ensuring technical success
and minimizing AEs of ERCP. Efforts should be made to improve quality above the proposed
minimum thresholds. Standardization of the performance, definition of competence in
ERCP, and demonstrating competence at the end of training are critical to improve
patient outcome.
Given that training in medicine has undergone major transformations, emphasis needs
to shift from absolute numbers of procedures performed toward well-defined and validated
competency thresholds. The quality of health care can be measured by comparing the
performance of an individual or a group of individuals with an ideal or benchmark
[22]. Tools to assess ERCP competence acquisition will facilitate training programs to
document progress of individual trainees. Real-time feedback can be provided on a
trainee’s performance and specific skill deficiencies can be detected earlier in training,
allowing restructuring and provision of additional training. It will also help to
increase comprehensive competency in the trainer, “teaching” him/her to teach the
trainer. Assessment of traineesʼ individual performance by means of learning curves
is much more robust and relevant for the individual trainee than simple threshold
numbers. We believe these are the first steps to be taken in the miles to go before
standardization.
