Introduction
Colon cancer is the second leading cause of cancer related deaths [1]. The sequence between adenoma to adenocarcinoma takes on average 10 years and can
be prevented by polypectomy [2]. Numerous observational studies have reported reduced colorectal cancer mortality
with the performance of colonoscopies and randomized clinical trials are underway
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]. Currently, colonoscopy is one of the major forms of colon cancer screening along
with fecal testing [12].
Quality assurance is an integral part of an effective screening colonoscopy program.
Adenoma detection rate (ADR) has emerged as one of the most important quality indicators.
ADRs less than 20 % have been associated with development of interval colon cancer
after a screening colonoscopy [13]
[14]. As a result, recent guidelines recommend an ADR ≥ 25 % [15]. However, there is considerable variation in the ADR between endoscopists [16]
[17]. Numerous interventions to improve the ADR, such as increased withdrawal time, enhanced
inspection techniques, and/or educational interventions have been largely ineffective
[18]. In this study, we sought to determine the effect of performance report cards on
the ADR over a 3-year period.
Patients and methods
Study design
The study was conducted at St. Joseph’s Health Care, an academic hospital affiliated
with Western University (London, Ontario, Canada), between April 1, 2012 and Mar 31,
2015 as a part of a colonoscopy quality assurance initiative through the ColonCancerCheck™
(CCC) program of the Southwestern Ontario Regional Cancer Program and Cancer Care
Ontario. The CCC was the first programmatic provincial colon cancer screening program
in Canada, whereby average risk individuals aged 50 to 75 are encouraged to have biennial
fecal occult blood testing (FOBT). All tests are processed at CCC approved labs using
Hemoccult II Sensa (Beckman Coulter, Mississauga, Canada). Those with positive tests
are then referred for colonoscopy. Patients with a family history of colon cancer
in a first-degree relative are considered “high risk” and referred directly for colonoscopy.
Thus, patients included in this study were those with either positive FOBT or a family
history of colon cancer in a first-degree relative having their first colonoscopy.
Colonoscopies completed on symptomatic patients, average risk screening, or surveillance
for colorectal neoplasm/polyps were excluded. All endoscopists involved in the program
were board-certified gastroenterologists or general surgeons who performed at least
200 colonoscopies each year for any indication. Residents and fellows were not involved
in any of the procedures. The protocol was approved by the university’s Research Ethics
Board.
Beginning on April 1, 2012, physicians were required to complete a colonoscopy reporting
tool for all colonoscopies documenting the procedural indication, bowel preparation
quality, cecal intubation, gross findings including detection of polyp(s), interventions
completed, and immediate adverse events. Bowel preparation quality was rated using
an ordinal scale as follows: good (mucosa seen throughout), fair (liquid contents,
exam adequate), and poor (solid contents, exam compromised). The decision to use a
simple ordinal scale was made by Cancer Care Ontario due to the lack of a standardized
bowel preparation score used throughout the province. The tool was completed immediately
after each procedure and participation was mandatory. In patients who were identified
as having at least 1 polyp removed, histological information was manually reviewed
to determine the polyp type. Because the quality assurance program was centered on
programmatic colonoscopy screening, and due to the very high number of annual colonoscopies
completed at the center, histology was not reviewed for other indications.
The study was divided into 3 phases. Between April 1, 2012 and March 31, 2013 (Baseline),
physicians were not aware that the data from the colonoscopy reporting tool would
be used to generate report cards. Personalized report cards were issued in April 2013
documenting a physician’s cecal intubation rate, preparation quality, ADR, polyp detection
rate (PDR), advanced ADR, adenocarcinoma detection rate, and perforation rate compared
to the institutional mean. A second report card was issued in April 2014. The period
between April 1, 2013 and March 31, 2014 defined Year 1 and the period between April
1, 2014 and March 31, 2015 defined Year 2. Report cards were confidential and not
shared with anyone other than the individual physician. There were no interventions
other than the report cards. In addition, physicians were not aware of the study during
the data collection period.
Outcome definitions
ADR was chosen as the primary outcome and defined as the proportion of patients undergoing
colonoscopy with 1 or more adenomas confirmed histologically. Secondary outcomes included
PDR, advanced ADR, and adenocarcinoma detection rate. PDR was defined as the proportion
of patients undergoing colonoscopy with 1 or more polyps of any histology. An advanced
adenoma was defined as having any of the following: ≥ 1 cm in diameter, villous histology,
or high-grade dysplasia.
Statistical analysis
Individual patient level data were collected prospectively for each procedure. Crude
analyses to compare the ADR, PDR, advanced ADR, and adenocarcinoma detection rates
between baseline and Year 1 and baseline and Year 2 were performed using analysis
of variance and pairwise comparisons. A Bonferonni correction was deemed unnecessary
because the comparisons were only between 3 groups [19]. Multivariate logistic regression using generalized estimating equations was performed
to adjust for patient (indication, age, gender) and procedural (physician specialty,
bowel preparation quality, and cecal intubation) factors, which were defined a priori based on subject knowledge rather than empiric methods [15]
[16]
[20]. Given that ADR may differ for colonoscopies performed for positive FOBT and family
history, we tested for effect modification by procedure indication. Furthermore, we
examined for effect modification by specialty (gastroenterology versus general surgery),
and baseline ADR (< 25 % versus ≥ 25) using interaction terms. The cut off for baseline
ADR was chosen to be 25 % based on current targets recommended by recent guidelines
[15]. A 2-sided P value < 0.05 was considered significant. All statistical analyses were performed
with Stata 14.0.
Results
Between April 1, 2012 and March 31, 2015, 17 physicians (11 gastroenterologists, 6
general surgeons) performed 3,118 colonoscopies on patients with a family history
of colorectal cancer (76.6 %) or a positive FOBT (23.4 %) ([Table 1]). The mean (SD) patient age was 58.6 (10.5) and 59.8 % were female. Bowel preparation
quality was rated as good, fair, and poor in 89.3 %, 8.6 %, and 2.1 %, respectively.
Cecal intubation was successful in 96.6 % of patients.
Table 1
Characteristics of 3,118 patients undergoing colonoscopies during the study period.
Age-mean (SD)
|
58.6 (10.5)
|
Female-no. (%)
|
1864 (59.8 %)
|
Indication-no.(%)
|
|
2388 (76.6 %)
|
|
730 (23.4 %)
|
Bowel preparation quality
|
|
2,784 (89.3 %)
|
|
269 (8.6 %)
|
|
65 (2.1 %)
|
Cecal intubation
|
|
3,012 (96.6 %)
|
|
94 (3.0 %)
|
|
12 (0.4 %)
|
1 Includes cases such as obstructing tumor and altered surgical anatomy
On crude analysis, ADR improved from a baseline of 34.5 % to 39.4 % in Year 1 (P = 0.0152) and to 41.2 % in Year 2 (P = 0.0016) ([Fig. 1] and [Table 2]). The increase in ADR between Year 1 and Year 2 was not statistically significant
(P = 0.33). The PDR increased from a baseline of 45 % to 48.8 % in Year 1 (0.0684) and
to 51.8 % in Year 2 (P = 0.0032). The advanced ADR and adenocarcinoma rates improved each year but did not
reach statistical significance.
Fig. 1 Detection rates by year.
Table 2
Crude analysis of adenoma, polyp, advanced adenoma, and adenocarcinoma detection rate
by year.
|
Detection rate-no. (%)
|
P value
|
|
Baseline (n = 1,133)
|
Year 1 (n = 1,172)
|
Year 2 (n = 813)
|
Global
|
Year 1[1]
|
Year 2[1]
|
Adenoma detection rate
|
391 (34.5 %)
|
462 (39.4 %)
|
338 (41.2 %)
|
0.0037
|
0.0152
|
0.0016
|
Polyp detection rate
|
510 (45.0 %)
|
572 (48.8 %)
|
421 (51.8 %)
|
0.0113
|
0.0684
|
0.0032
|
Advanced adenoma detection rate[2]
|
78 (6.9 %)
|
95 (8.1 %)
|
76 (9.4 %)
|
0.14
|
0.28
|
0.05
|
Adenocarcinoma detection rate
|
8 (0.7 %)
|
10 (0.9 %)
|
10 (1.2 %)
|
0.4721
|
0.7082
|
0.2271
|
1 Compared to baseline
2 Defined as > 1 cm in size, presence of villous component, or high grade dysplasia
On multivariate analysis controlling for patient age and gender, physician specialty,
indication, bowel preparation quality, and cecal intubation, ADR increased by 22 %
in Year 1 (OR 1.22, 95 % CI 1.02 – 1.45, p = 0.03) and 30 % in Year 2 (OR 1.30, 95 %
CI 1.07 – 1.58, P = 0.008) compared to baseline ([Table 3]) (see [Supplement] for model details). The effect of the report card intervention on the ADR did not
differ based on the indication for colonoscopy (interaction term P value > 0.05). However, there was evidence of effect modification by baseline ADR
when tested with an interaction term, indicating the effect of report cards differed
based on baseline ADR (interaction term P value = 0.03). Among physicians with a baseline ADR < 25 %, ADR increased 2.21 times
after Year 1 (P = 0.003) and 2.17 times (P = 0.004) after Year 2. Among physicians with a baseline ADR ≥ 25 %, ADR increased
by 18 % (P = 0.10) after Year 1 and 33 % (P = 0.01) after Year 2 ([Table 4]) (see [Supplement] for model details).
Table 3
Multivariate analysis[1] of adenoma, polyp, advanced adenoma, and adenocarcinoma detection rates compared
to baseline.
|
Year 1 OR (95 % CI)
|
P value
|
Year 2 OR (95 % CI)
|
P value
|
Adenoma
|
1.22 (1.02 – 1.45)
|
0.03
|
1.30 (1.07 – 1.58)
|
0.008
|
Polyp
|
1.14 (0.96 – 1.35)
|
0.13
|
1.28 (1.06 – 1.54)
|
0.01
|
Advanced adenoma[2]
|
1.12 (0.81 – 1.54)
|
0.50
|
1.27 (0.90 – 1.78)
|
0.18
|
Adenocarcinoma
|
1.30 (0.48 – 3.53)
|
0.61
|
2.04 (0.75 – 5.58)
|
0.16
|
1 Adjusted for endoscopist specialty, patient age, gender, procedural indication, bowel
preparation quality, and cecal intubation
2 Includes adenoma with any of the following features: > 1 cm in size, villous component,
or high-grade dysplasia
Table 4
Improvement in adenoma detection rate by baseline adenoma detection rate[1].
|
Crude analysis Adenoma detection rate (%)
|
Multivariate analysis[1]
OR (95 % CI) for adenoma detection compared to baseline
|
Baseline ADR
|
Baseline
|
Year 1
|
Year 2
|
Global p-value
|
P value[2]
(Year 1)
|
P value[2]
(Year 2)
|
Year 1
|
Year 2
|
P value (Year 1)
|
P value (Year 2)
|
< 25 %
|
28/176 (15.9 %)
|
67/254 (26.4 %)
|
60/210 (28.6 %)
|
0.009
|
0.01
|
0.004
|
2.21 (1.32 – 3.71)
|
2.17 (1.28 – 3.68)
|
0.003
|
0.004
|
≥ 25 %
|
363/957 (37.9 %)
|
395/918 (43.0 %)
|
278/603 (46.1 %)
|
0.004
|
0.03
|
0.001
|
1.18 (0.97 – 1.43)
|
1.33 (1.08 – 1.66)
|
0.10
|
0.01
|
1 Adjusted for endoscopist specialty, patient age, gender, procedural indication, bowel
preparation quality, and cecal intubation
2 Compared to baseline
The overall ADR was 40.6 % for gastroenterologists and 32.0 % for general surgeons
(P < 0.0001) ([Table 5]). Gastroenterologists were 52 % more likely to detect an adenoma than a general
surgeon on multivariate analysis (OR 1.52, 95 % CI 1.28 – 1.81) (see [Supplement] for model details). However, the change in ADR in Year 1 and Year 2 did not differ
significantly by specialty (interaction term P value > 0.05).
Table 5
Crude adenoma detection rate by specialty.
|
Adenoma detection rate (%)
|
|
Gastroenterologist
|
General Surgeon
|
P value
|
Baseline
|
308/847 (36.4 %)
|
83/286 (29.0 %)
|
0.02
|
Year 1
|
350/842 (41.6 %)
|
112/330 (33.9 %)
|
0.02
|
Year 2
|
259/527 (45.3 %)
|
79/241 (32.8 %)
|
0.0009
|
Total
|
917/2,216 (40.6 %)
|
274/857 (32.0 %)
|
< 0.0001
|
Discussion
Reduction in colorectal cancer mortality from screening colonoscopy relies on identification
and removal of precancerous polyps [21]. Accordingly, ADR has emerged as one of the most important quality metric in colonoscopy.
In this study, we found that issuing annual performance report cards significantly
improved ADR. On multivariate analysis, ADR improved by a striking 22 % in Year 1
and 30 % in Year 2 compared to baseline. The benefit of report cards depended on baseline
ADR. Among those who were below recommended ADR at baseline (< 25 %), it improved
2.2 times by the end of Year 2. Accordingly, ADR in this group increased from 15.9 %
at baseline to 28.6 % by the end of the study. Even among high performers who had
a baseline ADR ≥ 25 %, ADR improved by 33 % after Year 2. Thus, our results support
use of performance report cards to improve ADRs for all endoscopists.
In the absence of well-designed placebo-controlled, randomized clinical trials (RCTs),
observational studies have been used to justify the effectiveness of performance report
cards on improving ADR. Although our study is not the first to examine this intervention,
it is the first to do so in a more rigorous manner. Past studies examining the effect
of report cards on ADR or PDR either did not account for confounding variables [22]
[23]
[24]
[25] or controlled only for age and gender [26]. Given the strong association between variables such as patient age, gender, cecal
intubation, and bowel preparation quality with ADR [15]
[16]
[20], failure to adjust for these factors may lead to erroneous conclusions. As an example,
improvements in bowel preparation quality over time may have explained improvements
in ADR observed independent of the impact of report cards in prior uncontrolled studies.
This is particularly true since bowel preparation itself was a target for quality
improvement during the period during which the prior studies were conducted. In our
protocol, we prospectively measured potential confounders immediately after each colonoscopy
and adjusted for them in a multivariate analysis, leading to a more accurate assessment
of the impact of report cards on ADR. After controlling for confounders, we still
found a significant improvement in ADR. To our knowledge, our study provides the most
rigorous evidence for the effectiveness of performance report cards for improving
ADR to date.
A recent RCT comparing performance report cards to an intensive quality improvement
intervention bears mentioning [27]. In that study, colonoscopy screening center leaders who did not meet a target 25 %
ADR were randomized to receive either performance report cards or an intensive quality
improvement initiative. This consisted of a 2-hour pretraining assessment, 2 days
intensive hands-on training in the United Kingdom by expert colonoscopists, and further
evaluation of the first 30 colonoscopies performed upon return to their home endoscopy
center. Among endoscopists randomized to performance report cards, ADR increased by
2.3 %, which is lower than our observed 6.7 % increase. Although direct comparisons
between studies are difficult, we postulate several hypotheses to explain the difference.
First, their patient population consisted of average-risk screening and was thus inherently
different from ours. Second, endoscopists targeted in their study were those who did
not meet a benchmark ADR > 25 % and were mostly physicians in private practice. Third,
the inherent study design was different as theirs was experimental and ours observational.
Nonetheless, although the magnitudes of the effect estimates are different, it is
reassuring that both studies demonstrated a significant improvement in ADR after the
intervention.
In our study, there were no interventions other than performance report cards. Furthermore,
report cards were confidential and not shared with anyone other than the endoscopists.
There were no penalties, meetings with departmental chiefs, shaming, or other coercive
actions to induce practice changes. Unlike other studies, the only motivation was
the report card itself. We postulate that seeing one’s performance objectively compared
to others provided the introspection and motivation required for change.
Other than ADR, PDR improved over time as well. By the end of the study, the PDR had
increased by 28 % (P = 0.01). However, we chose to focus on ADR as our primary outcome since PDR is subject
to gaming. This is of particular importance for an intervention such as ours because
it is possible for unscrupulous endoscopists to remove small hyperplastic polyps from
the distal colon to improve this metric. There was also a trend towards improvement
in detection of advanced adenomas although that did not reach statistical significance,
likely due to the low event rate. Similarly, there were no more than 10 adenocarcinomas
each year, making it unfeasible to draw reasonable conclusions about the effects of
report cards on that outcome. Lastly, we found that ADR was affected by physician
specialty. Prior studies using administrative data to determine interval colorectal
cancer rates have reported differences between gastroenterologists, general surgeons,
internists, and family physicians, although the results have been inconsistent [28]
[29]
[30]
[31]. In our study, all endoscopists used the same equipment on the same patient population
and gastroenterologists were 52 % more likely to detect an adenoma than were general
surgeons. The reason for this discrepancy is beyond the scope of our study although
differences in formal endoscopy training have been postulated previously [30].
The main limitation of our study is the lack of randomized allocation of the study
intervention. Because this was a quality assurance intervention, all physicians were
issued report cards during the study. Without randomization, there is a risk for biased
results due to confounding variables. To address this possibility, we measured and
adjusted for known confounders of ADR as reported in the literature. Nonetheless,
there is still a risk for residual confounding by unmeasured factors. However, because
the criteria for CCC screening did not change during the study period, it is unlikely
to explain our findings. As a result, we believe our study provides a more robust
effect estimate of the impact of performance report cards on ADR than previous studies
that did not adjust for confounding. In our study, we chose to sample high-risk patients
who were undergoing colonoscopy due to a history of colon cancer in a first-degree
relative or who had a positive FOBT. We selected these patients because prior studies
only focused on the general screening population and identifying a benchmark ADR for
high-risk patients was previously identified as an unmet research need [15]. Nonetheless, there is no biological reason to expect that our intervention would
be ineffective in the general screening population although the magnitude of improvement
may differ, as it would when applied to different physicians, endoscopy units, and
geographic locations. Lastly, although we did not record details regarding the bowel
preparation itself (ie. type, dose, usage of split dosing, etc), our analysis adjusted
for the quality of bowel preparation at the time of the procedure to provide an unbiased
estimate of the treatment effect.
Conclusion
In conclusion, annual performance report cards increased ADR. The magnitude of the
benefit was greatest in those who were below the threshold of 25 % ADR although everyone
benefited. Our results support institutional adoption of this inexpensive and non-invasive
intervention to improve these quality metrics.
Supplement
Table S1
Multivariate analysis of factors influencing adenoma detection rate.
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
1.22 (1.02 – 1.45)
|
0.03
|
Year 2[1]
|
1.30 (1.07 – 1.58)
|
0.008
|
GI specialty[2]
|
1.52 (1.28 – 1.81)
|
< 0.001
|
Age
|
1.04 (1.03 – 1.05)
|
< 0.001
|
Male gender
|
1.96 (1.67 – 2.27)
|
< 0.001
|
Positive FOBT
|
1.20 (0.99 – 1.44)
|
0.05
|
Good bowel preparation[3]
|
4.32 (1.96 – 9.51)
|
< 0.001
|
Fair bowel preparation[3]
|
4.54 (2.14 – 9.66)
|
< 0.001
|
Cecal intubation
|
6.78 (3.30 – 13.91)
|
< 0.001
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation
Table S2
Multivariate analysis of factors influencing polyp detection rate.
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
1.14 (0.96 – 1.35)
|
0.13
|
Year 2[1]
|
1.28 (1.06 – 1.54)
|
0.01
|
GI specialty[2]
|
1.56 (1.32 – 1.84)
|
< 0.001
|
Age
|
1.03 (1.02 – 1.04)
|
< 0.0014
|
Female gender
|
0.54 (0.47 – 0.63)
|
< 0.001
|
Positive FOBT
|
1.05 (0.87 – 1.26)
|
0.61
|
Good bowel preparation[3]
|
3.35 (1.70 – 6.62)
|
< 0.001
|
Fair bowel preparation[3]
|
3.58 (1.89 – 6.79)
|
< 0.001
|
Cecal intubation
|
5.61 (3.09 – 10.20)
|
< 0.001
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation
Table S3
Multivariate analysis of high risk adenoma detection rate.
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
1.12 (0.81 – 1.54)
|
0.50
|
Year 2[1]
|
1.27 (0.90 – 1.78)
|
0.18
|
GI specialty[2]
|
1.30 (0.93 – 1.82)
|
0.13
|
Age
|
1.04 (1.02 – 1.05)
|
< 0.001
|
Female gender
|
0.55 (0.42 – 0.72)
|
< 0.001
|
Positive FOBT
|
3.42 (2.59 – 4.53)
|
< 0.001
|
Good bowel preparation[3]
|
5.62 (0.72 – 43.97)
|
0.10
|
Fair bowel preparation[3]
|
7.66 (1.03 – 57.13)
|
0.05
|
Cecal intubation
|
3.62 (1.10 – 11.89)
|
0.03
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation
Table S4
Multivariate analysis of adenocarcinoma detection rate
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
1.30 (0.48 – 3.53)
|
0.61
|
Year 2[1]
|
2.04 (0.75 – 5.58)
|
0.16
|
GI specialty[2]
|
1.62 (0.47 – 5.60)
|
0.45
|
Age
|
1.08 (1.04 – 1.12)
|
< 0.001
|
Female gender
|
1.00 (0.46 – 2.22)
|
0.99
|
Positive FOBT
|
12.28 (4.07 – 37.05)
|
< 0.001
|
Good bowel preparation[3]
|
–
|
–
|
Fair bowel preparation[3]
|
–
|
–
|
Cecal intubation
|
0.33 (0.10 – 1.08)
|
0.07
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation
Table S5
Multivariate analysis of adenoma detection rate for physicians with baseline ADR < 25 %.
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
2.21 (1.32 – 3.71)
|
0.003
|
Year 2[1]
|
2.17 (1.28 – 3.68)
|
0.004
|
GI specialty[2]
|
1.01 (0.67 – 1.52)
|
0.96
|
Age
|
1.04 (1.02 – 1.06)
|
< 0.001
|
Female gender
|
0.49 (0.33 – 0.73)
|
< 0.001
|
Positive FOBT
|
1.60 (1.00 – 2.55)
|
0.05
|
Good bowel preparation[3]
|
–
|
–
|
Fair bowel preparation[3]
|
–
|
–
|
Cecal intubation
|
5.47 (1.18 – 25.33)
|
0.03
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation
Table S6
Multivariate analysis of adenoma detection rate for physicians with baseline ADR ≥ 25 %.
|
Odds Ratio (95 % CI)
|
p-value
|
Year 1[1]
|
1.18 (0.97 – 1.43)
|
0.10
|
Year 2[1]
|
1.33 (1.07 – 1.66)
|
0.01
|
GI specialty[2]
|
1.42 (1.16 – 1.75)
|
0.001
|
Age
|
1.04 (1.03 – 1.05)
|
< 0.001
|
Female gender
|
0.50 (0.42 – 0.59)
|
< 0.001
|
Positive FOBT
|
1.17 (0.95 – 1.43)
|
0.14
|
Good bowel preparation[3]
|
4.83 (2.16 – 10.83)
|
< 0.001
|
Fair bowel preparation[3]
|
5.03 (2.34 – 10.82)
|
< 0.001
|
Cecal intubation
|
6.40 (2.81 – 14.60)
|
< 0.001
|
1 Compared to baseline
2 Compared to general surgery
3 Compared to poor bowel preparation