Introduction
Patient experience and complication rates are two of the seven key performance measures
for colonoscopy defined by the European Society of Gastrointestinal Endoscopy (ESGE)
guidelines [1].
Since 2014, the Polish Colorectal Cancer Screening Programme (PCSP) has routinely
used the Gastronet questionnaire to measure patient-reported outcomes [2]. Gastronet is a Norwegian quality-assurance (QA) program that was initiated in 2003
[3]. Patients fill in the questionnaire at home 1 day after colonoscopy and send it
back via traditional mail. Gastronet has proven to be an important QA platform, and
a network for research on quality issues, including endoscopy technique and technologies
– far beyond strict quality assurance [3]
[4]
[5]
[6]
[7]
[8]
[9]. Gastronet addresses an uncovered need to integrate reports on performance from
health care providers with those from patients and this information can be used for
research to improve health services. Gastronet also includes self-reporting of complications
[4] and it is currently considered the best source of information on colonoscopy complications
in Norway.
Even well-developed QA tools, however, are often only partially effective. Identified
barriers involve patients, professionals, interactions among professionals in teams,
the organizational context and the economic, political, and cultural context. Having
this in mind, there are several drawbacks to the current, paper-based Gastronet questionnaire.
First, the patient questionnaire response rate is lower than expected [10], not reaching the 90 % response rate recommended by the guidelines [1]. Second, it is time-consuming to scan and read paper-based questionnaires, which
impedes use of the tool on nationwide level in more densely populated countries like
Poland. Third, it is costly to return patient questionnaires via standard mail. Fourth,
the current complications assessment in Poland and Norway is not sufficient, as it
relies only on active self-reporting.
In 2015, around 80 % of households in Poland had Internet access and around 90 % of
the Polish population used mobile phones [11]. Widespread and constantly increasing access to digital media provides good background
for development and implementation of electronic QA questionnaires, which can possibly
overcome drawbacks of the current paper questionnaires.
This study aimed to investigate whether free choice of feedback form (paper questionnaire,
automated telephone response system or online questionnaire) results in better response
rates than the current, paper-based Gastronet form. Moreover, we evaluated whether
digital feedback is a valid method of obtaining information on screening colonoscopy
complications.
Patients and methods
PCSP design and subjects
The design of PCSP has been described previously [2]. Briefly, it is a programmatic, primary colonoscopy screening with roll-out that
began in 2012. All individuals aged 55 to 64 years who live in the geographic target
area of a screening center are considered eligible, but the program excludes people
with a colorectal cancer (CRC) diagnosis or previous screening colonoscopy. An updated
list of eligible individuals is acquired yearly from the national population registry
so that letters of invitation can be mailed. The central screening database includes
patient data, colonoscopy and histopathology reports, responses to the Gastronet questionnaire,
and information on screening centers and endoscopists.
Randomized controlled trial design
In our study, we included all primary colonoscopies performed within the PCSP framework
in two centers; in one from September 1, 2015 to December 31, 2016 and in the other
from September 1, 2015 to December 31, 2015. On registration to the screening program,
consecutive patients were randomized in a 1:1 ratio to either the control arm (practice
as usual, paper-based Gastronet filled in 1 day after colonoscopy) or the intervention
arm (free choice of paper or digital feedback), as shown in the flowchart in [Fig. 1]. Randomization was stratified by age and gender and automatically generated using
a computer-based randomization system. Participants randomized to the intervention
arm were asked about their preferred method of feedback (paper-based, automated telephone
or online survey) and were asked for the corresponding contact information (mobile
phone regardless of feedback type and email if online questionnaire was applied).
We did not choose the digital-only options in the intervention arm as in the initial
focus group evaluation, 60 % of patients would not use the digital method of feedback
when given a choice and leaving only digital options would hamper the response rate.
In the intervention study arm, patients were informed about a planned contact 30 days
after the procedure to assess complications. The general design of the intervention
arm is presented in [Fig. 2]. The primary endpoint was participant response to the feedback questionnaire. The
secondary endpoint was participant response to the complications questionnaire 30
days after telephone contact or self-reported complication on Gastronet paper questionnaire
received by the PCSP bureau. The study was registered as a randomized health services
study [12] at the Finnish Cancer Registry (registration number 008_2015_2_RHS, registry access
http://www.cancer.fi/rhs/).
Fig. 1 Study flowchart.
Fig. 2 Intervention arm design.
Gastronet questionnaire and digital implementation
The questionnaire (both paper-based and digital) includes five closed-ended questions:
(1) Were you satisfied with center’s quality? (yes; no); (2) Was the procedure painful?
(no; yes, slightly; yes, moderately; yes, very); (3) Did you feel any discomfort or
colicky abdominal pain after the procedure? (no; yes, slightly; yes, moderately; yes,
very). If the answer to (3) was yes, how long did you feel the aforementioned symptoms?
( < 1 hour; 1 – 3 hours; 3 – 6 hours; > 6 hours); (4) Are you satisfied with information
on the procedure itself and its results? (yes; no); (5) Did you experience any involuntary
leakage on your way back home? (yes; no). The language of all of the questions and
answers had been validated previously.
All screenees in the control arm and those in the intervention arm who chose paper
feedback were given the paper Gastronet questionnaire to be filled in at home on the
day after colonoscopy. The paper questionnaire was sent back via mail (in a prepaid
return envelope) to the coordinating office, where the forms were scanned and automatically
uploaded into the screening program database.
For screenees who chose automated telephone response, an SMS was sent the day after
the procedure, between 10 a.m. and noon, to remind them about planned contact. Patients
were permitted to respond if they were not able to give the feedback on that particular
day. In that case, contact could be on the next day (2 days after colonoscopy). Either
way, the automated telephone call was made between 5 p.m. and 7 p.m. After 2 days,
patients that did not reply to the first SMS, answer the phone or complete the telephone
survey received another SMS to which they could reply to trigger a telephone contact
at their convenience for 30 days after the procedure.
Screenees who chose the online questionnaire option received an email with a password-protected
link to the questionnaire. The online questionnaire could be filled in for 30 days
after the endoscopic procedure.
Complications assessment
Screenees in the control group did not receive any intervention in terms of complications
assessment. Data on complications were assessed routinely (self-reported by patients
on the paper Gastronet form).
Screenees in the intervention arm were contacted again for complications assessment
30 days after the procedure. The complications questionnaire includes up to four closed-type
questions: (1) Did you have any rectal bleeding during last 30 days? (yes; no); (2)
If yes: did the bleeding require hospitalization? (yes; no); (3) Did you have strong
abdominal pain during last 30 days that required hospitalization? (yes; no); (4) If
yes, was any surgical procedure necessary? (yes; no). Regardless of the first method
of contact (paper, telephone or email), the patients received (between 10 a.m. and
noon) an SMS reminder on planned telephone contact about complications. Screenees
were able to respond if they were not able to give the feedback on the current day.
In that case, the contact took place on the following day. Either way, the automated
telephone call was made between 5 p.m. and 7 p.m. As for non-compliers to “the-day-after”
response described above, non-compliers to telephone-based assessment of complications
received another SMS to which they could reply to trigger a telephone contact at their
convenience.
Later, data on complications received from the telephone survey were verified via
phone call from the PCSP bureau personnel to assess whether the complication truly
took place and the nature of it. Moreover, data collected with telephone contact in
the intervention arm were compared with the routine practice method of obtaining data
on complications (self-reported by screenees).
Statistical analysis
The power calculation was based on a patient questionnaire response rate within 2
weeks from the procedure (due to postal delay). We expected an increase in patient
questionnaire response rate from 80 % in the paper-based Gastronet group (data from
Gastronet program at the participating centers in Poland at the beginning of the trial)
to 85 % in the intervention group. Randomization was performed in a 1:1 ratio, stratified
by age and gender. Randomization was automatically performed within central database,
handling enrollment and arm assignment. To detect the difference in response rate
with a power of 0.90 at the 5 % level of significance, the study required 1,212 individuals
randomized to each of two groups. Assuming no dropout in the control arm (practice
as usual) and maximum of 5 % dropout in the intervention arm, we planned to include
approximately 1,260 individuals in each arm. Chi square test and exact Fisher test
were used to compare the groups. All tests were performed at 0.05 significance level.
Statistical analyses were performed using Stata software, version 13.1 (Stata Corporation,
College Station, Texas, United States).
Results
Baseline Characteristics
[Fig. 1] is the study flowchart. We included a total of 2,541 participants, randomized in
a 1:1 ratio either to the intervention arm (1,281 participants) or the control arm
(1,260 participants). Mean age was 59.88 years (SD 3.08) and 59.86 years (SD 3.04)
and the male to female ratio was 1.04 and 1.05 for the intervention and control arm,
respectively. Differences in participant characteristics between the intervention
and control arms were not statistically significant. In the intervention arm, 155
participants (12.1 %) chose the online survey, 287 (22.4 %) chose the automated telephone
survey and 839 (65.5 %) chose the paper-based survey. The first center included 2,323
participants (1,176 and 1,147 in the intervention and control arms, respectively)
and the second center included 218 participants (105 and 113 in the intervention and
control arms, respectively).
Gastronet response rate and survey results
The response rate for the total study population was 63.2 %. Overall response rate
was lower in participants aged less than 60 years compared to older individuals (57.8 %
vs 68.1 %, P < 0.001), male participants 61.3 % vs 65.1 % for women, P = 0.05), and in the smaller, private center no. 2, 49.1 % compared to 64.5 % in the
larger center no.1 (P < 0.001). [Table 1] shows a comparison of response rates in the intervention and control arms. There
was no statistically significant difference between the intervention and control arms
(64.8 % vs 61.5 %, P = 0.08). However, on subgroup analysis, we found a statistically significant increase
in response rates for younger patients, males and patients from the smaller center
in the intervention arm.
Table 1
Gastronet questionnaire response rates (%).
|
Intervention arm (free choice of paper, telephone or web-based questionnaire) (N = 1281)
|
Control arm (paper based) (N = 1260)
|
P value
|
|
Total (N = 2,541)
|
64.8 % (N = 830)
|
61.5 % (N = 775)
|
0.08
|
|
Per center
|
|
Center 1 (N = 2,323)
|
65.6 % (N = 771)
|
63.4 % (N = 727)
|
0.273
|
|
Center 2 (N = 218)
|
56.2 % (N = 59)
|
42.5 % (N = 48)
|
0.043
|
|
Per participant age
|
|
< 60 years old (N = 1,219)
|
60.8 % (N = 376)
|
54.7 % (N = 329)
|
0.031
|
|
≥ 60 years old (N = 1,322)
|
68.5 % (N = 454)
|
67.7 % (N = 446)
|
0.756
|
|
Per participant gender
|
|
Women (N = 1,243)
|
65.6 % (N = 412)
|
64.5 % (N = 397)
|
0.697
|
|
Men (N = 1,298)
|
64.0 % (N = 418)
|
58.6 % (N = 378)
|
0.045
|
|
Choice of paper questionnaire in the intervention arm (839 participants)
|
|
67.7 % (N = 568)
|
61.5 % (N = 775)
|
0.004
|
|
Choice of telephone questionnaire in the intervention arm (287 participants)
|
|
57.8 % (N = 166)
|
61.5 % (N = 775)
|
0.251
|
|
Choice of web-based questionnaire in the intervention arm (155 participants)
|
|
61.9 % (N = 96)
|
61.5 % (N = 775)
|
0.918
|
The response rate refers to properly filled in questionnaire.
P values refer to differences between control and intervention arm.
[Table 2] lists responses to Gastronet questions with regards to study arms. Statistically
significant differences in answers for some questions was due to higher number of
invalid answers in the intervention arm. Invalid answers were noted for 1.9 % of paper
questionnaires and in 9.6 % of telephone surveys (P < 0.001). No invalid answers were noted in online questionnaires. Colonoscopy was
moderately or severely painful in 17.3 % and 15.6 % of participants from the intervention
and control arms, respectively. Significant pain (moderate or severe) after colonoscopy
was reported in 13.3 % and 10.5 % of participants from the intervention and control
arms, respectively.
Table 2
Comparison of responses to Gastronet questions by study group.
|
Intervention arm
|
Control arm
|
P value
|
|
Were you satisfied with center’s quality?
|
|
Yes
|
99.0 %
|
98.8 %
|
0.93
|
|
No
|
0.4 %
|
0.5 %
|
|
NA
|
0.6 %
|
0.7 %
|
|
Was the procedure painful?
|
|
No pain
|
51.5 %
|
53.0 %
|
0.73
|
|
Slight pain
|
30.6 %
|
31.1 %
|
|
Moderate pain
|
11.3 %
|
10.4 %
|
|
Severe pain
|
6.0 %
|
5.2 %
|
|
NA
|
0.6 %
|
0.3 %
|
|
Did you feel any discomfort or colicky abdominal pain after the procedure?
|
|
No
|
54.5 %
|
55.2 %
|
0.005
|
|
Yes, slightly
|
31.1 %
|
34.3 %
|
|
Yes, moderately
|
8.3 %
|
7.4 %
|
|
Yes, very
|
5.0 %
|
3.1 %
|
|
NA
|
1.1 %
|
0
|
|
Are you satisfied with information on the procedure itself and its results?
|
|
No
|
1.0 %
|
0.7 %
|
0.019
|
|
Yes
|
92.8 %
|
95.9 %
|
|
Partially
|
4.6 %
|
3.0 %
|
|
NA
|
1.6 %
|
0.4 %
|
|
Did you experience any involuntary leakage on your way back home?
|
|
Yes
|
8.9 %
|
8.3 %
|
0.17
|
|
No
|
89.9 %
|
91.3 %
|
|
NA
|
1.2 %
|
0.4 %
|
NA, non-applicable (invalid answer)
Percentage values do not add up to 100 % as there were a small number of invalid answers
(i. e. pressing inappropriate button during phone survey or not answering or giving
more than one response in paper questionnaire).
Complications questionnaire coverage, response rate and complications rate
[Fig. 3] shows results of the complication questionnaire in the intervention arm. A total
of 79 participants (6.2 % of intervention arm) reported complications – 69 reported
bleeding and 14 abdominal pain requiring hospital stay (4 reported both complications).
On telephone call, most reported complications were verified as administrative errors,
leaving only two participants (0.2 %) who reported clinically relevant complications
– one had post-polypectomy bleeding requiring hospital admission and one had appendicitis
with appendectomy day after screening colonoscopy, without any procedures (polypectomy
and/or biopsy).
Fig. 3 Responses to complications questionnaire (including verification process).
In contrast, no participants in the control group reported any complications (self-reported).
Discussion
Today, health care should include modern pathways for patient feedback. Novel technologies
meet with patient acceptance in different medical fields [13]
[14]. Given the previously observed suboptimal response rate to the Gastronet questionnaire
[10], we were seeking alternative feedback methods to meet the ESGE quality criteria
[1]. Our study, to the best of our knowledge the first in a screening population, demonstrated
feasibility of digital patient feedback. Moreover, this is the first study with such
large sample size, evaluating utility of new technologies on the health service level.
We observed an increased overall feedback response rate. The difference, however,
was smaller than assumed and not statistically significant. Lack of significance is
probably due to too small a sample size, which was calculated to detect 5 % difference.
However, because this was a unique intervention, we had no previous data on which
to base our calculations. Still, in the intervention arm, most patients chose the
paper questionnaire, which may be attributed to older age. However, we assume that
in the upcoming years, patient preference will probably shift towards digital methods.
Importantly, we observed a significant difference for all subpopulations suffering
from too low response rate (poor responders). These subpopulations include: younger
participants (age < 60 years), men and participants screened at one of the centers.
Participants younger than 60 years had generally lower response rates, which most
likely can be attributed to the fact that most of them are still active workers [15], and therefore, they have less time to perform all the procedures associated with
sending back a paper questionnaire. For younger participants, response with a telephone
(and system programmed such that the call was placed after working hours) or a web-based
survey had a significant advantage over the paper version. On the other hand, men
were generally observed to be less responsive to medical interventions than women
[16]
[17]. In the case of PCSP, it results in worse program participation and lower response
rate to Gastronet questionnaire (historical PCSP data were not shown). Regarding one
of the centers with low response rates, we have previously observed significant differences
between centers in PCSP, which we attribute to differing ways of handling the questionnaire
(e. g., hand out by an administrative worker versus nurse or endoscopist). However,
it is not possible to identify the precise source of the problem. Geographical or
demographical differences were not relevant, as both centers are located in the same
city.
We did not find significant differences between the control and intervention arms
regarding patient satisfaction with center quality ([Table 2]). There were significantly different answers on satisfaction with information on
the procedure and results. However, the proportion of patients clearly not satisfied
was similar in both groups (the significance could be attributed to invalid answers).
Also, we observed significantly different answers regarding pain after the procedure.
The trial was not designed to explore differences in pain between groups and we did
not analyze whether they were significantly different with regard to previously reported
results on factors associated with painful colonoscopy [10]. On the other hand, the paper questionnaire could be filled in by patient either
too early (right after procedure) or too late (a few days after), resulting in biased
answers. The telephone survey was performed exactly 1 day after the procedure, so
the answer to this question could have been more precise.
Response rate improvement is the first milestone towards better understanding of patients’
experience with a colonoscopy screening program [10], ultimately leading to quality improvement. In the recent ESGE guidelines on quality
in colonoscopy [1], measuring patient experience was one of seven key performance measures. This further
implies the importance of response rate improvement. We cannot be certain of the opinions
of patients who did not respond to the Gastronet questionnaire nor do we know if they
were satisfied, and so, did not respond, or felt their responses would not matter,
or were so unsatisfied that they did not want any further interaction. Unfortunately,
there is no literature providing unbiased insight into reasons for non-response and
non-response also may reflect cultural differences.
In the medical field, most interventions to improve patient feedback focus not on
experience, but on ability to tailor medical interventions [14]. For example, in the field of chronic obstructive pulmonary disease, digital intervention
focuses on self-monitoring of symptoms, leading to improved self-management and earlier
intervention of medical professional [18]. Self-monitoring through a digital diary also has been tested in various other fields,
such as diabetes [19] or acquired brain injury [20]. To date, no trials exist on utility of digital tools to monitor patient experience
associated with colonoscopy or screening in general. All studies measuring patient
satisfaction, experience or pain associated with colonoscopy have been based solely
on either paper-based feedback sent by patients (e. g. Gastronet, VAS, GRS) or measurement
at the treatment site [3]
[7]
[9]
[21]
[22]
[23]
[24]
[25]. To gather the most objective and varied information on patient satisfaction, utilization
of new technologies should lead to significant improvement in this field.
Another important aspect covered by this trial is the feasibility of digital assessment
of colonoscopy complications. The previously mentioned ESGE guidelines emphasize the
importance of monitoring complications by including complication rate as one of the
key performance measures. As per definition of complications [26], we in fact monitored unscheduled further endoscopy procedures and emergency interventions,
focusing on post-colonoscopy bleeding and perforation. We used an automated phone
call as a “screening for complications,” allowing us to select participants to verify
answers. Even though false-positive reporting of complications did occur, in-person
verification was necessary only for a relatively small sample of participants (6.2 %).
This method has several strengths. It is cheap, fast and does not require additional
workforce. It is not reliant on access to registries of hospitalizations and deaths.
The very general nature of the questions reduces patient reluctance to respond, leaving
collection of more detailed information for the verification call. Because the call
is automated, patients are more open to answering the questions that might be embarrassing.
Data on complications can be gathered more efficiently than with a paper-based questionnaire,
because the evaluation covers 30 days after colonoscopy, whereas the paper questionnaire
could be sent back before a complication occurred. An automated phone call, however,
does have several limitations. The rate of response to the complications questionnaire
was similar to to the overall response rate to Gastronet, which resulted in one-third
of patients not giving feedback on adverse events. There also is a significant risk
that patients with complications will report incorrectly (stating that there were
no complication) or will not responding to the phone call, and there is a lack of
objectivity in comparison to analysis of registries or hospital records. However,
the main task of this trial was to show feasibility of automated monitoring. In terms
of this goal, the system proved its usefulness. Moreover, complication rates after
colonoscopy were similar to those reported previously in different settings [27]
[28]
[29]
[30]. The next step is to implement this approach on a nationwide scale, focus on conveying
the importance of monitoring both to screening program staff and participants and
objectify the findings through analysis of appropriate registries (hospitalization
and deaths), as the colonoscopy quality guidelines suggest [1]
[31]
[32]. We believe this approach will significantly increase response rates, leading to
more objective complication monitoring.
Our study has several strengths. It is the first trial to show the feasibility and
effectiveness of digital feedback in endoscopy and CRC screening settings. Moreover,
this is one of a few trials in this area of health services that was designed as a
randomized trial. It is worth noting that taking a digital approach to feedback is
a relatively new phenomenon in the field of medicine, therefore, the main goal of
the first trials (including this one) is to show the feasibility of digital systems.
On the other hand, there are a few limitations. We did not observe a significant increase
in response rate in the intervention group; it was seen only in specific subgroups
(suboptimal responders). The subgroups were not predefined, however, they emerged
using natural criteria: different sex, different centers etc. There is still uncertainty
about the colonoscopy experience of a large group of non-responders. However, even
though the trial was not designed in non-inferiority fashion, we believe that the
effect of free choice of feedback method is not worse than traditional, paper-based
Gastronet. Second, probably due to the small setting of trial (only two centers in
one city), we did not observe more significant changes. Third, we had no objective
method of verifying complications and we did not verify medical documentation for
all participants or in central registries.
Conclusion
In conclusion, our study showed the feasibility and effectiveness of an automated
system for monitoring patient feedback and complications after screening colonoscopy.
As the medical field becomes more patient- and technology-oriented, such changes in
health services should become more prevalent.
