Introduction
Capsule endoscopy (CE) is the preferred method for small bowel (SB) exploration. Despite
technological improvements, diagnostic yield of CE can be reduced by poor mucosal
visualization secondary to the presence of residue, bile, bubbles in the digestive
lumen, or as a result of insufficient or excessive brightness. The CE device has neither
washing nor suctioning capabilities. Given these limitations, the quality of bowel
preparation is of the utmost importance. There is no consensus regarding the preparation
regimen for CE, mainly because there is no validated tool to assess the quality of
mucosal visualization, contrary to colonoscopy for which the Boston Bowel Preparation
Scale (BPPS) is widely used [1]. Multiple bowel preparation regimens have been tested, aiming to improve mucosal
visualization, with conflicting results [2].
Qualitative and quantitative clinical scores aiming at the assessment of mucosal visualization
quality in CE have been established, with poor reproducibility, mostly due to the
fact that they are based on evaluation of thousands of CE still frames [3]
[4]
[5]
[6]
[7]
[8]. Brotz et al. proposed a 10-point grading scale comprising the following five items:
percentage of mucosa visualized, presence of fluid and debris, presence of bubbles,
bile and chime staining, and brightness [4]. This score is not validated but is widely used in the field of research. Computed
algorithms allowing automatic assessment of cleanliness of the SB during CE have been
developed. These algorithms are rapid and perfectly reproducible. Van Weyenberg et
al. made a proof of concept with a score based on the ratio of color intensities of
the red over green (R/G) channel of the tissue color bar of CE video segments [9].The concept of R/G ratio is based on the fact that a properly visible mucosa is
associated with red colors whereas a fecal-contaminated digestive lumen is associated
with green. This approach was also used by Abou Ali et al. at the still frame level
[10]. A R/G pixel ratio over 1.6 was found to yield a sensitivity and specificity of
91 % defining an adequate SB visualization. Pietri et al. recently developed a computer
algorithm based on a grey-level of co-occurrence matrix (GLCM) detector strategy,
also yielding high diagnostic performances (sensitivity and specificity of 95 %) in
terms of assessing the abundance of bubbles in SB-CE still frames [11].
The main objective of this study was to evaluate the diagnostic performance of three
computerized parameters (R/G ratio, bubble abundance, and brightness), individually
and combined, to assess the quality of SB mucosa visualization in third-generation
CE still frames.
Patients and methods
The study methodology is shown in a flowchart in [Fig. 1].
Fig. 1 Study methodology flowchart.
Still frame selection
Patients who had undergone third-generation SB-CE (Pillcam SB3, Medtronic, Minnesota)
in the setting of obscure gastrointestinal bleeding (OGIB) at Saint-Antoine Hospital,
Paris, were included in the study. All procedures were complete and normal, meaning
that the entire SB was examined and no lesion was observed. Incomplete and abnormal
SB-CE were not considered for inclusion. SB-CE videos were deidentified. According
to the French Data Protection Authority, no institutional ethical committee approval
was required for this retrospective analysis of de-identified SB frames. Finally,
600 SB still frames were randomly extracted from these video sequences.
Expert analysis
Three SB-CE expert readers (with more than 1000 CE readings) analyzed the 600 extracted
still frames independently, using a grid based on the quantitative scale by Brotz
et al. [4]. Two points were allotted to each of the five items (mucosa visibility, brightness,
bubbles, bile and chime, liquids and residues abundance) ([Table 1]) with an overall score per still frame ranging from 0 to 10. A still frame was categorized
as being of good visual quality when the mean of the three expertsʼ scores was ≥ 7/10.
This threshold is based on the distribution curve of the quantitative score developed
by Botz et al. [4]. By projection, regarding each item of the grading scale taking individually, a
mean score ≥ 1.4/2 was considered to be of good visual quality as well.
Table 1
Quality of mucosal visualization grading scale, based on Brotz et al. quantitative
scale [4].
|
Points
|
% of mucosa visualized
|
Liquids and residues abundance
|
Bubbles abundance
|
Chime/bile abundance
|
Reduction of brightness
|
|
0
|
< 80 %
|
Significant
|
Significant
|
Significant
|
Significant
|
|
1
|
80 – 89 %
|
Moderate
|
Moderate
|
Moderate
|
Moderate
|
|
2
|
≥ 90 %
|
Minimal/mild
|
Minimal/mild
|
Minimal/mild
|
Minimal/mild
|
Computer-aided and statistical analysis
Computerized analyses of the same 600 still frames were conducted using the MATLAB
software (MathWorks, Natick, Massachusetts, United States). For each still frame,
the following parameters were analyzed: (a) R/G ratio; (b) abundance of bubbles based
on a GLCM detector strategy; and (c) brightness index.
GLCM (Grey-Level Co-occurrence Matrix) is an algorithm that permits definition of
an image over a matrix showing the co-occurrence of gray-level by pair of pixels in
a given neighborhood. From this matrix, the contrast parameter is computed as proposed
by Haralick [12]. Index brightness concept is based on the fact that a variation of luminance or
color allows an image to be discernable.
We used the “random forests method” based on decision trees combination [13]. In the statistical learning area, decision trees describe how to classify a set
of complex data (such as a population, in our case, series of images) according to
a set of discriminating variables (here, R/G ratio, GLCM contrast, brightness) and
according to an objective set in terms of class number (here, images of “good” or
“bad” quality of visualization) based on a ground truth (here, the mean scores of
experts). To ensure better performance stability of this type of algorithm, decision
tree forests perform learning on multiple decision trees driven on slightly different
subsets of data created on the basis of the 500 images considered (strategy known
as “bootstrapping” [14]).
The first step consisted of an automated learning process, applied on a set of 500
randomly selected still frames among the 600 composing the database. The second step
consisted of a computed measurement of the remaining 100 still frames. To ensure the
statistical validity of these measures, both phases were repeated 10 times. Finally,
computerized measures were compared to the expertsʼ evaluation. A mean score of 7/10
was used as a threshold for adequate mucosal visualization. Performance of the computerized
analysis, sensitivity (Se), specificity (Sp), and positive (PPV) and negative (NPV)
predictive values were calculated for each of the three parameters individually and
then combined, using the expert analysis as reference. Se and Sp were computed from
ROC (receiver operating characteristic) curves obtained using the output of the algorithm
for each of the 100 test images: a probability of being of “good” or “bad” visualization.
More precisely for this study, we focused on the Se and Sp corresponding to the particular
“operating point” of the ROC curve for which the optimal trade-off was obtained.
The inter-observer correlation was calculated using κ statistics.
Study outcomes
The primary outcome of the study was the Se of the combined computerized score. The
secondary outcomes were the Sp, PPV and NPV of the combined score, the diagnostic
performances of each of the three parameters, and the time of analysis.
Results
Still frame dataset
Thirty patients with OGIB were selected. They all had a normal and complete SB-CE.
Six-hundred still frames were randomly extracted from these 30 videos. Examples of
selected still images are shown in [Fig. 2].
Fig. 2 Cleanliness of various small bowel still frames based on the 10-point grading scale
([Table 1]) adapted from Brotz et al. [4] quantitative scale. Red frames: inadequate mucosal visualization (score < 7/10)
images with score of a 1, b 2 c 3, d 4, e 5, f 6. Green frames: adequate mucosal visualization (score ≥ 7/10) images with score
of g 7, h 8, i 9, j 10.
Expert analysis
Among the still frames, 221 (37 %) were categorized as having adequate mucosal visualization
(mean score ≥ 7) ([Table 2]).
Table 2
Experts' analysis of small bowel capsule endoscopy still frames, using the 10-point
grading scale ([Table 1]) based on Brotz et al. quantitative scale [4].
|
Inadequate mucosal visualization (total score < 7)
|
Adequate mucosal visualization
(total score ≥ 7)
|
|
Expert 1 (n of images/%)
|
357 (59.5 %)
|
243 (40.5 %)
|
|
Expert 2 (n of images/%)
|
380 (63 %)
|
220 (37 %)
|
|
Expert 3 (n of images/%)
|
364 (61 %)
|
236 (39 %)
|
|
Mean score of 3 experts (n of images/%)
|
379 (63 %)
|
221 (37 %)
|
The interobserver correlation between the three experts ranged from good to excellent
(κ coefficient ranging between 0.81 and 0.87) ([Table 3]).
Table 3
Interobserver reproducibility of the expertsʼ analysis of small bowel capsule endoscopy
still frames.
|
Kappa coefficient
|
|
Expert 1-Expert 2
|
0.83
|
|
Expert 1-Expert 3
|
0.81
|
|
Expert 2-Expert 3
|
0.87
|
Computerized analysis and outcomes
The combination of the three parameters achieved the highest diagnostic performance,
with better discrimination between adequately and inadequately cleansed still frames
as compared to using two parameters combined or each parameter individually ([Table 4] and [Fig. 3]). Computerized analysis combining all 3 parameters demonstrated a Se of 90.0 % (95 %CI
[84.1 – 95.9]), a Sp of 87.7 % (95 %CI [81.3 – 94.2]), a PPV of 81.1 % (95 %CI [73.3 – 88.7]),
and a NPV of 93.7 % (95 %CI [88.9 – 98.4]). Reproducibility was optimal (κ coefficient = 1.0).
Mean time required to analyze a still frame using the computerized three-parameter
method was 34 ± 2 milliseconds using the MATLAB software. Extrapolated to a full-length
CE video comprising 50,000 images, analysis time would take 28 minutes.
Table 4
Diagnostic performances of computerized analysis to discriminate adequate from inadequate
still frames.
|
Computerized parameter(s)
|
Sensitivity
% (95 %CI)
|
Specificity
% (95 %CI)
|
|
R/G ratio
|
84.1 (76.9; 91.2)
|
78.6 (70.6; 86.7)
|
|
Abundance of bubbles
|
79.6 (71.7; 87.5)
|
73.6 (64.9; 82.2)
|
|
Brightness
|
73.9 (65.4; 82.6)
|
78.4 (70.3; 86.4)
|
|
R/G ratio + abundance of bubbles
|
85.2 (78.3; 92.2)
|
86.3 (79.6; 93.1)
|
|
Abundance of bubbles + brightness
|
85.2 (78.2; 92.2)
|
79.0 (71.0; 87.0)
|
|
R/G ratio + brightness
|
86.1 (76.3; 92.9)
|
86.2 (79.4; 92.9)
|
|
R/G ratio + abundance of bubbles + brightness
|
90.0 (84.1; 95.9)
|
87.7 (81.3; 94.2)
|
R/G, red on green pixel ratio; CI, confidence interval
Fig. 3 Receiver operating characteristic curves for computerized parameters evaluating the
cleanliness of small bowel capsule endoscopy still frames. R/G, red over green pixel
ratio; GLCM, grey-level of co-occurrence matrix
Discussion
We propose a multi-criterion computer-assisted algorithm to determine whether mucosal
visualization is adequate on SB-CE still frames. The combination of the R/G ratio,
abundance of bubbles, and brightness achieved a Se of 90.0 % and a Sp of 87.7 %, with
optimal reproducibility, compared to human expert analysis. Further, the time of analysis
was short.
One of the strengths of this study was the solid analysis of still frames, performed
using the 10-point grading scale, to obtain an adequate ground truth to which the
computerized analysis was later compared. Three experienced capsule readers, blinded
to the results of the computer-assisted analysis, performed this evaluation. A five-item
standardized and precise scale allowed reliable clinical assessment of still frame
quality of SB mucosa visualization [4]. Two of the three parameters used during computerized analysis had previously been
evaluated and validated [10]
[11]. We noted that the diagnostic performances of each individual parameter was lower
than in previous studies [10]
[11]. This difference can be explained by the fact that the expert analysis was more
accurate, containing various parameters evaluated.
Some limitations of this study should be mentioned. First, we selected and evaluated
still frames rather than video sequences. Most previously published studies aiming
to build a SB-CE cleansing score were based on video sequence analysis by human readers.
However, this could lead to heterogeneous results and variable conclusions [4]
[6]
[7]
[8]. Our hypothesis was that a computer-assisted analysis would be objective, rapid,
comprehensive, and reproducible. Taking into account that a computer-assisted scoring
system would be based on a frame-by-frame analysis, we used still frames to build
our ground truth. We used a grid based on quantitative scale developed by Brotz et
al. [4] to evaluate still frames, although it was developed to evaluate video sequences.
Interobserver correlation was good with κ coefficient > 0.80. Additionally, we have
started an evaluation of the multi-criterion score at the video level. Second, only
normal SB-CEs performed in the setting of OGIB were included, which is not representative
of the general population. We believe that any supposedly normal SB-CE should have
adequate bowel preparation to be considered reliable. Thus, assessment of the quality
of bowel preparation is much more significant in this setting, as opposed to when
active bleeding or an abnormality is identified, regardless of the quality of preparation.
Third, only cases of OGIB were selected, given it is the most prevalent indication
for SB-CE, and because this group thus provides a homogenous population. Finally,
we used the random forests method with two sets of 500 and 100 still frames. Despite
an imbalance of datasets, this method allows a more solid automatic learning step
and more reproducible results of the validation step.
Conclusion
In conclusion, this multi-criterion score constitutes a comprehensive, objective,
reproducible, reliable, automated, and rapid test for evaluation of the level of cleanliness
of SB-CE still frames. This automated score circumvents the subjectivity of qualitative
or quantitative grading systems based on human reading. Further research is warranted
to determine which proportion of adequately cleansed frames defines an acceptable
quality of preparation of SB-CE in clinical practice, which would then be incorporated
into CE software. For that purpose, a patent is pending at the European patent office.
