Validation of the GPAT – the Global Polypectomy Assessment Tool: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement

Abstract

Background Colorectal polypectomy is operator dependent, with variable rates of complete resection. The currently available assessment tools do not provide specific competency-based evaluation of provider technique. We aimed to validate the Global Polypectomy Assessment Tool (GPAT), a novel competency assessment tool for colorectal polypectomy.

Methods GPAT was derived from the ESGE Curriculum for Training in endoscopic mucosal resection in the colon. Members of the curriculum taskforce plus three invited trainees and three medical students (collectively: the assessors) anonymously assessed nine endoscopic-view only polypectomy videos. The primary end point was the correlation of the assessors’ GPAT scores with a consensus-derived reference GPAT score per video. Secondary end points were the assessors’ subjective impression versus their GPAT score and interobserver agreement among assessors’ GPAT scores.

Results 171 GPAT assessments by 19 assessors (consultant gastroenterologists [n = 10], trainee gastroenterologists [n = 4], consultant surgeons [n = 2], and medical students [n = 3]) were analyzed. Reference GPAT scores did not differ significantly from those of the assessors (73.1 % [95 %CI 64.6 %–81.6 %] vs. 69.3 % [95 %CI 64.9 %–81.2 %]; P = 0.47). There was moderate IOA in GPAT scores among gastroenterologists (intraclass correlation coefficient [ICC], 0.52 [moderate]) but not among nongastroenterologists (ICC 0.32 [poor]). GPAT correlated with assessors’ subjective impression of polypectomy quality (correlation coefficient 0.98 [95 %CI 0.90–1.00]; P < 0.001). Overall assessors’ qualitative usability scoring of GPAT was positive.

Conclusions GPAT allows standardized scoring of polypectomies, with moderate IOA among gastroenterologists and correlation with subjective impressions of polypectomy quality. GPAT could standardize assessment of trainee polypectomy competency offering structured feedback on performance.

Abbreviations

CRC: colorectal cancer

CSP: cold snare polypectomy

DOPyS: Direct Observation of Polypectomy Skills

EMR: endoscopic mucosal resection

ESGE: European Society of Gastrointestinal Endoscopy

GPAT: Global Polypectomy Assessment Tool

ICC: intraclass correlation coefficient

IOA: interobserver agreement

HSP: hot snare polypectomy

PCCRC: post-colonoscopy CRC

SMSA: size, morphology, site, and access

Introduction

Colorectal cancer (CRC) can be prevented by detection and complete resection of colorectal polyps using endoscopic polypectomy [1] [2]. Despite a large and growing body of published evidence on how to perform high quality polypectomy, rates of incomplete resection ranging from 1.5 % [3] to 17.7 % [4], even for small (< 10 mm) polyps, are reported [5] [6] [7]; the situation for larger polyps is even worse [8]. Incomplete resection risks propagation of residual polyp tissue, resource intensive follow-up [9], and even post-colonoscopy CRC (PCCRC) [10] [11] [12] [13] [14].

The observed variation in rates of incomplete polyp resection is likely to depend on training. This currently is often experiential, unstructured, and dependent on trainers without conscious competence in the technique. High quality training requires a competency framework to allow standardized communication of best practice between the trainer and trainee. The most commonly used tool to assess polypectomy competency is DOPyS [15] – a score derived in 2011, which largely focuses on subjective, nonevidence-based statements and requires significant experience for its effective use [16].

In conjunction with the development of the recent ESGE Curriculum for Training in endoscopic mucosal resection in the colon [17], we sought to develop a modern competency assessment tool (the Global Polypectomy Assessment Tool [GPAT]), based on a Delphi consensus process [18], defining best practice in all types of endoscopic polypectomy, and to validate it in a video-based study involving a varied population of endoscopists.

Methods

Development of the GPAT

The GPAT subtaskforce (Table 1 s, see online-only Supplementary material) selected statements from the final ESGE Curriculum for Training in endoscopic mucosal resection in the colon ([Fig. 1]) [17] (hereafter referred to as “the Curriculum”) that focused on polypectomy technique and which could be assessed from a video. These statements were included in the preliminary GPAT.

Fig. 1 Example images illustrating best practice techniques (with B3,4 and C3,4, poor practice examples) in endoscopic mucosal resection for: A global competencies; B injection technique; C snare placement technique; D defect assessment technique; showing: A1,2 full appreciation/demonstration of the extent of the polyp to be resected; A3 best positioning with respect to the polyp (at 6 o’clock, close to the colonoscope); A4 selection of the appropriate technique for the polyp to be resected (e. g. correct decision for en bloc cold snare polypectomy in this example); A5 tip control (controlled stable and purposeful, resulting in uniform application of snare-tip soft coagulation); B1,2 injection performed in the correct plane, facilitating access to the lesion; B3 transmural nondynamic injection (poor practice); B4 nonlifting, with repeated and failed submucosal injection resulting in intramucosal blebs (poor practice); C1 the snare oriented near to 6 o’clock, visualization of the snare V during closure, with the snare near to the colonoscope; C2 use of the transected tissue edge as a guide (within the defect), and visualization of the snare V during closure, with the snare close to the colonoscope; C3 the snare positioned far from the colonoscope, with too much snare extended from the sheath and no margin of normal tissue (poor practice); C4 poor tissue capture and too far from the colonoscope (likely to result in scraping of polyp tissue and incomplete mucosal layer excision; poor practice); D1 complete margin ablation, following systematic application with entire margin ablation achieved; D2,4 incomplete mucosal layer excision with evidence of residual polyp tissue on islands of muscularis mucosae; D3 residual polyp tissue at the resection margin; D5 evidence of deep mural injury (DMI) type I, with residual polyp tissue at the defect edge, and failed submucosal injection resulting in an intramucosal bleb; D6 intraprocedural bleeding; D7 DMI type I; D8 DMI type III associated with an area of fibrosis; D9 DMI type IV; D10 complete closure of a post-endoscopic mucosal resection (EMR) mucosal defect to prevent post-EMR bleeding.

Structure of the GPAT

The GPAT was designed so that it could be used for all types and sizes of polyps in the colon, and such that it adapts depending on whether cold snare polypectomy (CSP) or hot snare polypectomy (HSP) is performed. Certain statements that are broadly applicable were defined as mandatory for both CSP and HSP, whereas some were defined as mandatory for HSP only ([Table 1]). Non-mandatory questions can be skipped and, if skipped, do not contribute to the denominator or numerator of the GPAT.

Table 1 Components of the Global Polypectomy Assessment Tool (GPAT). An abbreviated version for use during an endoscopic procedure is shown in Table 2 s.

Component	Possible responses and scoring		Mandatory[1]		Curriculum statement[2]	Maximum score
			Hot	Cold
Domain: Global competencies						20
Tip control	1 Very poor	Uncontrolled, shaky and undirected	X	X		5
	5 Very good	Controlled, stable and purposeful
Fully appreciates/ demonstrates extent of the polyp to be resected	1 Very poor	Focuses on one area, does not demonstrate appreciation of the entire polyp	X	X	3(i), 3(ii), 3(iii)	5
	5 Very good	Clearly appreciates entire extent of the polyp; approach and resection reflect this
Positioning with respect to the polyp	1 Very poor	Lesion not at 6 o’clock, far from the colonoscope, fluid covering lesion (poor use of gravity)	X	X	17(iii), 20(iii)	5
	5 Very good	Lesion at or near 6 o’clock, close to the colonoscope, fluid lies away from lesion (good use of gravity)
Technique selected is appropriate for the polyp	1 Very poor	No clear need for en bloc resection if selected, lesion unsuitable for cold snare, hot snare for polyp < 10 mm	X	X	4	5
	5 Very good	Correct decision for en bloc vs. piecemeal resection, hot vs. cold appropriate for the polyp
Domain: Injection technique (Best Practice Video Chapter 1)3						15
Injection is performed in the correct plane	1 Very poor	Injection infrequently results in sustained submucosal lifting (transmural / intramucosal injection)	X		16(iv), 16(v), 16(vi), 16(vii) (cold snare) 19(v), 19(vi), 19(vii), 19(viii) (hot snare)	5
	5 Very good	The submucosal plane is quickly found and rapidly results in sustained mucosal lifting
Injection is performed dynamically	1 Very poor	Once the needle is situated in the submucosa, there is no movement of the needle away from the muscularis toward the center of the lumen	X		16(iv) (cold snare) 19(v) (hot snare)	5
	5 Very good	Once the needle is in the submucosa, there is graduated movement of the needle away from the muscularis towards the center of the lumen
Injection is used to improve lesion access	1 Very poor	Injection does not facilitate access to the target lesion	X		16(i), 16(viii) (cold snare) 19(i), 19(ii), 19(ix) (hot snare)	5
	5 Very good	Injection clearly facilitates access to the target lesion
Domain: Snare placement technique (Best Practice Video Chapter 2 and 3)3						25
Appropriate snare size/type selected	1 Very poor	Snare clearly too large/small and of incorrect type (thin wire vs. thick wire) for the polyp	X	X	17(i), 17(ii), 20(i), 20(ii)	5
	5 Very good	Snare of appropriate size and type for the polyp
Stable position with lesion at 6 o’clock OR transformed to 6 o’clock	1 Very poor	Snare position is not consistently maintained at 6 o’clock and/or the position is unstable	X	X	17(iii), 20(iii)	5
	5 Very good	Snare position is consistently maintained at 6 o’clock and the position is stable
Maximizing snare capture	1 Very poor	Poor capture of tissue/scrapes the surface of the polyp/no use of downward pressure/no use of gas aspiration/may result in incomplete mucosal layer excision	X	X	17(iii), 20(iii)	5
	5 Very good	Good capture of polyp tissue within snare/use of downward pressure/use of gas aspiration resulting in complete capture of adequate target tissue
Snare V precisely visualized during placement and closure	1 Very poor	Snare V not visualized during closure and far from the colonoscope	X	X	17(vii), 17(ix), 17(x) 20(vii), 20(ix), 20(x), 20(xi)	5
	5 Very good	Snare V visualized consistently during closure and near to the colonoscope
Residual tissue islands avoided if piecemeal resection or complete if en bloc	1 Very poor	Snare placement does not include normal margin (at edge) or does not use transected tissue edge (within lesion) as a guide resulting in tissue islands/incomplete en bloc resection	X	X	17(iii) 20(iii)	5
	5 Very good	Snare placement includes > 2–3 mm normal margin (at edge) of tissue or uses transected tissue edge as a guide (within defect) resulting in no tissue islands/complete en bloc
Domain: Safety checks prior to resection (Best Practice Video Chapter 4)3						10
Moves closed snare to confirm independent movement from deeper structures	1 Very poor	Does not check tissue mobility prior to transection with respect to deeper structures	X		20(xv), 20(xvi)	5
	5 Very good	Checks mobility prior to transection with respect to deeper structures
Lifts snare away from muscularis propria prior to cutting	1 Very poor	Does not lift the snare prior to applying electrosurgical energy	X		20(xx)	5
	5 Very good	Lifts the snare away from the muscularis prior to the application of electrosurgical energy
Doman: Defect assessment after resection (Best Practice Video Chapter 5 and 6)3						20
Mucosa: looks for, detects, and removes residual adenomatous tissue at margin and within defect	1 Very poor	Does not ostensibly and systematically check for residual adenomatous tissue at the defect margin or within the defect, and/or does not remove it successfully	X	X	22(i), 22(ii), 22(iii), 22(iv)	5
	5 Very good	Ostensibly and systematically checks for residual adenomatous tissue within the defect and at the defect margin, and removes it successfully
Thermal ablation of the post-EMR margin	1 Very poor	Unsteady application, results in areas of incomplete ablation, ablates visible polyp tissue, messy result			22(iv)	5
	5 Very good	Steady systematic application, does not ablate visible polyp tissue, complete ablation of the entire margin achieved
Submucosa: looks for, detects, and treats any bleeding vessels within the defect	1 Very poor	Neither detects nor treats bleeding vessels in submucosa; treats benign submucosal appearances	X		22(v), 22(vi), 22(vii), 22(viii)	5
	5 Very good	Detects and treats bleeding vessels in the submucosa; does not treat other submucosal appearances including herniating vessels
Muscularis: looks for, detects, and treats deep mural injury ≥ II (Sydney classification)	1 Very poor	Misses signs of deep mural injury (types II–V) which require clip closure	X		22(ix), 22(x), 22(xi)	5
	5 Very good	Detects and treats types II–V deep mural injury or confirms they are not present
Domain: Accessory techniques in polypectomy (Best Practice Video Chapter 7)[3]						15
Placement of through- the-scope clips	1 Very poor	Poor tissue capture, poor use of suction and positioning to maximize correct orientation and amount of tissue captured			29(x) 30(iv), 30(v), 30(vi)	5
	5 Very good	Good use of suction, positioning, and rotation to capture required tissue and achieves secure appearing closure
Use of polyp retrieval device	1 Very poor	Poor positioning, does not capture all pieces, does not use sequential place and retrieve technique			25(i)	5
	5 Very good	6 o’clock position, sequential place and retrieve technique applied, captures all pieces successfully
Use of coagulation grasper	1 Very poor	Does not use water, does not wait for cessation of bleeding after forceps closure prior to application of electrosurgical energy, does not tent vessel away from the muscularis to apply electrosurgical energy			29(ix)	5
	5 Very good	Uses water to identify the causative vessel, confirms correct placement with cessation of bleeding after closure, tents vessel away from the muscularis to apply electrosurgical energy
Total GPAT score						105[4]

EMR, endoscopic mucosal resection.

¹ “Hot” and “cold” refer to snare polypectomy with and without use of electrosurgical energy, respectively.

² Refers to statements from the ESGE Curriculum for Training in endoscopic mucosal resection in the colon [17].

³ Refers to the best practice video [19].

⁴ Denotes maximum denominator; the actual denominator depends on the questions filled in.

Once selected, GPAT statements were modified for the purpose of an online tool and grouped into domains ([Table 1]) [19]. To aid interpretation, text was attached to each GPAT statement describing important aspects of best/poor practice. Statements could be scored from 1 (poor) to 5 (very good) on a Likert scale. Example videos were created and attached to the statements to illustrate best practice [20].

Once all required fields are completed, an overall score (x/y [the GPAT score]) is generated by this tool, providing an indication of the quality of the polypectomy. Whilst the numerator (x) of this fraction defines the sum of all allocated scores, the denominator (y) reflects the number of questions answered (five points for each question) depending on the type of polypectomy (HSP vs. CSP), whether any accessory techniques were used (clips, polyp retrieval devices, or coagulation graspers), and if margin ablation was performed. GPAT (with all optional fields filled in) is distributed in the following way over its domains, with a maximum denominator of 105: global competencies, 20/105 (19.0 %); injection technique, 15/105 (14.3 %); snare placement technique, 25/105 (23.8 %); safety checks prior to resection, 10/105 (9.5 %); defect assessment, 20/105 (19.0 %); accessory techniques, 15/105 (14.3 %).

The “size, morphology, site, and access” (SMSA) score [21], a widely established and validated tool in grading the difficulty of polypectomy, is calculated alongside the GPAT. Two further published tools to grade EMR difficulty [21] [22] were combined by the authors into the SMSA+ score and included in the GPAT. The SMSA and SMSA+ scores were included to grade procedural complexity and did not contribute to the GPAT score.

Round 1

In the first round of the Delphi process, the statements for the GPAT and the accompanying best practice statements and videos were subjected to review by the whole curriculum taskforce. Statements which did not reach a level of agreement of ≥ 80 % were removed. Best practice statements/videos that were not agreed on were modified by the GPAT subtaskforce. A subsequent iterative round of voting was then performed using the same methodology.

Round 2

The GPAT subtaskforce used the agreed GPAT statements to assess polypectomy videos in an anonymous online environment to assess their usability for this validation study. The candidate videos of polypectomies (endoscopic-view only) were selected from a prospectively collected database during a 6-month period (November 2020–April 2021) by endoscopists with varying levels of experience. Both endoscopists and patients consented to inclusion in the study. The endoscopists did not know that the videos would be used for polypectomy technique assessment, nor were they aware which parameters would be used.

The study was approved by the institutional review board of the Ghent University Hospital in March 2023.

A total of 43 polypectomy videos were anonymized and edited to show one polypectomy (video starting at the inspection of the polyp and ending with inspection of the post-polypectomy defect). Videos were rejected if: (i) the total length of the polypectomy was > 10 minutes (five videos); or (ii) it was considered that poor video quality or an incomplete video would potentially interfere with interpretation (15 videos). From the remaining 23 videos, nine were selected, which represented a balance of expert performed, piecemeal versus en bloc resection, polyp size, cold versus hot snare resection, and location left versus right colon [23]. More information can be found in Appendix 1 s, Table 3 s.

Round 3

In the third round, members of the curriculum taskforce (10 consultant gastroenterologists, two consultant surgeons, and one trainee gastroenterologist) plus three extra invited trainee gastroenterologists and three invited medical students (the assessors) were requested to rate the nine videos retained in round 2. The GPAT subtaskforce authors, who were also involved in statement review, separately rated the videos in a consensus meeting and this single score per video served as the reference GPAT score. The medical students involved had never observed a colonic polypectomy. Assessors were required to give their explicit consent for the use of their entered data in this study.

To facilitate anonymous rating, an online survey was created using SurveyMonkey (Momentive, USA). Before starting the survey, the assessors were asked to watch a short introductory video of 4 minutes 41 seconds (explaining the design of the survey [2 minutes 4 seconds] and how to use the GPAT [2 minutes 37 seconds] ([Video 1]) [24]. All assessors were asked to apply the GPAT to the same nine polypectomy videos, which were presented in a random order.

Video 1 Video used as an introduction to the online survey prior to video rating.
The first part (0:00–0:38) explains the purpose of the survey. The second part (0:39–01:38) provides practical information about the survey. The third part (01:39–03:39) explains the GPAT tool and how to use it. The final part (03:40–4:40) explains how to fill in the survey after calculating the GPAT score using the online calculator.

Assessor demographic data were collected. For each polyp, the location was given in the introductory text. The first survey question was a subjective score (out of 10) on the quality of the polypectomy (“overall subjective impression score”). The second question required the assessor to use the online version of the GPAT to rate the video. The third (qualitative) question enquired about aspects of the polypectomy that could not be scored using the tool but that the participant felt were worth mentioning. The fourth qualitative question enquired about the quality of the assessed video (Appendix 2 s).

End points

The primary end point was the correlation of assessor GPAT scores with the reference GPAT score.

The secondary end points were:

• interobserver agreement (IOA) among assessors’ GPAT scores

• assessors’ subjective impression score versus their GPAT score

• correlation of GPAT scores overall and per domain with the reference GPAT score.

Statistics

Data were analyzed using R Studio, version 4.2.2 [25]. The IOA was interpreted using the intraclass correlation coefficient (ICC) with criteria proposed by Koo and Li [26]: < 0.50, poor agreement; > 0.5 and < 0.75, moderate agreement; > 0.75 and < 0.9, good agreement; > 0.9, excellent agreement. Correlation between the endoscopists’ overall impression and the GPAT was calculated using the Pearson correlation coefficient, and other comparisons between continuous variables were calculated using a t test. Two-sided P values < 0.05 were considered significant.

Results

Demographics

A total of 171 GPAT assessments were collected. The assessors, who originated from seven countries, were consultant gastroenterologists (n = 10; 47.3 %), trainee gastroenterologists (n = 4; 21.1 %), consultant surgeons (n = 2; 10.5 %), and medical students (n = 3; 15.8 %), of whom 11 (57.8 %) had performed over 1000 colonoscopies (Table 4 s).

Assessors’ GPAT scores versus reference GPAT score

The GPAT scores assigned by the assessors and the reference GPAT score were positively correlated (correlation coefficient 0.75 [95 %CI 0.17–0.94]; P = 0.02) (Fig. 1 s) and there was no significant difference between the overall GPAT scores when considering all polyps together (mean reference GPAT score 73.1 % [95 %CI 64.6 %–81.6 %] vs. mean assessor score 69.3 % [95 %CI 64.9 %–81.2 %]; P = 0.47) ([Table 2]; [Fig. 2]).

Table 2 Primary outcome: comparison of assessors’ Global Polypectomy Assessment Tool (GPAT) scores versus the reference score, per polyp and overall.

		Polyp number and method of polypectomy									All
		1	2	3	4	5	6	7	8	9
		Cold	Cold	Cold	Cold	Hot	Hot	Hot	Hot	Hot
General	RS, %	65	80	90	75	75	60	85	100	85	79.4
	AS, %	53	82.5	77.5	67.2	75	67.2	72.8	80	78.3	72.6
	Δ	−12	2.5	−12.5	−7.8	0	7.2	−12.2	−20	−6.7	−6.8
	P value	**	ns	***	**	ns	ns	***	***	*	ns
Injection	RS, %	26.7	80	NA	NA	53.3	NA	100	73.3	66.7	65.7
	AS, %	40.7	78.1	NA	NA	70.4	NA	71.2	78	78.1	67
	Δ	14	−1.9	NA	NA	17.1	NA	−28.8	4.7	11.4	1.3
	P value	**	ns	NA	NA	**	NA	***	ns	***	ns
Snare	RS, %	55	70	75	70	60	65	80	90	80	71.7
	AS, %	48.2	85.7	72.6	55.7	77	67.8	64.2	79.6	77.3	69.8
	Δ	−6.8	15.7	−2.4	−14.3	17	2.8	−15.8	−10.4	−2.7	−1.9
	P value	ns	***	ns	***	***	ns	***	**	ns	ns
Safety	RS, %	NA	80	NA	NA	70	60	NA	90	70	74
	AS, %	NA	85	NA	NA	65	62.9	NA	82.3	79.4	74.9
	Δ	NA	5	NA	NA	−5	2.9	NA	−7.7	9.4	0.9
	P value	NA	*	NA	NA	ns	ns	NA	*	*	ns
Defect	RS, %	60	60	60	80	80	66.6	60	86.7	80	70.3
	AS, %	51.5	75.8	74.4	66.1	70.6	53.2	68.2	70	79.2	67.7
	Δ	−8.5	15.8	14.4	−13.9	−9.4	−13.4	8.2	−16.7	−0.8	−2.6
	P value	ns	***	**	**	**	*	ns	***	ns	ns
Accessories	RS, %	NA	NA	NA	NA	60	60	NA	40	NA	53.3
	AS, %	NA	NA	NA	NA	71.5	63.1	NA	57.3	NA	64
	Δ	NA	NA	NA	NA	11.5	3.1	NA	17.3	NA	10.7
	P value	NA	NA	NA	NA	**	ns	NA	**	NA	ns
GPAT	RS, %	51.7	74.1	80	73.3	67.7	62.3	85	85.9	77.6	73.1 (11.1)
	AS (SD), %	48.1 (14.5)	81.7 (8.8)	74.5 (11.9)	61.5 (10.6)	72.4 (11.9)	61.8 (15.8)	69.1 (12)	76.7 (10.9)	78.2 (10.8)	69.3 (10.5)
	Δ	−3.6	7.6	−5.5	−11.8	4.7	−0.5	−15.9	−9.2	0.6	−3.8
	P value	0.3	0.02	0.06	< 0.001	0.11	0.88	< 0.001	0.002	0.81	0.47

AS, assessors’ GPAT score; NA, not applicable (polyp did not contain this domain); RS, reference GPAT score; Δ, difference between AS and RS.

* P < 0.05; ** P < 0.01; *** P < 0.001; ns, not significant.

Fig. 2 Comparison of Global Polypectomy Assessment Tool (GPAT) score determined by assessors with the reference GPAT score per polyp.
* P < 0.05; ** P < 0.01; *** P < 0.001; ns, nonsignificant.

Statistically significant differences were seen between the reference GPAT score and the assessors’ scores in 4/9 polyps (numbers 2, 4, 7, and 8); three of these polypectomies were performed using CSP. An attempt by the GPAT subtaskforce to explain these discrepancies can be found in Appendix 3 s.

When considering specific domains of the GPAT individually, there was no significant difference in any domain when all nine polyps were considered together between the mean assessor GPAT scores and the reference GPAT score ([Table 2]; Appendix 4 s; Fig. 2 s). Differences were only seen when analysis was restricted to single domains of single polyps.

Assessors’ subjective impression of the polypectomy versus GPAT

The GPAT scores were positively correlated with the assessors’ overall subjective impression scores (correlation coefficient 0.98 [95 %CI 0.90–1.00]; P < 0.001) ([Fig. 3]). The assessors’ mean overall subjective impression score was similar to their mean GPAT score (61.6 % [95 %CI 54.5 %–79.5 %] vs. 69.3 % [95 %CI 64.9 %–81.2 %]; P = 0.23).

Fig. 3 Scatterplot of the mean subjective overall impression score and Global Polypectomy Assessment Tool (GPAT) score with a line of best fit. 

Interassessor agreement for specific polypectomies

When considering only gastroenterologists, moderate agreement was found in the overall GPAT scores for both consultants and trainees (ICC 0.51 [95 %CI 0.28–0.81] and ICC 0.60 [95 %CI 0.29–0.87], respectively). When considering consultant gastroenterologists and trainee gastroenterologists together (forming the group of assessors with experience in polypectomy), the agreement was still moderate (ICC 0.52 [95 %CI 0.30–0.81]). In contrast, when considering assessors without any experience in polypectomy (medical students), poor agreement was found (ICC 0.32 [95 %CI 0.01–0.72]). The overall IOA for GPAT was poor (ICC 0.25 [95 %CI 0.10–0.64]). For comparison, the SMSA IOA was similarly poor (ICC 0.42 [95 %CI 0.22–0.74]) ([Table 3]).

Table 3 Interobserver agreement in the Global Polypectomy Assessment Tool (GPAT) and SMSA scores for each category of assessor.

	GPAT		SMSA score
	ICC (95 %CI)	Agreement	ICC (95 %CI)	Agreement
Gastroenterologist	0.52 (0.30–0.81)	Moderate	0.46 (0.24–0.79)	Poor
Consultant	0.51 (0.28–0.81)	Moderate	0.44 (0.20–0.78)	Poor
Trainee	0.60 (0.29–0.87)	Moderate	0.57 (0.21–0.87)	Moderate
Nongastroenterologist	0.27 (0.05–0.64)	Poor	0.17 (0.00– 0.56)	Poor
Medical student	0.32 (0.01–0.72)	Poor	0.26 (0.06–0.71)	Poor
Consultant surgeon	0.47 (0.11–0.86)	Poor	0.37 (0.35–0.81)	Poor

ICC, intraclass correlation coefficient; SMSA, size, morphology, site, and access.

Discussion

Colonoscopic polypectomy practice is known to depend significantly on the operator with wide variation in rates of incomplete resection [3] [4] [5] [6] [7] [8]. The ESGE Curriculum for Training in endoscopic mucosal resection in the colon [17] fully deconstructed the technical approach to polypectomy of nonpedunculated polyps ≥ 10 mm. This study demonstrates that the GPAT, an assessment tool based on the curriculum, allows a standardized description of the quality of polypectomy, requires minimal training, correlates with overall endoscopist subjective impression, and is consistent among raters with polypectomy experience.

Standardized assessment tools are critical for procedural specialties and are widespread in surgery [27]. DOPyS is currently the most widely known tool for evaluating the quality of endoscopic colorectal polypectomy. Similarly derived from expert consensus and validation, the DOPyS has been widely implemented in the UK accreditation scheme for colonoscopy [15] and has been used to demonstrate improvement in competency after simulation training in polypectomy [28]. The cold snare polypectomy assessment tool (CSPAT) is another tool to assess polypectomy quality, focusing on cold snare resection of polyps of < 1 cm [29]. To the authors’ knowledge, neither of these tools has been linked to polypectomy outcomes. A summary of the existing tools, with their advantages and disadvantages, is given in Appendix 5 s and Table 5 s.

The GPAT is a modern online self-calculating polypectomy-specific competency framework with seven domains, which follows a sequential deconstructed approach to the resection of any colorectal polyp. Its advantages over the existing tools include the fact that each item contains statements detailing best versus poor practice (supported by videos), which are scored on a 5-point Likert scale. The GPAT is applicable to live and video-based assessment and has an associated logbook that allows tracking of progress over time. It includes an SMSA score calculator to establish the difficulty of the polypectomy, but the SMSA score does not influence the GPAT score. The tool is intuitive to use with minimal training (only a short introductory video).

Whilst the GPAT is currently not linked to polypectomy provider outcomes, important observations from this study suggest it reliably determines the quality of polypectomy. GPAT scoring correlates strongly with the subjective assessor impression suggesting internal validity (with GPAT adding per-polypectomy domain scoring and the potential for deconstructed feedback). Furthermore, among consultant and trainee gastroenterologists (groups who will commonly use GPAT), there was moderate IOA (similar to other established scores, such as SMSA (Appendix 6 s) and BBPS [30]). Per-domain (stage of technique) of the score, there was a strong positive correlation between the reference GPAT score assigned to each video and the assessors’ responses. Finally, in the groups without any experience in endoscopy and polypectomy, the IOA was poor, indicating that some experience with colonoscopic polypectomy is required to use the score, further adding to the validity of the score.

A major advantage of the GPAT over existing tools is its applicability to video assessment: feedback on polypectomy practice can be given separately from the often busy clinical practice hours using videos recorded during the procedure; this approach is backed by a growing body of literature [31]. Video assessment has other advantages including facilitating educational research and helping to eliminate the Hawthorne and Halo effects on procedural assessment [20] [28] [31] [32].

The GPAT also enables structured feedback on a polypectomy, an approach grounded in educational theory [33] [34] [35]. It was specifically developed as a score card to provide structured feedback per domain (e. g. injection technique, snare placement technique, etc.). After each assessment, a trainee can see which domains scored well and which need improvement, both for a single polypectomy and over multiple procedures. This specific and targeted feedback, in combination with the descriptive text and explainer videos, allows rapid skills acquisition.

The next steps for the GPAT include considering polypectomy difficulty in the calculation of the GPAT score itself. For example, a subsequent version of the GPAT could include a mathematical weighting to the final score depending on the SMSA score (e. g. 0.25 for SMSA 2; 0.5 for SMSA 3; 0.75 for SMSA 4; and 1 for polyps with any SMSA+ criterion). This would be a natural extension of including the SMSA score to assess difficulty and is used in many fields of academic assessment. This approach could finally allow the comparison of polypectomy technique whilst controlling for the most significant confounders.

This study demonstrates credible initial validation of the GPAT. Limitations include the small study population (19 assessors completed the survey) and low number of videos (9 videos of < 10 minutes were included). These two factors could have introduced bias into the study. Future studies should include more and longer videos, with a variety of endoscopist performers and assessors at different stages of their training. Based on qualitative feedback from assessors, future iterations of the GPAT might include an option to click on “nonapplicable” for certain criteria. The GPAT could also be broadened to include nonvideo observations with endoscopic nontechnical skills (ENTS)-type parameters [36]. These could be presented separately to allow video and live assessment.

In addition, although the overall (mean) GPAT score, when considering all polyps combined, did not show a statistically significant difference between the reference GPAT score and the scores of the other assessors, some polyps showed statistically significant differences between assessors within specific domains; however, there was no discernable pattern that specific domains were always significantly variable between raters. This issue could also be due to the low number of videos and/or raters. One approach to address this issue might be to provide more clarity at the level of the individual statements about exactly what constitutes very poor versus very good practice, using more narrated deconstructed videos.

In conclusion, a novel freely accessible web-based assessment tool for colorectal polypectomy (GPAT) has been demonstrated to deliver standardized scoring of polypectomy competency after only a 3-minute training video for consultant and trainee gastroenterologists. Pending larger validation studies, the GPAT may allow standardized assessment of polypectomy competency, feedback on areas of poor performance, demonstration of improvement over time, assessment of more difficult cases, and a method to accredit endoscopists in different levels of polypectomy.

Conflict of interest

R. Bisschops has received speaker’s fees and been on the advisory boards of Pentax, Medtronic, and Fujifilm, and has been on the advisory boards of Cook Medical, Boston Scientific, and Olympus; his department has received research grants and organisational support for events from Pentax and Medtronic, plus organisational support for events from Erbe, Ovesco, and Olympus (all within the last 3 years). D.J. Tate has received consultancy fees from Olympus Medical EMEA (2019 to present) and Fujifilm (2021 to present), and educational grants from Pentax, Olympus, Fujifilm, Boston Scientific, Prion Medical, Ovesco, Medtronic, CREO Medical, and Cook Medical (2021 to present). H. Thorlacious is a co-founder and owner of CarpoNovum (2002 to present). J. Anderson, M.E. Argenziano, P. Bhandari, I. Boškoski, M.J. Bourke, M. Bugajski, L. Debels, A. de Crem, L. Desomer, L. Fuccio, S.J. Heitman, H. Kashida, R.R.T. Lee, I. Lyutakov, L. Rivero-Sánchez, C. Schoonjans, S. Smeets, T. Tham, and S. Thomas Gibson declare that they have no conflicts of interest.

Acknowledgments

The authors would like to thank the experts who contributed to the original Delphi consensus.

^‡ Joint first authors

• References

• 1 Winawer SJ, Zauber AG, Ho MN. et al. The National Polyp Study Workgroup. Prevention of colorectal cancer by colonoscopic polypectomy. NEJM 1993; 329: 1977-1981
• 2 Zauber AG, Winawer SJ, O'Brien MJ. et al. Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. NEJM 2012; 366: 687-696
• 3 Sidhu M, Forbes N, Tate DJ. et al. A randomized controlled trial of cold snare polypectomy technique: technique matters more than snare wire diameter. Am J Gastroenterol 2022; 117: 100
• 4 Arimoto J, Chiba H, Higurashi T. et al. Risk factors for incomplete polyp resection after cold snare polypectomy. Int J Colorectal Dis 2019; 34: 1563-1569
• 5 Motchum L, Djinbachian R, Rahme E. et al. Incomplete resection rates of 4- to 20-mm non-pedunculated colorectal polyps when using wide-field cold snare resection with routine submucosal injection. Endosc Int Open 2023; 11: E480-E489
• 6 Liu W, Gong J, Gu L. The efficacy and safety of cold snare versus hot snare polypectomy for endoscopic removal of small colorectal polyps: a systematic review and meta-analysis of randomized controlled trials. Int J Colorectal Dis 2023; 38: 136
• 7 von Renteln D, Djinbachian R, Benard F. et al. Incomplete resection of 4–20 mm colorectal polyps when using cold snare and associated factors. Endoscopy 2023; 55: 929-937
• 8 Pohl H, Srivastava A, Bensen SP. et al. Incomplete polyp resection during colonoscopy-results of the complete adenoma resection (CARE) study. Gastroenterology 2013; 144: 74-80.e1
• 9 Sung H, Ferlay J, Siegel RL. et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021; 71: 209-249
• 10 Adler J, Toy D, Anderson JC. et al. Metachronous neoplasias arise in a higher proportion of colon segments from which large polyps were previously removed, and can be used to estimate incomplete resection of 10–20 mm colorectal polyps. Clin Gastroenterol Hepatol 2019; 17: 2277-2284
• 11 Djinbachian R, Iratni R, Durand M. et al. Rates of incomplete resection of 1- to 20-mm colorectal polyps: a systematic review and meta-analysis. Gastroenterology 2020; 159: 904-914 e12
• 12 Tollivoro TA, Jensen CD, Marks AR. et al. Index colonoscopy-related risk factors for postcolonoscopy colorectal cancers. Gastrointest Endosc 2019; 89: 168-176.e3
• 13 Samadder NJ, Curtin K, Tuohy TM. et al. Characteristics of missed or interval colorectal cancer and patient survival: a population-based study. Gastroenterology 2014; 146: 950-960
• 14 Anderson R, Burr NE, Valori R. Causes of post-colonoscopy colorectal cancers based on world endoscopy organization system of analysis. Gastroenterology 2020; 158: 1287-1299.e2
• 15 Gupta S, Anderson J, Bhandari P. et al. Development and validation of a novel method for assessing competency in polypectomy: direct observation of polypectomy skills. Gastrointest Endosc 2011; 73: 1232-1239.e2
• 16 Gupta S, Bassett P, Man R. et al. Validation of a novel method for assessing competency in polypectomy. Gastrointest Endosc 2012; 75: 568-575
• 17 Tate DA, Argenziano ME, Anderson J. et al. Curriculum for training in endoscopic mucosal resection in the colon: European Society of Gastrointestinal Endoscopy (ESGE) Position Statement. Endoscopy 2023; 55: 645-679
• 18 Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Inf Manag 2004; 42: 15-29
• 19 GPAT – The global polypectomy assessment tool. Accessed: 8 January 2025. www.gieqs.com/gpat
• 20 GPAT best practice video. Accessed: 8 January 2025. www.gieqs.com/videos/1215
• 21 Gupta S, Miskovic D, Bhandari P. et al. A novel method for determining the difficulty of colonoscopic polypectomy. Frontline Gastroenterol 2013; 4: 244-248
• 22 Sidhu M, Tate DJ, Desomer L. et al. Mo1662. SMSA-EMR score is a novel endoscopic risk assessment tool for predicting critical endoscopic mucosal resection outcomes. Gastrointest Endosc 2018; 87: AB467-AB468
• 23 GPAT showcase. Accessed: 8 January 2025. https://vimeo.com/showcase/10417029
• 24 Introduction to the study. Accessed: 8 January 2025. https://vimeo.com/562536661
• 25 Posit team. RStudio: Integrated Development Environment for R (2022). Accessed: 8 January 2025. https://posit.co
• 26 Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155-163
• 27 Miskovic D, Wyles SM, Carter F. et al. Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program. Surg Endosc 2011; 25: 1136-1142
• 28 Patel RV, Barsuk JH, Cohen ER. et al. Simulation-based training improves polypectomy skills among practicing endoscopists. Endosc Int Open 2021; 9: E1633-E1639
• 29 Patel SG, Duloy A, Kaltenbach T. et al. Development and validation of a video-based cold snare polypectomy assessment tool (with videos). Gastrointest Endosc 2019; 89: 1222-1230.e2
• 30 Crapé L, Debels L, Schoonjans C. et al. Development and validation of a novel score for the completeness of caecal intubation - the CCIS (completeness of caecal intubation score). Gastrointest Endosc 2022; 95: AB85-AB86
• 31 Jeyalingam T, Walsh CM. Video-based assessments: a promising step in improving polypectomy competency. Gastrointest Endosc 2019; 89: 1231-1233
• 32 Scaffidi MA, Grover SC, Carnahan H. et al. A prospective comparison of live and video-based assessments of colonoscopy performance. Gastrointest Endosc 2018; 87: 766-775
• 33 Patel K, Faiz O, Rutter M. et al. The impact of the introduction of formalised polypectomy assessment on training in the UK. Frontline Gastroenterol 2017; 8: 104-109
• 34 Anderson J, Lockett M. Training in therapeutic endoscopy: meeting present and future challenges. Frontline Gastroenterol 2019; 10: 135-140
• 35 Samuel A, Konopasky A, Schuwirth LWT. et al. Five principles for using educational theory: strategies for advancing health professions education research. Acad Med 2020; 95: 518-522
• 36 Hitchins CR, Metzner M, Edworthy J. et al. Non-technical skills and gastrointestinal endoscopy: a review of the literature. Frontline Gastroenterol 2018; 9: 129-134

Publikationsverlauf

Artikel online veröffentlicht:
14. März 2025