A Novel Microsurgical Suturing Pilot Course for Ophthalmology Residents Based on Kern's Model for Curriculum Development

Abstract

Background Sutureless ophthalmic procedures are becoming more commonplace, reducing opportunities for ophthalmology residents to learn microsurgical suturing techniques. There is no standard curriculum in place to address this gap in clinical training among ophthalmology residency programs.

Objective The aim of this study was to design, implement, and evaluate a preliminary microsurgical suturing curriculum for ophthalmology residents using Kern's six-step approach for curriculum development as a guideline, and the principles of distributed practice and guided, self-directed practice.

Methods We designed a faculty-led teaching session on fundamental microsurgical suturing techniques for all 15 ophthalmology residents from Yale University over one academic year. Suturing skills were evaluated, followed by a guided teaching session, 30 days of self-directed practice time, and a re-evaluation of skills. The residents were evaluated through a written knowledge assessment and practical skills assessment. The residents also evaluated their skill level before and after the teaching session and practice period through written Likert-scale surveys. Data were evaluated in Excel using descriptive statistics and the paired t-test.

Results After the session, postgraduate year 2 (PGY-2) residents felt more confident in recognition and use of surgical instruments (p < 0.01). PGY-3 residents felt less confident in their knowledge of microsurgical suturing after the session (p = 0.02). PGY-4 residents felt they were better able to identify different suture types after the session (p = 0.02). All residents improved on the written knowledge assessment (p < 0.001) and in all categories of the practical skills assessment (p < 0.001).

Conclusions Implementation of a faculty-led microsurgical suturing training session, followed by 1-month of practice time, significantly improved residents' knowledge and practical application of various microsurgical suturing techniques that are necessary for performing common ophthalmic procedures.

Keywords

microsurgical suturing - microsurgical training - surgical skills assessment - resident training - Kern's model - distributed practice

In ophthalmology, microsurgical suturing is essential for procedures such as open globe repair, corneal transplant suturing, and strabismus surgery.[1] Yet restrictions on resident training hours and advances in sutureless ophthalmic procedures may limit opportunities for residents to practice microsurgical suturing.[2] [3] [4] [5] While the Accreditation Council for Graduate Medical Education requires residency training programs to have a microsurgical skills development resource (wet laboratory or simulators), research shows that residents may find value in (additional) microsurgical skills training courses.[2] [6]

The purpose of our study was to design, implement, and evaluate a faculty-guided, self-directed, distributed (e.g., practice interspersed with periods of rest) microsurgical skills pilot course for ophthalmology residents based on Kern's six-step approach to curriculum development. We hoped to enhance microsurgical suturing skills acquisition and retention among ophthalmology residents. Kern's model for curriculum development was selected as its purpose is to “provide a practical, theoretically sound approach to developing, implementing, evaluating and continually improving educational experiences in medicine.”[7] Kern's model was initially sculpted by educators who have developed over 100 medical curricula for topics ranging from clinical reasoning to surgical skills assessment.[7] The model utilizes a six-step approach ([Table 1]) that emphasizes flexibility based on evaluation results, changes in trainee population, and changes in material (or skills) requiring mastery.[8] No study to date has evaluated the utility of Kern's six-step approach to acquisition and retention of microsurgical suturing skills.

Table 1 Overview of Kern's six-step approach to curriculum development and how the steps apply to our training program

Step	Definition	Application of Kern's approach to our pilot curriculum
1. Problem identification	• Identify and characterize the healthcare problem that will be addressed by the curriculum, how the problem is currently being addressed and how it should ideally be addressed	• Healthcare problem: Based on statements from published literature and direct input from ophthalmology attendings at Yale University, we have determined that there is a need for enhanced microsurgical suturing training for ophthalmology residents  • How the problem is being addressed: Annual wet laboratory-based training for ophthalmology residents in addition to direct surgical experience through the apprenticeship model  • How the problem should ideally be addressed: In addition to our current program, we should implement a microsurgical suturing course that provides faculty guidance to residents, while allowing for individual practice time, and evaluation and feedback
2. Targeted needs assessment	• Assess the needs of one's targeted group of learners and their medical institution/learning environment	• Needs of learners and learning environment: Ophthalmology faculty members and the residency program director were interviewed concerning opportunities for residents to practice microsurgical suturing skills. Published research has also noted a reduced chance for ophthalmology residents to practice microsurgical suturing skills in the current climate of reduced training hours and advances in sutureless procedures
3. Goals and objectives	• Target the curriculum to address the needs of learners  • An end toward which an effort is directed	• Primary goals and objectives: To provide residents with an opportunity to assess their baseline confidence in microsurgical suturing, and knowledge of sutures and surgical instruments. To evaluate the effectiveness of a faculty-guided microsurgical suturing training course for the enhancement of microsurgical suturing skills, using results from pre- and post-test practical skills assessments To evaluate the effectiveness of a distributed practical model within a microsurgical suturing training course
4. Educational strategies	• Chose curriculum content and educational methods that will most likely achieve the educational objectives	• Curriculum content: Our curriculum focuses on components that are essential to enhancing microsurgical suturing skills. Specifically, we focus on knowledge of sutures and suture material and practical application of this knowledge to basic microsurgical suturing technique  • Educational methods: Based on prior literature showing the benefits of distributed practice, we decided to implement a faculty-led teaching course followed by a 30-day period of individual study and practice. We concluded by providing an additional practice and evaluation session at the end of the 30-day period, followed by individual and summative feedback
5. Implementation	• Implement the educational intervention and its evaluation  • Components: Obtain political support, identify and procure resources, identify and address barriers to implementation, introduce the curriculum, administer the curriculum and refine the curriculum over successive cycles	• Obtain political support: Not applicable to this study  • Identify and procure resources: Resources for the curriculum (latex gloves, plastic platforms, suture material and needles, surgical instruments) were made available through the Yale Department of Ophthalmology and Visual Science  • Identify and address barrier to implementation: There were no specific barriers to implementation  • Introduce the curriculum: Residents were notified that a pilot curriculum was to be conducted within their current training course  • Administer the curriculum: The course was administered over a 1-month period  • Refine the curriculum over successive cycles: We plan to revise the pilot curriculum based on evaluation results and eventually formally integrate the course within the resident training program. We also plan to create an online module that can be applied to any residency program requiring trainees to master basic microsurgical suturing knowledge and skills
6. Evaluation and feedback	• Assess the performance of individuals and the curriculum. The purpose of the evaluation may be formative (provide ongoing feedback) or summative (provide a final grade or evaluation)	• Evaluation of individuals: Each resident was evaluated independently based on their scores on both the written knowledge assessment and the practical skills assessment  • Evaluation of the curriculum: The curriculum as a whole was evaluated by de-identifying and pooling data from all resident classes and assessing results of survey data, written knowledge assessments and practical skills assessments both before and after the administration of the course

We emphasized principles of faculty-guided, self-directed practice, and distributed practice within the framework of Kern's six-step approach. Through integrating these principles into a curriculum model, we propose a novel method for implementing and evaluating a microsurgical suturing skills program that is generalizable to all ophthalmology residents, throughout all stages of training.

Methods

Curriculum Design and Test Population

The six steps of Kern's model are as follows: (1) Problem identification and general needs assessment; (2) targeted needs assessment; (3) goals and objectives; (4) educational strategies; (5) implementation; and (6) feedback and evaluation.[8] For specific definitions of each step and how we apply each step to the design of our curriculum, see [Table 1].

A total of 15 ophthalmology residents (5 postgraduate year 2 [PGY-2], 5 PGY-3, and 5 PGY-4) participated in the pilot curriculum. All residents took part in the survey, written knowledge assessment, and practical skills assessment. The pilot curriculum was run from March through August of 2017 at the Yale University Department of Ophthalmology and Visual Science. Of note, “pre-test” refers to any time period before the teaching session and 30-day practice period, whereas “post-test” refers to the time period after the teaching session and 30-day practice period.

Pre-Test Survey and Pre-Test Written Knowledge Assessment

Prior to the pre-test practical skills assessment and faculty-guided teaching session, residents were given the following: a pre-test Likert-scale survey (1 through 5 with 5 listed as “strongly agree” and 1 listed as “strongly disagree”) and a pre-test written knowledge assessment. The survey was designed as a self-assessment for residents to evaluate their practical abilities and knowledge of various components of microsurgical suturing before the teaching session and practice period. Residents' opinions on the utility of practicing suturing were also assessed.

The pre-test written knowledge assessment was designed to evaluate residents' knowledge of needles, suture material, and their clinical applications. Examples of both the pre-test survey and pre-written knowledge assessment are provided as [supplemental materials].

Pre-Test Practical Skills Assessment and Faculty-Guided Teaching Session

Two ophthalmology attendings (CCT and JHC), including the residency program director (JHC), scheduled a 1-hour faculty-guided teaching session for each class of ophthalmology residents. Before the teaching session, residents' microsurgical suturing skills were evaluated through a pre-test practical skills assessment. Components of the practical skills assessment included the following: (a) loading the needle; (b) 1–1-1–1 square knot; (c) 3–1-1 forehand; d. 3–1-1 backhand; (e)1–1-1 slip knot; (f) closure running suture; (g) closure running locking suture. Residents used taut latex, a low-fidelity model (e.g., does not mimic real tissue), to practice their sutures as it is a cost-effective method and research has shown that “surgical skills training on low-fidelity…models appears to be as effective as high-fidelity model training for the acquisition of technical skill among novice surgeons.”[9]

During the pre-test practical skills assessment, residents were evaluated independently by the ophthalmology attendings based on the Global Rating Scale of Operative Performance ([Table 2]).[10] Of note, economy of movement, confidence of movement, respect for tissue, and overall performance were graded as separate categories. Knowledge and precision of technique were based on each of the listed components of the practical skills assessment (e.g., skill in loading the needle, completing a 1–1-1–1 square knot). After the pre-test practical assessment, residents participated in the faculty-guided teaching session that consisted of a practice period with individual guidance from each ophthalmology faculty member. Residents were then given a 30-day period of self-directed practice where they were encouraged to utilize faculty instructions from the practice session and online resources from the University of Iowa EyeRounds Web site: (http://webeye.ophth.uiowa.edu/eyeforum/tutorials/Iowa-OWL/suture/suturing-and-tying-techniques.htm).

Table 2 Template for faculty scoring of practical microsurgical suturing skills using the global rating scale of operative performance

Score	1	2	3	4	5
Economy of movement (time and motion)	Many unnecessary moves		Efficient time/motion but some unnecessary moves		Economy of movement and maximum efficiency
Confidence of movement (instrument handling) and use of nondominant hand	Repeatedly makes tentative or awkward moves with instruments		Competent use of instruments although occasionally stiff or awkward		Fluid moves with instruments with no awkwardness
Respect for tissue	Frequently used unnecessary force on tissues or caused damage by inappropriate use of instruments		Careful handling of tissue but occasionally caused inadvertent damage		Consistently handled tissues appropriately with minimal damage
Knowledge and precision of technique	Imprecise, wrong technique. Required specific instruction at most steps		Careful technique with occasional errors. Knew all important steps		Fluent, secure and correct technique in all stages of suturing. Familiar with all steps
Overall performance	Unable to perform independently		Competent. Could perform with minimal assistance		Superior, able to perform independently with confidence

Post-Test Survey and Post-Test Written Knowledge Assessment

After the self-directed practice period, residents returned to the ophthalmology department for a post-test survey and post-test written knowledge assessment. The post-test survey gave residents the opportunity to re-evaluate their practical skills and knowledge of microsurgical suturing. The post-test written knowledge assessment gave residents the opportunity to demonstrate their retention of specific details of microsurgical suturing instruments and suture materials learned from the faculty-guided teaching session and 30-day practice period.

Post-Test Practical Skills Assessment and Feedback

Following the post-test survey and post-test written knowledge assessment, residents were again evaluated on each component of the practical skills assessment. The two ophthalmology attendings who initially evaluated their pre-test microsurgical skills graded residents' performance using the same Global Rating Scale of Operative Performance. The attendings then gave the residents another practice period with individual feedback on how the residents performed relative to their initial practical skills assessment and on how they can continue to improve. All feedback given was formative, or designed to be ongoing throughout the remainder of the residents' training period. A final competency assessment was therefore not included in the evaluation.

The study was determined to be exempt by the Yale University Institutional Review Board.

Statistical Analysis

All data were collected by one investigator (ACR) and entered into Excel Version 15.21.1. Scoring of written knowledge assessments was performed by one investigator (ACR) using an answer key provided by the residency program director. Practical skills assessments were graded independently by each faculty member. Descriptive statistics and paired t-tests were used as appropriate. Cohen's kappa statistic was calculated to evaluate inter-rater reliability between the two attendings on the practical skills assessments.

Results

Resident Self-Assessment Survey

Results are reported as mean ± standard deviation with results from pre-test surveys reported first followed by results from post-test surveys. PGY-2 residents felt significantly more confident on the following items after the course: “I know the names of the instruments” (2 ± 0.0 vs. 3.4 ± 0.5; p < 0.01); “I know the proper way to hold instruments” (2.8 ± 0.4 vs. 4 ± 0.0; p < 0.01); and “I know the proper way to handle instruments” (2.6 ± 0.5 vs. 3.8 ± 0.4; p < 0.01). PGY-3 residents did not feel significantly more confident on any of the survey items. After the course, they felt significantly less confident on the item, “I know how to suture” (3.8 ± 0.8 vs. 3.0 ± 1.0; p < 0.05). PGY-4 residents felt significantly more confident on the item, “I know the different types of sutures” after the course (3.4 ± 0.5 vs. 4.2 ± 0.4; p < 0.05).

When all years of training were combined, residents felt more confident in the following areas after the course: “I know the names of the instruments” (2.5 ± 0.7 vs. 3.2 ± 0.8; p < 0.01); “I know the proper way to hold instruments” (2.9 ± 0.5 vs. 3.7 ± 0.8; p < 0.01); and “I know the different types of needles” (3.0 ± 0.6 vs. 3.6 ± 0.5; p < 0.05). Residents overall felt significantly less confident on the following items after the course: “I feel like a suturing teaching session will help me better understand how to suture,” “I feel like I have a better understanding of how to suture” (4.8 ± 0.4 vs. 4.2 ± 0.4; p < 0.001) and “I feel that practicing suturing will make me better” (4.9 ± 0.3 vs. 4.5 ± 0.5; p < 0.05) ([Fig. 1)].

Fig. 1 Survey results from before and after the microsurgical suturing course and 30-day practice period for postgraduate year 2 (PGY-2) (A), PGY-3 (B), PGY-4 (C), and all resident years combined (D). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired t-test results, as seen on the figure.

Resident Written Knowledge Assessment

Results are reported as median percent correct out of 12 items, with corresponding first and third interquartile range. Median percent correct for the pre-test assessment is listed first, followed by the median percent correct for the post-test assessment. All resident classes scored significantly higher on their written knowledge assessment after the teaching session and practice period. PGY-2 (42.3% [34.6, 42.3] vs. 65.4% [65.4, 69.2]; p <0.05); PGY-3 (50.0% [46.1, 53.9] vs. 65.4% [61.5, 73.1]; p < 0.05); and PGY-4 (57.7% [50.0, 65.4] vs. 80.1% [76.9, 84.6]; p <0.01). When PGY-3 and PGY-4 years were combined, the median knowledge assessment test score was 51.92% (47.1, 63.5) versus 75.0% (62.5,79.8); p < 0.001 ([Fig. 2)].

Fig. 2 Results of the pre- and post-intervention written knowledge assessments for individual resident years (A) and postgraduate year 2 (PGY-3) and PGY-4 combined, with PGY-2 for comparison (B). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired t-test results, as seen on the figure.

Faculty-Evaluated Practical Skills Assessment

PGY-2 residents improved in all areas of the practical skills assessment: economy of movement (1.6 ± 0.6 vs. 2.75 ± 0.5; p < 0.001); confidence of movement (1.6 ± 0.5 vs. 2.7 ± 0.5; p < 0.001); respect for tissue (2.4 ± 0.2 vs. 3.0 ± 0.0; p < 0.01); overall performance (1.7 ± 0.5 vs. 3.0 ± 0.3; p < 0.01); and knowledge and precision of technique (1.6 ± 0.2 vs. 2.9 ± 0.2; p < 0.001). See [Table 2] for a description of the grading scale

PGY-3 residents also improved in all areas of the practical skills assessment: economy of movement (2.8 ± 0.6 vs. 3.5 ± 0.4; p < 0.05); confidence of movement (2.5 ± 0.4 vs. 3.6 ± 0.6; p < 0.01); respect for tissue (2.7 ± 0.3 vs. 3.6 ± 0.4; p < 0.001); overall performance (2.8 ± 0.5 vs. 3.6 ± 0.6; p < 0.05); and knowledge and precision of technique (2.6 ± 0.2 vs. 3.7 ± 0.6; p < 0.01).

PGY-4 residents improved in the following areas of the practical skills assessment: economy of movement (3.3 ± 0.4 vs. 4.0 ± 0.4; p < 0.001); confidence of movement (3.5 ± 0.4 vs. 4.3 ± 0.4; p < 0.01); overall performance (3.5 ± 0.3 vs. 4.2 ± 0.4; p < 0.001); and knowledge and precision of technique (3.4 ± 0.3 vs. 4.3 ± 0.3; p < 0.01).

When combined, all residents improved in their practical skills assessment after the course: economy of movement (2.6 ± 0.9 vs. 3.4 ± 0.7; p < 0.001); confidence of movement (2.5 ± 0.9 vs. 3.6 ± 0.8; p < 0.001); respect for tissue (3.0 ± 0.7 vs. 3.6 ± 0.6; p < 0.001); overall performance (2.6 ± 0.8 vs. 3.6 ± 0.6; p < 0.001); and knowledge and precision of technique (2.5 ± 0.8 vs. 3.6 ± 0.7; p < 0.001) ([Fig. 3)].

Fig. 3 Results of pre- and post-intervention practical skills assessments using the Global Rating Scale of Operative Performance for postgraduate year 2 (PGY-2) (A), PGY-3 (B), PGY-4 (C), and all resident years combined (D). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired t-test results, as seen on the figure.

Cohen's kappa ranged from 0.06 (–0.2–0.3) to 0.6 (0.2–0.9) for all components of the practical skills assessment.

Discussion

We applied Kern's six-step approach to curriculum development to a pilot course for basic microsurgical suturing. We integrated distributed practice and self-directed practice, which have been shown to improve acquisition and retention of surgical skills.[11] [12] [13] After the course, residents improved in their written knowledge and practical skills assessments. PGY-2 residents felt more confident in their ability to recognize and use surgical instruments and PGY-4 residents felt more confident in their ability to identify different types of sutures. PGY-3 residents felt less confident in their knowledge of microsurgical suturing. All resident classes felt less confident that practicing suturing will make them better. Our pilot course was useful in improving practical skills and trainee knowledge base. However, resident lack of confidence and/or initial cognitive bias may be important concerns to address.

A study of general surgery residents showed that “junior residents in the middle of clinical training years were most worried about feeling confident enough to perform procedures independently by the end of training.”[14] These feelings of worry may reflect increased levels of operative responsibility, while also possibly having to make decisions regarding (fellowship) training.[14] This may reflect similar attitudes of PGY-3 ophthalmology residents and the transition from PGY-3 to PGY-4 may be a critical stepping point for residents to gain confidence in both their operative skills.

A reduction in trainee confidence that practicing suturing would help improve their skills may be explained by the Dunning–Kruger effect, which is based on the theory that “[novice] individuals do not possess the degree of metacognitive skills necessary for accurate self-assessment.”[15] Prior to the course, residents may have overinflated their ability to improve their microsurgical suturing skills through practice alone. After the course, they may have realized that they also needed to improve on their technical knowledge and may be in need of guidance from faculty. Future research on resident attitudes toward practicing microsurgical suturing on their own versus with a faculty mentor may be useful.

PGY-2 residents seemed to gain the most information in their knowledge of how to handle and identify surgical instruments. This finding suggests that knowledge of handling instruments in the operating room may be a key area to focus on in subsequent courses. PGY-4 residents seemed to benefit most from learning to differentiate different types of sutures, indicating that they may already have a solid grasp on basic suturing techniques and instrument handling and may require more detailed instruction on use of specific suture types. Therefore, the course could be improved through tailored instruction to each resident class. Of note, PGY-3 and PGY-4 residents were combined when assessing pre- and post-curriculum scores on the written knowledge assessment for the purposes of evaluating the overall impact of the course on fundamental microsurgical suturing knowledge among trainees who had at least 1-year of ophthalmology training. Results supported our findings that the course is beneficial even among more senior resident groups.

Limitations include use of self-reported survey results, small sample size of trainees in the pilot program, small number of faculty evaluators, and lack of control for microsurgical suturing training prior to ophthalmology residency. Cohen's kappa statistic (“the extent to which the data collected in the study are correct representations of variables measured”[16]) ranged from 0.06 to 0.6. Any kappa below 0.6 may indicate inadequate agreement among raters.[16] However, given that the highest kappa statistic is 0.6, we cannot definitively state that results are not consistent between the two raters. The wide range of kappa scores may be explained by variation in expectation of resident performance based on PGY level, differences in faculty training techniques, and/or discrepancies in resident performance during the practical skills assessment. Additional faculty evaluators may help mitigate differences in scoring of resident practical skills assessments in future studies.

Lastly, once established as a formal curriculum within our ophthalmology residency program, we hope to transition this curriculum to an online module, which would allow for greater ease of dissemination to other programs that may like to establish their own resident microsurgical suturing skills course.

Conclusion

Implementation of our pilot course enhances trainee practical and knowledge-based skill in basic microsurgical suturing. Integration of our model within residency curricula may help improve acquisition and retention of microsurgical suturing skills. Prior to the course it may be useful to conduct a needs assessment for each resident class to address the most pressing concerns during the faculty teaching session.

Conflict of Interest

A. C. R. reports other from Yale School of Medicine Office of Student Research, during the conduct of the study.

Note

This study was presented with preliminary data as a poster at the 2018 Yale Medical Education Day.

• References

• 1 Macsai M. Ophthalmic Microsurgical Suturing Techniques. Berlin: Springer; 2007
• 2 Ezra DG, Aggarwal R, Michaelides M. et al. Skills acquisition and assessment after a microsurgical skills course for ophthalmology residents. Ophthalmology 2009; 116 (02) 257-262
• 3 Sandford-Smith J. Sutureless cataract surgery: principles and steps. Community Eye Health 2003; 16 (48) 49-53
• 4 Velazquez AJ, Carnahan MA, Kristinsson J, Stinnett S, Grinstaff MW, Kim T. New dendritic adhesives for sutureless ophthalmic surgical procedures: in vitro studies of corneal laceration repair. Arch Ophthalmol 2004; 122 (06) 867-870
• 5 Selber JC, Chang EI, Liu J. et al. Tracking the learning curve in microsurgical skill acquisition. Plast Reconstr Surg 2012; 130 (04) 550e-557e
• 6 The Accreditation Council for Graduate Medical Education, ACGME Program Requirements for Graduate Medical Education in Ophthalmology. 2017: 1-34 . Available at: https://www.acgme.org/Specialties/Documentsand-Resources/pfcatid/13/Ophthalmology
• 7 Kern DE. Introduction. In: Patricia A, Thomas DEK, Mark THughes, Belinda Y.Chen. , eds. Curriculum Development for Medical Education: A Six-Step Approach Third ed. Baltimore: Johns Hopkins University Press; 2016
• 8 Kern DE. Overview: A six-step approach to curriculum development. In: Patricia A, Thomas DEK, Mark THughes, Belinda Y.Chen. , eds. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore: Johns Hopkins University Press; 2016
• 9 Grober ED, Hamstra SJ, Wanzel KR. et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg 2004; 240 (02) 374-381
• 10 Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology 2005; 112 (10) 1655-1660
• 11 Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A. Surgical skill acquisition with self-directed practice using computer-based video training. Am J Surg 2007; 193 (02) 237-242
• 12 Benjamin AS, Tullis J. What makes distributed practice effective?. Cognit Psychol 2010; 61 (03) 228-247
• 13 Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R. Teaching surgical skills: what kind of practice makes perfect? a randomized, controlled trial. Ann Surg 2006; 244 (03) 400-409
• 14 Bucholz EM, Sue GR, Yeo H, Roman SA, Bell Jr RH, Sosa JA. Our trainees' confidence: results from a national survey of 4136 US general surgery residents. Arch Surg 2011; 146 (08) 907-914
• 15 Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol 1999; 77 (06) 1121-1134
• 16 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012; 22 (03) 276-282

Publication History

Received: 08 November 2019

Accepted: 11 August 2020

Article published online:
04 November 2020