Measuring Quality at the Surgeon Level

Abstract

Patients expect high-quality surgical care and increasingly are looking for ways to assess the quality of the surgeon they are seeing, but quality measurement is often more complicated than one might expect. Measurement of individual surgeon quality in a manner that allows for comparison among surgeons is particularly difficult. While the concept of measuring individual surgeon quality has a long history, technology now allows for new and innovative ways to measure and achieve surgical excellence. However, some recent efforts to make surgeon-level quality data publicly available have highlighted the challenges of this work. Through this chapter, the reader will be introduced to a brief history of surgical quality measurement, learn about the current state of quality measurement, and get a glimpse into what the future holds.

Keywords

surgical quality - technical skills - video-based assessment - surgical quality

Publication History

Article published online:
24 March 2023

• References

• 1 Chun J, Bafford AC. History and background of quality measurement. Clin Colon Rectal Surg 2014; 27 (01) 5-9
• 2 Roberts JS, Coale JG, Redman RR. A history of the Joint Commission on Accreditation of Hospitals. [published correction appears in JAMA 1987 Nov 20;258(19):2698] JAMA 1987; 258 (07) 936-940
• 3 Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q 2005; 83 (04) 691-729
• 4 Hannan EL, Cozzens K, King III SB, Walford G, Shah NR. The New York State cardiac registries: history, contributions, limitations, and lessons for future efforts to assess and publicly report healthcare outcomes. J Am Coll Cardiol 2012; 59 (25) 2309-2316
• 5 Ferraris VA, Ferraris SP, Wehner PS, Setser ER. The dangers of gathering data: surgeon-specific outcomes revisited. Int J Angiol 2011; 20 (04) 223-228
• 6 Shih T, Cole AI, Al-Attar PM. et al. Reliability of surgeon-specific reporting of complications after colectomy. Ann Surg 2015; 261 (05) 920-925
• 7 Hall BL, Huffman KM, Hamilton BH. et al. Profiling individual surgeon performance using information from a high-quality clinical registry: opportunities and limitations. J Am Coll Surg 2015; 221 (05) 901-913
• 8 Quinn CM, Bilimoria KY, Chung JW, Ko CY, Cohen ME, Stulberg JJ. Creating individual surgeon performance assessments in a statewide hospital surgical quality improvement collaborative. J Am Coll Surg 2018; 227 (03) 303-312.e3
• 9 Khuri SF, Daley J, Henderson W. et al. Relation of surgical volume to outcome in eight common operations: results from the VA National Surgical Quality Improvement Program. Ann Surg 1999; 230 (03) 414-429 , discussion 429–432
• 10 Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. Surgeon volume and operative mortality in the United States. N Engl J Med 2003; 349 (22) 2117-2127
• 11 Porter GA, Soskolne CL, Yakimets WW, Newman SC. Surgeon-related factors and outcome in rectal cancer. Ann Surg 1998; 227 (02) 157-167
• 12 Parry JM, Collins S, Mathers J, Scott NA, Woodman CB. Influence of volume of work on the outcome of treatment for patients with colorectal cancer. Br J Surg 1999; 86 (04) 475-481
• 13 Engelhardt KE, Bilimoria KY, Stulberg JJ. Surgeon scorecards: accurate or not?. Adv Surg 2018; 52 (01) 1-14
• 14 Friedberg MW, Pronovost PJ, Shahian DM. et al. A methodological critique of the ProPublica surgeon scorecard. Rand Health Q 2016; 5 (04) 1
• 15 Ban KA, Cohen ME, Ko CY. et al. Evaluation of the ProPublica surgeon scorecard “adjusted complication rate” measure specifications. Ann Surg 2016; 264 (04) 566-574
• 16 Jaffe TA, Hasday SJ, Dimick JB. Power outage-inadequate surgeon performance measures leave patients in the dark. JAMA Surg 2016; 151 (07) 599-600
• 17 Regenbogen SE, Read TE, Roberts PL, Marcello PW, Schoetz DJ, Ricciardi R. Urinary tract infection after colon and rectal resections: more common than predicted by risk-adjustment models. J Am Coll Surg 2011; 213 (06) 784-792
• 18 Burack JH, Impellizzeri P, Homel P, Cunningham Jr JN. Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. Ann Thorac Surg 1999; 68 (04) 1195-1200 , discussion 1201–1202
• 19 Burns EM, Pettengell C, Athanasiou T, Darzi A. Understanding the strengths and weaknesses of public reporting of surgeon-specific outcome data. Health Aff (Millwood) 2016; 35 (03) 415-421
• 20 Birkmeyer JD, Finks JF, O'Reilly A. et al; Michigan Bariatric Surgery Collaborative. Surgical skill and complication rates after bariatric surgery. N Engl J Med 2013; 369 (15) 1434-1442
• 21 Stulberg JJ, Huang R, Kreutzer L. et al. Association between surgeon technical skills and patient outcomes. [published correction appears in JAMA Surg. 2020 Oct 1;155(10):1002] [published correction appears in JAMA Surg. 2021 Jul 1;156(7):694] JAMA Surg 2020; 155 (10) 960-968
• 22 Mackenzie H, Ni M, Miskovic D. et al. Clinical validity of consultant technical skills assessment in the English National Training Programme for laparoscopic colorectal surgery. Br J Surg 2015; 102 (08) 991-997
• 23 Bilgic E, Valanci-Aroesty S, Fried GM. Video assessment of surgeons and surgery. Adv Surg 2020; 54: 205-214
• 24 Merkow RP, Massarweh NN. Looking beyond perioperative morbidity and mortality as measures of surgical quality. Ann Surg 2022; 275 (02) e281-e283
• 25 Fecso AB, Szasz P, Kerezov G, Grantcharov TP. The effect of technical performance on patient outcomes in surgery: a systematic review. Ann Surg 2017; 265 (03) 492-501
• 26 Martin JA, Regehr G, Reznick R. et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84 (02) 273-278
• 27 Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. N Engl J Med 2006; 355 (25) 2664-2669
• 28 Brajcich BC, Stulberg JJ, Palis BE. et al. Association between surgical technical skill and long-term survival for colon cancer. JAMA Oncol 2021; 7 (01) 127-129
• 29 Wood TC, Raison N, Haldar S. et al. Training tools for nontechnical skills for surgeons—a systematic review. J Surg Educ 2017; 74 (04) 548-578
• 30 Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery 2003; 133 (06) 614-621
• 31 Baker DP, Beaubien JM, Holtzman AK. DoD Medical Team Training Programs: An Independent Case Study Analysis. Washington, DC: American Institutes for Research; 2003
• 32 Sinyard RD, Rentas CM, Gunn EG. et al. Managing a team in the operating room: The science of teamwork and non-technical skills for surgeons. Curr Probl Surg 2022; 59 (07) 101172
• 33 Noonan M, Olaussen A, Mathew J, Mitra B, Smit V, Fitzgerald M. What is the clinical evidence supporting trauma team training (TTT): a systematic review and meta-analysis. Medicina (Kaunas) 2019; 55 (09) 551
• 34 Tan SB, Pena G, Altree M, Maddern GJ. Multidisciplinary team simulation for the operating theatre: a review of the literature. ANZ J Surg 2014; 84 (7-8): 515-522
• 35 Forse RA, Bramble JD, McQuillan R. Team training can improve operating room performance. Surgery 2011; 150 (04) 771-778
• 36 Chheang V, Fischer V, Buggenhagen H. et al. Toward interprofessional team training for surgeons and anesthesiologists using virtual reality. Int J CARS 2020; 15 (12) 2109-2118
• 37 Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons' non-technical skills. Med Educ 2006; 40 (11) 1098-1104
• 38 Pradarelli JC, Gupta A, Lipsitz S. et al. Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA. Br J Surg 2020; 107 (09) 1137-1144
• 39 Neily J, Mills PD, Young-Xu Y. et al. Association between implementation of a medical team training program and surgical mortality. JAMA 2010; 304 (15) 1693-1700
• 40 Abahuje E, Johnson J, Halverson A, Stulberg JJ. Intraoperative Assessment of Non-Technical Skills for Surgeons (NOTSS) and qualitative description of their effects on intraoperative performance. J Surg Educ 2022; 79 (05) 1237-1245
• 41 Louridas M, Sachdeva AK, Yuen A, Blair P, MacRae H. Coaching in surgical education: a systematic review. Ann Surg 2022; 275 (01) 80-84
• 42 Pradarelli JC, Hu YY, Dimick JB, Greenberg CC. The value of surgical coaching beyond training. Adv Surg 2020; 54: 31-47
• 43 Kreutzer L, Hu YY, Stulberg J, Greenberg CC, Bilimoria KY, Johnson JK. Formative evaluation of a peer video-based coaching initiative. J Surg Res 2021; 257: 169-177
• 44 Rivard SJ, Varlamos C, Hibbard CE. et al. A national qualitative study of surgical coaching: opportunities and barriers for colorectal surgeons. Surgery 2022; 172 (02) 546-551