Interobserver Variability of the Diagnosis of Scaphoid Proximal Pole Fractures

Abstract

Purpose Fractures of the proximal pole of the scaphoid are prone to adverse outcomes such as nonunion and avascular necrosis. Distinction of scaphoid proximal pole fractures from waist fractures is important for management but it is unclear if the distinction is reliable.

Methods A consecutive series of 29 scaphoid fractures from one tertiary hospital was collected consisting of 5 scaphoid proximal pole and 24 scaphoid waist fractures. Fifty-seven members of the Science of Variation Group (SOVG) were randomized to diagnose fracture location and displacement by using radiographs alone or radiographs and a computed tomography (CT) scan.

Results Observers reviewing radiographs alone and observers reviewing radiographs and CT scans both had substantial agreement on fracture location (κ = 0.82 and κ = 0.80, respectively; p = 0.54). Both groups had only fair agreement on fracture displacement (κ = 0.28 and κ = 0.35, respectively; p = 0.029).

Conclusion Proximal pole fractures are sufficiently distinct from proximal waist fractures that CT does not improve reliability of diagnosis.

Level of Evidence Level IV interobserver reliability case-control study.

Keywords

scaphoid fracture - radiograph - CT scan - interobserver study

Fractures of the proximal pole of the scaphoid are prone to adverse outcomes, such as nonunion and avascular necrosis.[1] [2] [3] [4] Operative treatment is more readily considered for proximal pole fractures than for waist fractures.[5] The data regarding the management of proximal pole fractures may be clouded by imprecision in distinction of waist fractures from proximal pole fractures. Relatively few scaphoid fracture classification systems describe how to distinguish proximal pole and scaphoid waist fractures ([Table 1]).

Table 1 Fractures of the scaphoid based on fracture location

Author (y)	Cases	Type of fractures	Definition proximal pole
Cooney et al (1980)[10]	45	1 Tuberosity 2 Distal articular surface 3 Distal one-third 4 Waist, middle one-third 5 Proximal pole	Proximal third on PA radiograph
Schernberg et al (1984)[11]	325	I Proximal pole II, III, IV Waist V Distal pole VI-a, b, c Distal tubercle	Involving about a third of the articular surface of the scaphoid with the radius and exiting at the capitolunate articulation.
Müller et al (1963)[12]	−	A1 Proximal pole, noncomminuted A2 Waist, noncomminuted A3 Distal pole, noncomminuted B2 Waist, comminuted	Not clearly described
Herbert and Fisher/Filan and Herbert (1984/1996)[13] [14]	200/431	A1 Stable, tubercle A2 Stable, incomplete waist B1 Unstable, distal oblique B2 Unstable, complete waist B3 Unstable, proximal pole B4 Unstable fracture dislocation B5 Unstable, comminuted	Not clearly described

Abbreviation: PA, posteroanterior.

Given the strong association between fracture displacement and nonunion for scaphoid waist fractures, it is also important to identify whether the fracture is displaced or not.[6] [7] We know little about the accuracy and reliability of imaging methods to identify fracture displacement of proximal pole scaphoid fractures. Lozano-Calderón et al concluded that an additional computed tomography (CT) scan improves the reliability of diagnosis of scaphoid displacement compared with radiographs alone.[8] Buijze et al showed that radiographs and CT scans did not accurately identify fracture displacement of scaphoid waist fractures.[6] They also found that the interobserver reliability was poor and improved only slightly with training.[9]

This study tested the primary null hypothesis that there is no difference in interobserver agreement for the diagnosis of proximal pole fractures of the scaphoid between observers that view radiographs alone and those that review radiographs and CT scans. It also addressed the secondary null hypothesis that there is no difference for the identification of fracture displacement.

Methods

After approval of our institutional review board, members of the Science of Variation Group (SOVG; 377 upper extremity surgeons) were invited to participate in this study. Of these, 84 (22%) surgeons responded and 57 (15%) completed the questionnaire. This is not a response rate since the email list of SOVG participants is not updated or filtered. Invitations were sent via email in December 2015, followed by a reminder 10 days later. The SOVG is an international collaboration of orthopaedic surgeons with upper extremity specialization. The aim of the collaboration is to study variation in definition, in this case scaphoid fracture location, and treatment of illness without financial incentives.

CT scans and radiographs of both patients with scaphoid proximal pole fractures and scaphoid waist fractures were obtained from a multi-institutional Research Patient Data Registry (RPDR) in a period from 2003 to 2015. RPDR is a centralized clinical data registry holding diagnostic codes (International Classification of Diseases, 9th revision code), Current Procedural Terminology (CPT) codes, demographic information (e.g., sex, date of birth, and race), radiology and operative reports, and visit notes.

Inclusion criteria were: patients of 18 years or older, availability of scaphoid radiograph series (posteroanterior wrist with ulnar deviation, lateral wrist, oblique wrist, and a scaphoid view [20°–30° tube angle]), and a CT scan that could be reformatted in the planes defined by the long axis of the scaphoid.[15] To obtain a sufficient number of patients with a proximal pole fracture, a CT scan, and to have a representative series, a consecutive series of scaphoid fractures, was obtained until there were five scaphoid proximal pole fractures. This resulted in 29 scaphoid fractures of which 5 were proximal pole fractures, identified by consensus between a radiologist and a hand surgeon without the use of a classification.

For this study, participants of the SOVG were divided in two groups at random. Using SurveyMonkey (Palo Alto, CA), participants were asked to decide on a diagnosis and displacement of either a waist or a proximal pole fracture, using radiographs alone or radiographs and a CT scan. No training, guidance, or measurement was provided to test what surgeons do in their daily practice.

The following explanatory variables were obtained: sex of observer, location of practice, years of practice, and specialization of observer.

Statistical Analysis

The multirater kappa measure described by Siegel and Castellan[16] was used to determine interobserver agreement. Using the guidelines of Landis and Koch, the generated kappa values were interpreted where a value of: 0.01 to 0.20 defines slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 0.99, almost perfect agreement; and 1.00, perfect agreement. Zero indicates no agreement beyond chance alone; –1.00, total disagreement.[17] A two-sample z-test was used to compare the kappa values and p-values of < 0.05 were considered significant.

Fig. 1 Anteriorposterior radiographs showing scaphoid fractures. Fractures are indicated with an arrow. Left: scaphoid proximal pole fracture, rated as proximal pole by 89% of observers and as a waist fracture by 11%; middle: scaphoid waist fracture where most interobserver variability was seen, rated as proximal pole by 23% of observers and as waist fracture by 77%; right: scaphoid waist fracture, rated as scaphoid waist fracture by all observers.

Fig. 2 Anteriorposterior radiograph of the scaphoid indicating the fracture zone where the distinction of proximal pole and waist fractures was less reliable.

Observer Characteristics

The majority of the 57 participants were men (n = 52, 91%) and practiced in the United States (n = 43, 75%) ([Table 2]). Most participants specialized in orthopaedic traumatology (n = 40, 70%) and supervised trainees in the operating room (n = 51, 89%).

Table 2 Observer demographics (n = 57)

Demographic	Radiographs	Radiographs plus CT	Total
	N (%)	N (%)	N
Sex
Male	27 (100)	25 (83)	52
Female	0	5 (17)	5
Area
United States	17 (63)	26 (87)	43
Europe	6 (22)	3 (10)	9
Other	4 (15)	1 (3)	5
Years in independent practice
0–5	9 (33)	10 (33)	19
6–10	8 (30)	4 (13)	12
11–20	8 (30)	13 (43)	21
21–30	2 (7)	3 (10)	5
Specialization
Traumatology	20 (74)	20 (67)	40
Shoulder and elbow	2 (7)	5 (17)	7
Hand and wrist	5 (19)	5 (17)	10
Supervises trainees in the operating room
Yes	24 (89)	27 (90)	51
No	3 (11)	3 (10)	6

Abbreviation: CT, computed tomography.

Results

Observers reviewing radiographs alone and observers reviewing radiographs and CT scans had substantial agreement on fracture location (κ = 0.82 and κ = 0.80, respectively; p = 0.53) ([Table 3]).

Table 3 Overall interobserver agreement

Question	Radiographs				Radiographs plus CT-scan				p-Value
	ĸ	SE	Agreement	95% CI[a]	ĸ	SE	Agreement	95% CI[a]
Is it a scaphoid proximal pole or waist fracture?	0.82	0.0269	Almost perfect	0.76–0.87	0.80	0.0186	Substantial	0.76–0.83	0.532
Is the fracture displaced?	0.35	0.0119	Fair	0.32–0.37	0.28	0.0288	Fair	0.22–0.34	0.034

Abbreviations: CI, confidence interval; ĸ, Cohen's kappa coefficient; SE, standard error; SEM, standard error of the mean.

^a CI, calculated (ĸ – [1.96 × SEM], ĸ + [1.96 × SEM].

Both groups had only fair agreement on fracture displacement, but the kappa value was significantly lower in the group reviewing radiographs and CT scans compared with those reviewing radiographs alone (κ = 0.28 and κ = 0.35, respectively; p = 0.029) ([Table 3]) ([Fig. 1]).

Discussion

Since scaphoid fracture treatment is based in part on location, reliable diagnosis of fracture location and displacement are important. This study compared the reliability of diagnosis of scaphoid fracture location and displacement using radiographs alone versus radiographs and CT scans. CT scans did not have a meaningful influence on the reliability of diagnosis on average. The diagnosis of displacement is unreliable.

This study has several limitations. First, the fractures were retrieved from a single tertiary hospital, which could limit generalizability, although that seems unlikely. Second, our results could be subject to a spectrum bias since proximal pole fractures were overrepresented in our sample. Third, most SOVG participants work in an academic setting (90% supervises trainees) and their values, training, and practice might differ from the larger community of surgeons. Fourth, there is no consensus definition of scaphoid proximal pole fractures and we did not use any training.[18] An advantage is that the study reflects daily clinical practice. A subset of observers might change their judgment when viewing radiographs and CT scans in the typical fashion. Finally, in this study—as in all SOVG studies—we did not assess intraobserver variability because it is always greater than interobserver variability and it is more difficult to study.

There was no difference in agreement on diagnosis of scaphoid fracture location with the use of either radiographs alone or when radiographs were accompanied by a CT scan. The reliability was high for both. This suggests that proximal pole scaphoid fractures are sufficiently distinct from waist fractures and that the added detail provided by CT does not improve the reliability of diagnosis of fracture location ([Fig. 2]).

In contrast, diagnosis of fracture displacement was only fair, with CT providing no advantage. Lozano-Calderón et al studied the reliability of diagnosis of fracture displacement of 30 scaphoid fractures among 6 observers with arthroscopy as the reference standard and demonstrated a reliability in kappa values of 0.43 for CT alone, 0.48 for CT and radiography, and 0.27 for observers just using radiographs; the average accuracy was 68, 77, and 75%, respectively.[8] Buijze et al studied displacement in 44 scaphoid waist fractures based on radiographs and CT scans, and also used arthroscopy as the reference standard.[6] They found that radiographs had an accuracy of 70% and CT scans of 82% for fracture displacement diagnosis when compared with arthroscopic examination. The study of Bernard et al found diagnosis of displacement had an estimated overall accuracy of 72% using conventional radiography in a cadaver study with artificially created scaphoid waist fractures. Of the 90 total possible pairwise agreements between interpreters regarding fracture displacement, there were 54 actual agreements (60%) with a kappa of 0.31 indicating fair interobserver agreement.[19] Collectively, these studies suggest that CT scans are useful for ruling out displacement, but cannot reliably or accurately diagnose the presence of displacement.

This study suggests that CT does not meaningfully improve the reliability of diagnosis of fracture location (which was almost perfect) or displacement (which was fair) in the scaphoid waist and proximal pole. Future studies might address dynamic imaging such as fluoroscopy or the impact of training and clear definitions of specific fracture types and displacement. It may be that proximal pole fractures are sufficiently distinct from proximal waist fractures that sophisticated imaging is unnecessary.

Conflict of Interest

None.

Science of Variation Group

H.L. Kimball, F.T.D. Kaplan, J. Isaacs, S. Dodds, R.S. Page, R. de Bedout, C. Ekholm, P. Muhl, A. Markiewitz, P. Hahn, J. C. Fanuele, J.M. Patiño, M. Calcagni, L. Katolik, N.E.L. Felipe, G.J. Bayne, M.J. Richard, A.B. Shafritz, C. Lomita, E. Hofmeister, J.M. Erickson, M. Baskies, K. Erol, F. Verstreken, H.B. Bamberger, J.A. Izzi, C.J.R. Barreto, M. Nancollas, K.J. Prommersberger, R.S. Gilbert, C. Rodner, S. Kakar, C. Swigart, D. Polatsch, B.F. Hearon, F. Walter, S.M. Jacoby, G. DeSilva, D.M. McKee, J.T. Capo, M.J. Palmer, M. Richardson, P.J. Evans, S.A. Kennedy, T. Baxamusa, C.M. Jones, M. Rizzo, D.L. Nelson, A.J.H. Vochteloo, B.P.D. Wills, S.F. Duncan, N.M. Akabudike, J. Sandoval, L. Weiss, C.L. Moreno-Serrano, D. Eygendaal, and L.B. Lane.

Note

This work was performed at the Orthopedic Hand and Upper Extremity Service and Department of Behavioral Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts. The Massachusetts General Hospital Institutional Review Board has approved the human protocol for this investigation under number 2009P001019.

• References

• 1 Buijze GA, Ochtman L, Ring D. Management of scaphoid nonunion. J Hand Surg Am 2012; 37 (05) 1095-1100
• 2 Krimmer H. Management of acute fractures and nonunions of the proximal pole of the scaphoid. J Hand Surg [Br] 2002; 27 (03) 245-248
• 3 Ring D, Jupiter JB, Herndon JH. Acute fractures of the scaphoid. J Am Acad Orthop Surg 2000; 8 (04) 225-231
• 4 Szabo RM, Manske D. Displaced fractures of the scaphoid. Clin Orthop Relat Res 1988; (230) 30-38
• 5 Adams JE, Steinmann SP. Acute scaphoid fractures. Orthop Clin North Am 2007; 38 (02) 229-235
• 6 Buijze GA, Jørgsholm P, Thomsen NO, Bjorkman A, Besjakov J, Ring D. Diagnostic performance of radiographs and computed tomography for displacement and instability of acute scaphoid waist fractures. J Bone Joint Surg Am 2012; 94 (21) 1967-1974
• 7 Singh HP, Taub N, Dias JJ. Management of displaced fractures of the waist of the scaphoid: meta-analyses of comparative studies. Injury 2012; 43 (06) 933-939
• 8 Lozano-Calderón S, Blazar P, Zurakowski D, Lee SG, Ring D. Diagnosis of scaphoid fracture displacement with radiography and computed tomography. J Bone Joint Surg Am 2006; 88 (12) 2695-2703
• 9 Buijze GA, Guitton TG, van Dijk CN, Ring D. ; Science of Variation Group. Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement. Clin Orthop Relat Res 2012; 470 (07) 2029-2034
• 10 Cooney WP, Dobyns JH, Linscheid RL. Fractures of the scaphoid: a rational approach to management. Clin Orthop Relat Res 1980; (149) 90-97
• 11 Schernberg F, Elzein F, Gerard Y. [Anatomo-radiological study of fractures of the carpal scaphoid bone. Problems of abnormal callus]. Rev Chir Orthop Reparatrice Appar Mot 1984; 70 (Suppl. 02) 55-63
• 12 Müller ME, Allgöwer M, Willenegger H. Technik der operativen Frakturenbehandlung. 253 1st ed. Berlin: Springer; 1963
• 13 Herbert TJ, Fisher WE. Management of the fractured scaphoid using a new bone screw. J Bone Joint Surg Br 1984; 66 (01) 114-123
• 14 Filan SL, Herbert TJ. Herbert screw fixation of scaphoid fractures. J Bone Joint Surg Br 1996; 78 (04) 519-529
• 15 Sanders WE. Evaluation of the humpback scaphoid by computed tomography in the longitudinal axial plane of the scaphoid. J Hand Surg Am 1988; 13 (02) 182-187
• 16 Siegel S. Castellan JN. Nonparametric Statistics for the Behavioral Sciences. Vol. 2. New York: McGraw-Hill; 1988
• 17 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
• 18 Ten Berg PW, Drijkoningen T, Strackee SD, Buijze GA. Classifications of acute scaphoid fractures: a systematic literature review. J Wrist Surg 2016; 5 (02) 152-159
• 19 Bernard SA, Murray PM, Heckman MG. Validity of conventional radiography in determining scaphoid waist fracture displacement. J Orthop Trauma 2010; 24 (07) 448-451