ABSTRACT
Study design: Interobserver and intraobserver reliability
Objective: To measure and compare the interobserver and intraobserver reliability of the cervical
spine injury severity score (CSISS), the subaxial injury classification (SLIC) and
severity scale, and the Allen-Ferguson system in patients with subaxial cervical spine
injuries presenting to the emergency department.
Methods: Five examiners independently reviewed c-spine x-rays (CT/MRI) of 50 consecutive patients
with subaxial cervical-spine injuries. They classified each case using CSISS, SLIC,
and the Allen-Ferguson system. Examiners also documented if they believed the case
required surgical management. At least 6 weeks later, the above steps were repeated
for ten randomly chosen cases.
Results: The interobserver and intraobserver reliability for the total CSISS and total SLIC
score are excellent. There is poor interobserver reliability and excellent intraobserver
reliability when a total kappa score is calculated using all 21 groups for the Allen-Ferguson
system. With respect to surgical management decisions, the interobserver agreement
is moderate and the intraobserver agreement is excellent.
Conclusions: There is no universally accepted classification scheme for subaxial cervical-spine
injuries. A useful classification system must have excellent reliability to consistently
and accurately describe injury patterns between different observers and allow for
comparison across systems or cohorts. Both the CSISS and the SLIC and severity scale
are promising classification systems with excellent interobserver and intraobserver
reliability. Future studies will need to determine if their quantitative scores correlate
with management and clinical outcomes.
STUDY RATIONALE AND CONTEXT
The identification and appropriate treatment of subaxial cervical-spine injuries is
essential to optimize outcomes. Injuries to the cervical spine are present in only
1 %–3 % of people who sustain blunt trauma; however, the morbidity and mortality associated
with these injuries can be devastating [1], [2]. Numerous classification systems have been proposed to describe these injuries,
predict stability, and dictate treatment; still, none of them are universally accepted
[3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17]. The „ideal” classification system must have excellent interobserver and intraobserver
reliability, quantify stability, predict prognosis, and dictate treatment. We rely
on a universal classification system as a prerequisite for comparison of clinical
outcomes across different techniques and researchers. Newer systems have started to
attempt to quantify injuries on a continuum in the form of objectively obtainable
injury severity scales instead of differentiating injuries into various subtypes.
To date, no studies have simultaneously evaluated the CSISS and the SLIC as two examples
of a severity scale for cervical-spine injuries, and the Allen-Ferguson system as
the most representative example of a typical classification system with a phylogeny
of injury categories.
OBJECTIVE
To measure and compare the interobserver and intraobserver reliability of CSISS, SLIC
and the Allen-Ferguson system in patients with subaxial cervical-spine injuries.
METHODS
Study design:
Interobserver and intraobserver reliability.
Inclusion criteria:
Patients seen in the emergency department with significant subaxial cervical injury
with adequate imaging showing the morphology on computed tomography (CT) and / or
magnetic resonance imaging (MRI).
Exclusion criteria:
Patients with spine fractures outside the subaxial cervical region were excluded.
Patient population:
Fifty consecutive patients seen in the emergency department at Harborview Medical
Center (Seattle, WA) from April 2007 to August 2007 meeting the inclusion criteria.
CT was available for 100 % of patients and MRI was available for 70 % of patients.
Classification systems evaluated: (please see web appendix at www.aospine.org / ebsj
for additional details)
-
CSISS is an ordinal score which divides the subaxial cervical spine into four columns:
anterior, posterior, right pillar (right lateral column), and left pillar (left lateral
column) and takes into account fractures as well as ligamentous injuries. Each column
is given a score from zero (no injury) to five (most significant injury possible to
that column) based on the severity of injury (Fig [1]). The total quantitative scored is determined by adding the scores for each column
at a given level of injury for a maximum score of 20. If there are multiple levels
of injury, the highest quantitative score is used after determining the score for
each individual level of injury (Figs [2] a–b) [4], [14].
-
SLIC and severity scale is an ordinal score comprised of three components: (1) injury
morphology as determined by the pattern of spinal column disruption on available imaging
studies; (2) integrity of the discoligamentous complex (DLC) represented by both anterior
and posterior ligamentous structures as well as the intervertebral disc, and (3) neurological
status of the patient [16]. Higher scores represent more severe injuries (Table [1], Figs [2] a–b). Although both CSISS and SLIC are based on injury morphology and the integrity
of the DLC, only SLIC takes into account neurological status.
-
The Allen-Ferguson system is based on mechanism of injury. Six different phylogenies
(compressive flexion, vertical compression, distractive flexion, compressive extension,
distractive extension, and lateral flexion) are evaluated. There are different stages,
based on severity, within each phylogeny for a total of 21 different possible classification
types. It is a nonordinal system that does not quantify severity or dictate treatment.
Assessment process:
Patient studies were de-identified and a new identity number randomly assigned to
facilitate reviewer blinding. The images were copied to DVDs and distributed to the
reviewers. Original papers and quick reference guides describing each classification
system were provided to five spine surgeons who independently reviewed cervical spine
radiographs (CT and MRI). To determine neurological status for the SLIC, reviewers
were provided documented physical examinations from each patient’s chart. The reviewers
independently classified each patient’s injuries for all three classification systems
and recorded whether surgery was indicated. At least 6 weeks after the interobserver
data were collected, 10 of the original 50 cases were randomly chosen for the intraobserver
results. The above steps were then repeated. Reviewers were blinded to the results
of the previous assessment.
Analysis:
Interobserver reliability for both CSISS and SLIC and severity scale was determined
with intraclass correlation coefficient (ICC) using two-way random effects. ICC was
used since we had more than two raters and because these systems are ordinal with
higher scores representing more severe injuries. Interobserver reliability was calculated
for the Allen-Ferguson system as well as for management of these injuries using kappa
(INTER_RATER.MAC in SAS version 9.1.3 for Windows). We used kappa for the Allen-Ferguson
classification since this is a nominal system with no natural ordering to the different
phylogenies. Cohen’s kappa was not used since it determines agreement between two
raters only [18], [19], [20]. We considered ICC and kappa scores > 0.75 as excellent, 0.4–0.75 as moderate, and
scores < 0.4 as poor [21].
Additional methodological and technical details are provided in the web appendix at
www.aospine.org/ebsj.
AN ILLUSTRATIVE CASE
The images in Figs [2] a and b demonstrate how the CSISS and SLIC scores are calculated. The anterior column
of this patient is displaced more than 5 mm, so according to the analog scale of the
CSISS it would receive a score of 5. Both the right and lateral pillars have no injuries
so they are assigned a score of 0. The posterior column has mild displacement (˜2
mm), scoring a value of 2. Therefore, the total CSISS score for all four columns is
7. If there were multiple levels of injury within the subaxial cervical spine, the
level with the highest total CSISS score would be used. With respect to the SLIC scale
theit would receive a score of 2 for morphology, since it is a burst-type fracture.
The discoligamentous complex is intact (this was confirmed also on MRI), scoring 0
for that component. The final component of the SLIC score is neurological status.
The examinations from the patient’s chart show that the patient had 0 / 5 strength
in the lower extremities and no sensation below T2 including absent perianal sensation.
Based on these clinical findings the score for neurological status would be a 2 as
the patient has a complete cord injury. Therefore, the total SLIC score is 4.
