Korean J Radiol.  2015 Aug;16(4):889-898. 10.3348/kjr.2015.16.4.889.

MRI Inter-Reader and Intra-Reader Reliabilities for Assessing Injury Morphology and Posterior Ligamentous Complex Integrity of the Spine According to the Thoracolumbar Injury Classification System and Severity Score

• Lee GY
• Lee JW
• Choi SW
• Lim HJ
• Sun HY
• Kang Y
• Chai JW
• Kim S
• Kang HS

• KMID: 2155564
• DOI: http://doi.org/10.3348/kjr.2015.16.4.889

Abstract

OBJECTIVE
To evaluate spine magnetic resonance imaging (MRI) inter-reader and intra-reader reliabilities using the thoracolumbar injury classification system and severity score (TLICS) and to analyze the effects of reader experience on reliability and the possible reasons for discordant interpretations.
MATERIALS AND METHODS
Six radiologists (two senior, two junior radiologists, and two residents) independently scored 100 MRI examinations of thoracolumbar spine injuries to assess injury morphology and posterior ligamentous complex (PLC) integrity according to the TLICS. Inter-reader and intra-reader agreements were determined and analyzed according to the number of years of radiologist experience.
RESULTS
Inter-reader agreement between the six readers was moderate (k = 0.538 for the first and 0.537 for the second review) for injury morphology and fair to moderate (k = 0.440 for the first and 0.389 for the second review) for PLC integrity. No significant difference in inter-reader agreement was observed according to the number of years of radiologist experience. Intra-reader agreements showed a wide range (k = 0.538-0.822 for injury morphology and 0.423-0.616 for PLC integrity). Agreement was achieved in 44 for the first and 45 for the second review about injury morphology, as well as in 41 for the first and 38 for the second review of PLC integrity. A positive correlation was detected between injury morphology score and PLC integrity.
CONCLUSION
The reliability of MRI for assessing thoracolumbar spinal injuries according to the TLICS was moderate for injury morphology and fair to moderate for PLC integrity, which may not be influenced by radiologist' experience.

Keyword

Thoracolumbar; Trauma; MRI; Reliability; Kappa; Radiologist

MeSH Terms

Adolescent
Adult
Aged
Aged, 80 and over
Child
Female
Humans
*Injury Severity Score
Ligamentum Flavum/pathology
Lumbar Vertebrae/*injuries/pathology
Magnetic Resonance Imaging/*methods
Male
Middle Aged
Observer Variation
Reproducibility of Results
Retrospective Studies
Spinal Injuries/*classification/*pathology
Thoracic Vertebrae/*injuries/pathology
Thoracic Wall/pathology
Young Adult

Figure

• 

  Fig. 1 Case of disagreement about posterior ligamentous complex (PLC) integrity by readers. 49-year-old man showed burst injury of L1 body on T2-weighted sagittal image (A) with bulging contour of posterior cortex on T2-weighted axial scan (B, arrows), which was agreed by all readers during first and second reviews. Suspicious increase of signal intensity was detected along PLC on T2-weighted short time inversion recovery sagittal image (C, arrow), which became disagreement during first ("intact" by one; "indeterminate" by four; "injured" by one) and second reviews ("intact" by one; "indeterminate" by five readers).

• 

  Fig. 2 Case of disagreement about injury morphology and posterior ligamentous complex (PLC) integrity by readers. 89-year-old man suffered traumatic lesion of L1 body with epidural hemorrhage at T12-L2 level (arrows) on T2- (A) and T1-weighted sagittal images (B), with minimal bulging of posterior cortex, in which injury morphology became disagreement during first ("compression" by three; "burst" by three readers) and second reviews ("compression" by five; "burst" by one). All readers agreed on intact PLC on T2-weighted short time inversion recovery sagittal image after first review (C, dashed arrow) but there was disagreement during second review ("intact" by five; "indeterminate" by one).

• 

  Fig. 3 Case of disagreement on injury morphology by readers. Posterior ligamentous complex (PLC) disruption was clearly demonstrable and was agreed upon by all six readers during first and second reviews of T2-weighted non-fat saturation (A) and short time inversion recovery sagittal images (B) and T2-weighted axial scan (C) of 33-year-old man (arrows). However, readers failed to agree on injury morphology: "compression" by one, "burst" by one, and "distraction" by four readers during first review and "compression" by one, "burst" by two, and "distraction" by three on second review.

• 

  Fig. 4 Case of disagreement about posterior ligamentous complex (PLC) integrity by readers. 51-year-old man demonstrated traumatic lesions at L1 and L4 bodies on T2- (A) and T1-weighted sagittal images (B) with epidural hemorrhage at L3-5 level (arrows, A, B). L1 lesion revealed bulging of posterior cortex on T2-weighted axial scan (C, arrows), which was agreed to be burst injury by all readers during first review and by five during second review. However, no consensus was reached about PLC integrity on T2-weighted short time inversion recovery sagittal images (D, E, arrow), which produced variety of scores for PLC intergtity ("intact" by two readers; "indeterminate" by four in first and "intact" by three; "indeterminate" by two; and "injured" by one during second review). In contrast, PLC integrity (E, dashed arrow) of L4 body lesion was not evaluated using thoracolumbar injury classification system and was originally classified as most severe injury level.

Reference

1. Hu R, Mustard CA, Burns C. Epidemiology of incident spinal fracture in a complete population. Spine (Phila Pa 1976). 1996; 21:492–499.

2. Patel AA, Vaccaro AR. Thoracolumbar spine trauma classification. J Am Acad Orthop Surg. 2010; 18:63–71.

3. Ferguson RL, Allen BL Jr. A mechanistic classification of thoracolumbar spine fractures. Clin Orthop Relat Res. 1984; (189):77–88.

4. Watson-Jones R. The results of postural reduction of the fractures of the spine. J Bone Joint Surg Am. 1938; 20:567–586.

5. Denis F. The three column spine and its significance in the classification of acute thoracolumbar spinal injuries. Spine (Phila Pa 1976). 1983; 8:817–883.

6. Magerl F, Aebi M, Gertzbein SD, Harms J, Nazarian S. A comprehensive classification of thoracic and lumbar injuries. Eur Spine J. 1994; 3:184–201.

7. Mirza SK, Mirza AJ, Chapman JR, Anderson PA. Classifications of thoracic and lumbar fractures: rationale and supporting data. J Am Acad Orthop Surg. 2002; 10:364–377.

8. Holdsworth F. Fractures, dislocations, and fracture-dislocations of the spine. J Bone Joint Surg Am. 1970; 52:1534–1551.

9. Wood KB, Khanna G, Vaccaro AR, Arnold PM, Harris MB, Mehbod AA. Assessment of two thoracolumbar fracture classification systems as used by multiple surgeons. J Bone Joint Surg Am. 2005; 87:1423–1429.

10. Oner FC, Ramos LM, Simmermacher RK, Kingma PT, Diekerhof CH, Dhert WJ, et al. Classification of thoracic and lumbar spine fractures: problems of reproducibility. A study of 53 patients using CT and MRI. Eur Spine J. 2002; 11:235–245.

11. Blauth M, Bastian L, Knop C, Lange U, Tusch G. [Inter-observer reliability in the classification of thoraco-lumbar spinal injuries]. Orthopade. 1999; 28:662–681.

12. Vaccaro AR, Lehman RA Jr, Hurlbert RJ, Anderson PA, Harris M, Hedlund R, et al. A new classification of thoracolumbar injuries: the importance of injury morphology, the integrity of the posterior ligamentous complex, and neurologic status. Spine (Phila Pa 1976). 2005; 30:2325–2333.

13. Vaccaro AR, Zeiller SC, Hulbert RJ, Anderson PA, Harris M, Hedlund R, et al. The thoracolumbar injury severity score: a proposed treatment algorithm. J Spinal Disord Tech. 2005; 18:209–215.

14. Vaccaro AR, Baron EM, Sanfilippo J, Jacoby S, Steuve J, Grossman E, et al. Reliability of a novel classification system for thoracolumbar injuries: the Thoracolumbar Injury Severity Score. Spine (Phila Pa 1976). 2006; 31:11 Suppl. S62–S69. discussion S104

15. Raja Rampersaud Y, Fisher C, Wilsey J, Arnold P, Anand N, Bono CM, et al. Agreement between orthopedic surgeons and neurosurgeons regarding a new algorithm for the treatment of thoracolumbar injuries: a multicenter reliability study. J Spinal Disord Tech. 2006; 19:477–482.

16. Whang PG, Vaccaro AR, Poelstra KA, Patel AA, Anderson DG, Albert TJ, et al. The influence of fracture mechanism and morphology on the reliability and validity of two novel thoracolumbar injury classification systems. Spine (Phila Pa 1976). 2007; 32:791–795.

17. Ratliff J, Anand N, Vaccaro AR, Lim MR, Lee JY, Arnold P, et al. Regional variability in use of a novel assessment of thoracolumbar spine fractures: United States versus international surgeons. World J Emerg Surg. 2007; 2:24.

18. Patel AA, Whang PG, Brodke DS, Agarwal A, Hong J, Fernandez C, et al. Evaluation of two novel thoracolumbar trauma classification systems. Indian J Orthop. 2007; 41:322–326.

19. Patel AA, Vaccaro AR, Albert TJ, Hilibrand AS, Harrop JS, Anderson DG, et al. The adoption of a new classification system: time-dependent variation in interobserver reliability of the thoracolumbar injury severity score classification system. Spine (Phila Pa 1976). 2007; 32:E105–E110.

20. Lenarz CJ, Place HM, Lenke LG, Alander DH, Oliver D. Comparative reliability of 3 thoracolumbar fracture classification systems. J Spinal Disord Tech. 2009; 22:422–427.

21. Koh YD, Kim DJ, Koh YW. Reliability and Validity of Thoracolumbar Injury Classification and Severity Score (TLICS). Asian Spine J. 2010; 4:109–117.

22. Lenarz CJ, Place HM. Evaluation of a new spine classification system, does it accurately predict treatment? J Spinal Disord Tech. 2010; 23:192–196.

23. Joaquim AF, Fernandes YB, Cavalcante RA, Fragoso RM, Honorato DC, Patel AA. Evaluation of the thoracolumbar injury classification system in thoracic and lumbar spinal trauma. Spine (Phila Pa 1976). 2011; 36:33–36.

24. Lewkonia P, Paolucci EO, Thomas K. Reliability of the thoracolumbar injury classification and severity score and comparison with the denis classification for injury to the thoracic and lumbar spine. Spine (Phila Pa 1976). 2012; 37:2161–2167.

25. Pizones J, Izquierdo E, Alvarez P, Sánchez-Mariscal F, Zúñiga L, Chimeno P, et al. Impact of magnetic resonance imaging on decision making for thoracolumbar traumatic fracture diagnosis and treatment. Eur Spine J. 2011; 20:Suppl 3. 390–396.

26. Winklhofer S, Thekkumthala-Sommer M, Schmidt D, Rufibach K, Werner CM, Wanner GA, et al. Magnetic resonance imaging frequently changes classification of acute traumatic thoracolumbar spine injuries. Skeletal Radiol. 2013; 42:779–786.

27. Donner A, Rotondi MA. Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat. 2010; 6:Article 31.

28. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971; 76:378–382.

29. Patel AA, Dailey A, Brodke DS, Daubs M, Harrop J, Whang PG, et al. Thoracolumbar spine trauma classification: the Thoracolumbar Injury Classification and Severity Score system and case examples. J Neurosurg Spine. 2009; 10:201–206.

30. Lee JY, Vaccaro AR, Lim MR, Oner FC, Hulbert RJ, Hedlund R, et al. Thoracolumbar injury classification and severity score: a new paradigm for the treatment of thoracolumbar spine trauma. J Orthop Sci. 2005; 10:671–675.

31. Sethi MK, Schoenfeld AJ, Bono CM, Harris MB. The evolution of thoracolumbar injury classification systems. Spine J. 2009; 9:780–788.

32. Schweitzer KM Jr, Vaccaro AR, Lee JY, Grauer JN. Spine Trauma Study Group. Confusion regarding mechanisms of injury in the setting of thoracolumbar spinal trauma: a survey of The Spine Trauma Study Group (STSG). J Spinal Disord Tech. 2006; 19:528–530.

33. McArdle CB, Crofford MJ, Mirfakhraee M, Amparo EG, Calhoun JS. Surface coil MR of spinal trauma: preliminary experience. AJNR Am J Neuroradiol. 1986; 7:885–893.

34. Emery SE, Pathria MN, Wilber RG, Masaryk T, Bohlman HH. Magnetic resonance imaging of posttraumatic spinal ligament injury. J Spinal Disord. 1989; 2:229–233.

35. Kliewer MA, Gray L, Paver J, Richardson WD, Vogler JB, McElhaney JH, et al. Acute spinal ligament disruption: MR imaging with anatomic correlation. J Magn Reson Imaging. 1993; 3:855–861.

36. Petersilge CA, Pathria MN, Emery SE, Masaryk TJ. Thoracolumbar burst fractures: evaluation with MR imaging. Radiology. 1995; 194:49–54.

37. Terk MR, Hume-Neal M, Fraipont M, Ahmadi J, Colletti PM. Injury of the posterior ligament complex in patients with acute spinal trauma: evaluation by MR imaging. AJR Am J Roentgenol. 1997; 168:1481–1486.

38. Lee HM, Kim HS, Kim DJ, Suk KS, Park JO, Kim NH. Reliability of magnetic resonance imaging in detecting posterior ligament complex injury in thoracolumbar spinal fractures. Spine (Phila Pa 1976). 2000; 25:2079–2084.

39. Haba H, Taneichi H, Kotani Y, Terae S, Abe S, Yoshikawa H, et al. Diagnostic accuracy of magnetic resonance imaging for detecting posterior ligamentous complex injury associated with thoracic and lumbar fractures. J Neurosurg. 2003; 99:1 Suppl. 20–26.

40. Crosby CG, Even JL, Song Y, Block JJ, Devin CJ. Diagnostic abilities of magnetic resonance imaging in traumatic injury to the posterior ligamentous complex: the effect of years in training. Spine J. 2011; 11:747–753.

41. Vaccaro AR, Lee JY, Schweitzer KM Jr, Lim MR, Baron EM, Oner FC, et al. Assessment of injury to the posterior ligamentous complex in thoracolumbar spine trauma. Spine J. 2006; 6:524–528.

42. Dai LY, Ding WG, Wang XY, Jiang LS, Jiang SD, Xu HZ. Assessment of ligamentous injury in patients with thoracolumbar burst fractures using MRI. J Trauma. 2009; 66:1610–1615.

43. Schweitzer KM, Vaccaro AR, Harrop JS, Hurlbert J, Carrino JA, Rechtine GR, et al. Interrater reliability of identifying indicators of posterior ligamentous complex disruption when plain films are indeterminate in thoracolumbar injuries. J Orthop Sci. 2007; 12:437–442.

44. Vaccaro AR, Rihn JA, Saravanja D, Anderson DG, Hilibrand AS, Albert TJ, et al. Injury of the posterior ligamentous complex of the thoracolumbar spine: a prospective evaluation of the diagnostic accuracy of magnetic resonance imaging. Spine (Phila Pa 1976). 2009; 34:E841–E847.

45. Rihn JA, Yang N, Fisher C, Saravanja D, Smith H, Morrison WB, et al. Using magnetic resonance imaging to accurately assess injury to the posterior ligamentous complex of the spine: a prospective comparison of the surgeon and radiologist. J Neurosurg Spine. 2010; 12:391–396.

46. Pizones J, Zúñiga L, Sánchez-Mariscal F, Alvarez P, Gómez-Rice A, Izquierdo E. MRI study of post-traumatic incompetence of posterior ligamentous complex: importance of the supraspinous ligament. Prospective study of 74 traumatic fractures. Eur Spine J. 2012; 21:2222–2231.