Endocrinol Metab.  2022 Aug;37(4):674-683. 10.3803/EnM.2022.1461.

Development of a Spine X-Ray-Based Fracture Prediction Model Using a Deep Learning Algorithm

• Kong SH
• Lee JW
• Bae BU
• Sung JK
• Jung KH
• Kim JH
• Shin CS

• KMID: 2532867
• DOI: http://doi.org/10.3803/EnM.2022.1461

Abstract

Background
Since image-based fracture prediction models using deep learning are lacking, we aimed to develop an X-ray-based fracture prediction model using deep learning with longitudinal data.
Methods
This study included 1,595 participants aged 50 to 75 years with at least two lumbosacral radiographs without baseline fractures from 2010 to 2015 at Seoul National University Hospital. Positive and negative cases were defined according to whether vertebral fractures developed during follow-up. The cases were divided into training (n=1,416) and test (n=179) sets. A convolutional neural network (CNN)-based prediction algorithm, DeepSurv, was trained with images and baseline clinical information (age, sex, body mass index, glucocorticoid use, and secondary osteoporosis). The concordance index (C-index) was used to compare performance between DeepSurv and the Fracture Risk Assessment Tool (FRAX) and Cox proportional hazard (CoxPH) models.
Results
Of the total participants, 1,188 (74.4%) were women, and the mean age was 60.5 years. During a mean follow-up period of 40.7 months, vertebral fractures occurred in 7.5% (120/1,595) of participants. In the test set, when DeepSurv learned with images and clinical features, it showed higher performance than FRAX and CoxPH in terms of C-index values (DeepSurv, 0.612; 95% confidence interval [CI], 0.571 to 0.653; FRAX, 0.547; CoxPH, 0.594; 95% CI, 0.552 to 0.555). Notably, the DeepSurv method without clinical features had a higher C-index (0.614; 95% CI, 0.572 to 0.656) than that of FRAX in women.
Conclusion
DeepSurv, a CNN-based prediction algorithm using baseline image and clinical information, outperformed the FRAX and CoxPH models in predicting osteoporotic fracture from spine radiographs in a longitudinal cohort.

Keyword

Osteoporotic fractures; Deep learning; X-rays; Risk assessment

Figure

• 

  Fig. 1. Flow chart of patient selection. SNUH, Seoul National University Hospital; AP, anteroposterior.

• 

  Fig. 2. The architecture of the deep learning-based survival prediction model. HRNet, high-resolution net; ResNet, residual network; BMI, body mass index; FC, fold change.

• 

  Fig. 3. Performance of the fracture prediction model in the training set using Cox proportional hazard and DeepSurv methods (A) according to clinical models and (B) analytic methods. Model 1 adjusted for age and sex, model 2 additionally adjusted for body mass index, and model 3 additionally adjusted for the use of glucocorticoids and secondary osteoporosis. The C-index values were as follows: Model 1 (Cox proportional hazard [CoxPH], 0.712; 95% confidence interval [CI], 0.652 to 0.773; DeepSurv without images, 0.765; 95% CI, 0.693 to 0.837; DeepSurv with images, 0.794; 95% CI, 0.760 to 0.828); Model 2 (CoxPH, 0.709; 95% CI, 0.648 to 0.771; DeepSurv without images, 0.737; 95% CI, 0.683 to 0.791; DeepSurv with images, 0.782; 95% CI, 0.755 to 0.810); Model 3 (CoxPH, 0.712; 95% CI, 0.654 to 0.770; DeepSurv without images, 0.740; 95% CI, 0.686 to 0.795; DeepSurv with images, 0.764; 95% CI, 0.739 to 0.789). aP<0.05 between groups.

• 

  Fig. 4. Performance of fracture prediction model in the test set using Cox proportional hazard and DeepSurv methods (A) according to clinical models and (B) analytic methods. Model 1 adjusted for age, model 2 additionally adjusted for body mass index, and model 3 additionally adjusted for the use of glucocorticoids and secondary osteoporosis. The C-index values were as follows: Model 1 (Cox proportional hazard [CoxPH], 0.552; 95% confidence interval [CI], 0.550 to 0.554; DeepSurv without images, 0.568; 95% CI, 0.521 to 0.585; DeepSurv with images, 0.544; 95% CI, 0.481 to 0.607); Model 2 (CoxPH, 0.553; 95% CI, 0.552 to 0.555; DeepSurv without images, 0.558; 95% CI, 0.521 to 0.595; DeepSurv with images, 0.610; 95% CI, 0.576 to 0.644); Model 3 (CoxPH, 0.594; 95% CI, 0.584 to 0.604; DeepSurv without images, 0.433; 95% CI, 0.510 to 0.579; DeepSurv with images, 0.612; 95% CI, 0.571 to 0.653). FRAX, Fracture Risk Assessment Tool. aP<0.05 between groups.

• 

Reference

1. Tran O, Silverman S, Xu X, Bonafede M, Fox K, McDermott M, et al. Long-term direct and indirect economic burden associated with osteoporotic fracture in US postmenopausal women. Osteoporos Int. 2021; 32:1195–205.

2. Williams SA, Daigle SG, Weiss R, Wang Y, Arora T, Curtis JR. Economic burden of osteoporosis-related fractures in the US Medicare population. Ann Pharmacother. 2021; 55:821–9.

3. National Bone Health Policy Institute. New report on burden of osteoporosis highlights huge and growing economic and human toll of the disease [Internet]. Arlington: BHOF;2019. [cited 2022 Jul 11]. Available from: https://www.bonehealthandosteoporosis.org/news/new-report-on-burden-of-osteoporosis-highlights-huge-and-growing-economic-and-human-toll-of-the-disease/.

4. Stone KL, Seeley DG, Lui LY, Cauley JA, Ensrud K, Browner WS, et al. BMD at multiple sites and risk of fracture of multiple types: long-term results from the Study of Osteoporotic Fractures. J Bone Miner Res. 2003; 18:1947–54.

5. Kanis JA, Harvey NC, Johansson H, Oden A, Leslie WD, McCloskey EV. FRAX update. J Clin Densitom. 2017; 20:360–7.

6. Dimai HP. Use of dual-energy X-ray absorptiometry (DXA) for diagnosis and fracture risk assessment: WHO-criteria, T- and Z-score, and reference databases. Bone. 2017; 104:39–43.

7. Aspray TJ. New horizons in fracture risk assessment. Age Ageing. 2013; 42:548–54.

8. Hoiberg MP, Rubin KH, Hermann AP, Brixen K, Abrahamsen B. Diagnostic devices for osteoporosis in the general population: a systematic review. Bone. 2016; 92:58–69.

9. Marshall D, Johnell O, Wedel H. Meta-analysis of how well measures of bone mineral density predict occurrence of osteoporotic fractures. BMJ. 1996; 312:1254–9.

10. Siris ES, Chen YT, Abbott TA, Barrett-Connor E, Miller PD, Wehren LE, et al. Bone mineral density thresholds for pharmacological intervention to prevent fractures. Arch Intern Med. 2004; 164:1108–12.

11. Keel S, Wu J, Lee PY, Scheetz J, He M. Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma. JAMA Ophthalmol. 2019; 137:288–92.

12. Sung J, Park S, Lee SM, Bae W, Park B, Jung E, et al. Added value of deep learning-based detection system for multiple major findings on chest radiographs: a randomized crossover study. Radiology. 2021; 299:450–9.

13. Yasaka K, Akai H, Abe O, Kiryu S. Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology. 2018; 286:887–96.

14. Yamashita R, Nishio M, Do R, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights Imaging. 2018; 9:611–29.

15. Derkatch S, Kirby C, Kimelman D, Jozani MJ, Davidson JM, Leslie WD. Identification of vertebral fractures by convolutional neural networks to predict nonvertebral and hip fractures: a registry-based cohort study of dual X-ray absorptiometry. Radiology. 2019; 293:405–11.

16. Bluthgen C, Becker AS, Vittoria de Martini I, Meier A, Martini K, Frauenfelder T. Detection and localization of distal radius fractures: deep learning system versus radiologists. Eur J Radiol. 2020; 126:108925.

17. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017; 88:581–6.

18. Yasaka K, Akai H, Kunimatsu A, Kiryu S, Abe O. Prediction of bone mineral density from computed tomography: application of deep learning with a convolutional neural network. Eur Radiol. 2020; 30:3549–57.

19. Loffler MT, Jacob A, Scharr A, Sollmann N, Burian E, El Husseini M, et al. Automatic opportunistic osteoporosis screening in routine CT: improved prediction of patients with prevalent vertebral fractures compared to DXA. Eur Radiol. 2021; 31:6069–77.

20. Shin CS, Kim MJ, Shim SM, Kim JT, Yu SH, Koo BK, et al. The prevalence and risk factors of vertebral fractures in Korea. J Bone Miner Metab. 2012; 30:183–92.

21. Sun K, Xiao B, Liu D, Wang J. Deep high-resolution representation learning for human pose estimation. In : Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2019 Jun 16-20; Long Beach, CA. Available from: https://openaccess.thecvf.com/content_CVPR_2019/papers/Sun_Deep_High-Resolution_Representation_Learning_for_Human_Pose_Estimation_CVPR_2019_paper.pdf.

22. Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol. 2018; 18:24.

23. Iyer S, Sowmya A, Blair A, White C, Dawes L, Moses D. A novel approach to vertebral compression fracture detection using imitation learning and patch based convolutional neural network. In : Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI); 2020 Apr 3-7; Iowa City, IA. Piscataway, NJ: IEEE;2020. p. 726–30.

24. de Vries B, Hegeman JH, Nijmeijer W, Geerdink J, Seifert C, Groothuis-Oudshoorn C. Comparing three machine learning approaches to design a risk assessment tool for future fractures: predicting a subsequent major osteoporotic fracture in fracture patients with osteopenia and osteoporosis. Osteoporos Int. 2021; 32:437–49.

25. Xiao X, Wu Q. The utility of genetic risk score to improve performance of FRAX for fracture prediction in US postmenopausal women. Calcif Tissue Int. 2021; 108:746–56.

26. El-Hajj Fuleihan G, Chakhtoura M, Cauley JA, Chamoun N. Worldwide fracture prediction. J Clin Densitom. 2017; 20:397–424.

27. Han X, Zhang Y, Shao Y. On comparing 2 correlated C indices with censored survival data. Stat Med. 2017; 36:4041–9.

28. Ghosh S, Raja’S A, Chaudhary V, Dhillon G. Automatic lumbar vertebra segmentation from clinical CT for wedge compression fracture diagnosis. In : SPIE Medical Imaging 2011; Computer-Aided Diagnosis. 2011 Feb 15; Orlando, FL: https://doi.org/10.1117/12.878055.

29. Wang Y, Yao J, Burns JE, Summers R. Osteoporotic and neoplastic compression fracture classification on longitudinal CT. In : Proceedings of the 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI); 2016 Apr 13-16; Prague, CZ. Piscataway, NJ: IEEE;2016. p. 1181–4.

30. Bar A, Wolf L, Amitai OB, Toledano E, Elnekave E. Compression fractures detection on CT. In : SPIE Medical Imaging 2017: Computer-Aided Diagnosis; 2017 Feb 13-16; Orlando, FL. https://doi.org/10.1117/12.2249635.

31. Tomita N, Cheung YY, Hassanpour S. Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Comput Biol Med. 2018; 98:8–15.

32. Muehlematter UJ, Mannil M, Becker AS, Vokinger KN, Finkenstaedt T, Osterhoff G, et al. Vertebral body insufficiency fractures: detection of vertebrae at risk on standard CT images using texture analysis and machine learning. Eur Radiol. 2019; 29:2207–17.

33. Tecle N, Teitel J, Morris MR, Sani N, Mitten D, Hammert WC. Convolutional neural network for second metacarpal radiographic osteoporosis screening. J Hand Surg Am. 2020; 45:175–81.

34. Yamamoto N, Sukegawa S, Kitamura A, Goto R, Noda T, Nakano K, et al. Deep learning for osteoporosis classification using hip radiographs and patient clinical covariates. Biomolecules. 2020; 10:1534.

35. Su Y, Kwok T, Cummings SR, Yip B, Cawthon PM. Can classification and regression tree analysis help identify clinically meaningful risk groups for hip fracture prediction in older American men (the MrOS cohort study)? JBMR Plus. 2019; 3:e10207.

36. Kong SH, Ahn D, Kim BR, Srinivasan K, Ram S, Kim H, et al. A novel fracture prediction model using machine learning in a community-based cohort. JBMR Plus. 2020; 4:e10337.

37. Engels A, Reber KC, Lindlbauer I, Rapp K, Buchele G, Klenk J, et al. Osteoporotic hip fracture prediction from risk factors available in administrative claims data: a machine learning approach. PLoS One. 2020; 15:e0232969.

38. Kalmet P, Sanduleanu S, Primakov S, Wu G, Jochems A, Refaee T, et al. Deep learning in fracture detection: a narrative review. Acta Orthop. 2020; 91:215–20.