Pattern Recognition in Musculoskeletal Imaging Using Artificial Intelligence

Natalia Gorelik; Jaron Chong; Dana J. Lin

doi:10.1055/s-0039-3400266

Seminars in Musculoskeletal Radiology, Table of Contents

Semin Musculoskelet Radiol 2020; 24(01): 38-49
DOI: 10.1055/s-0039-3400266

Review Article

Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Pattern Recognition in Musculoskeletal Imaging Using Artificial Intelligence

Authors

Natalia Gorelik

¹Department of Diagnostic Radiology, McGill University Health Center, Montreal, Quebec, Canada
Jaron Chong

¹Department of Diagnostic Radiology, McGill University Health Center, Montreal, Quebec, Canada
Dana J. Lin

²Division of Musculoskeletal Radiology, Department of Radiology, NYU Langone Health, New York, New York

Abstract

Artificial intelligence (AI) has the potential to affect every step of the radiology workflow, but the AI application that has received the most press in recent years is image interpretation, with numerous articles describing how AI can help detect and characterize abnormalities as well as monitor disease response. Many AI-based image interpretation tasks for musculoskeletal (MSK) pathologies have been studied, including the diagnosis of bone tumors, detection of osseous metastases, assessment of bone age, identification of fractures, and detection and grading of osteoarthritis. This article explores the applications of AI for image interpretation of MSK pathologies.

Keywords

musculoskeletal - deep learning - neural network

Full Text

References

References
1 Choy G, Khalilzadeh O, Michalski M. , et al. Current applications and future impact of machine learning in radiology. Radiology 2018; 288 (02) 318-328
2 European Society of Radiology (ESR). What the radiologist should know about artificial intelligence—an ESR white paper. Insights Imaging 2019; 10 (01) 44
3 Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Artificial intelligence in musculoskeletal imaging: current status and future directions. AJR Am J Roentgenol 2019; 213 (03) 506-513
4 Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18 (08) 500-510
5 Tang A, Tam R, Cadrin-Chênevert A. , et al; Canadian Association of Radiologists (CAR) Artificial Intelligence Working Group. Canadian Association of Radiologists white paper on artificial intelligence in radiology. Can Assoc Radiol J 2018; 69 (02) 120-135
6 Mutasa S, Chang PD, Ruzal-Shapiro C, Ayyala R. MABAL: a novel deep-learning architecture for machine-assisted bone age labeling. J Digit Imaging 2018; 31 (04) 513-519
7 Ping YY, Yin CW, Kok LP. Computer Aided Bone Tumor Detection and Classification Using X-ray Images. Berlin, Germany: Springer; 2008
8 Hammon M, Dankerl P, Tsymbal A. , et al. Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur Radiol 2013; 23 (07) 1862-1870
9 Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 2018; 287 (01) 313-322
10 Chung SW, Han SS, Lee JW. , et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 2018; 89 (04) 468-473
11 Antony J, McGuinness K, O'Connor NE, Moran K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. Paper presented at: 23rd International Conference on Pattern Recognition (ICPR); December 4–8, 2016; Cancún, Mexico
12 Syed AB, Zoga AC. Artificial intelligence in radiology: current technology and future directions. Semin Musculoskelet Radiol 2018; 22 (05) 540-545
13 The Lodwick Award. Massachusetts General Hospital. www.massgeneral.org/imaging/about/lodwick-award.aspx . Accessed June 1, 2019
14 Lodwick GS, Haun CL, Smith WE, Keller RF, Robertson ED. Computer diagnosis of primary bone tumors: a preliminary report. Radiology 1963; 80: 273-275
15 Kahn Jr CE, Laur JJ, Carrera GF. A Bayesian network for diagnosis of primary bone tumors. J Digit Imaging 2001; 14 (02) (Suppl. 01) 56-57
16 Lejbkowicz I, Wiener F, Nachtigal A, Militiannu D, Kleinhaus U, Applbaum YH. Bone Browser a decision-aid for the radiological diagnosis of bone tumors. Comput Methods Programs Biomed 2002; 67 (02) 137-154
17 Do BH, Langlotz C, Beaulieu CF. Bone tumor diagnosis using a naïve Bayesian model of demographic and radiographic features. J Digit Imaging 2017; 30 (05) 640-647
18 Reinus WR, Wilson AJ, Kalman B, Kwasny S. Diagnosis of focal bone lesions using neural networks. Invest Radiol 1994; 29 (06) 606-611
19 Bandyopadhyay O, Biswas A, Bhattacharya BB. Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging 2019; 32 (02) 300-313
20 Kumar R, Suhas MV. Classification of benign and malignant bone lesions on CT images using support vector machine: A comparison of kernel functions. Paper presented at: IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology; May 20–21, 2016; Bangalore, India
21 Mishra A, Suhas MV. Classification of benign and malignant bone lesions on CT images using Random Forest. Paper presented at: IEEE International Conference on Recent Trends in Electronics Information Communication Technology; May 20–21, 2016; Bangalore, India
22 Banerjee I, Kurtz C, Devorah AE, Do B, Rubin DL, Beaulieu CF. Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: application to bone tumor radiographs. J Biomed Inform 2018; 84: 123-135
23 Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM. Automated detection of sclerotic metastases in the thoracolumbar spine at CT. Radiology 2013; 268 (01) 69-78
24 Wiese T, Yao J, Burns JE, Summers RM. Detection of sclerotic bone metastases in the spine using watershed algorithm and graph cut. Proc. SPIE 8315, Medical Imaging 2012: Computer-Aided Diagnosis, 831512, February 23, 2012; https://doi.org/10.1117/12.911700
25 Roth HR, Lu L, Liu J. , et al. Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 2016; 35 (05) 1170-1181
26 Yao J, O'Connor SD, Summers R. Computer aided lytic bone metastasis detection using regular CT images. Proc. SPIE 6144, Medical Imaging 2006: Image Processing, 614459, March 15, 2006; https://doi.org/10.1117/12.652288
27 O'Connor SD, Yao J, Summers RM. Lytic metastases in thoracolumbar spine: computer-aided detection at CT--preliminary study. Radiology 2007; 242 (03) 811-816
28 Wels M, Kelm BM, Tsymbal A. , et al. Multi-stage osteolytic spinal bone lesion detection from CT data with internal sensitivity control. Proc. SPIE 8315, Medical Imaging 2012: Computer-Aided Diagnosis, 831513, February 23, 2012; https://doi.org/10.1117/12.911169
29 Huang SF, Chiang KH. Automatic detection of bone metastasis in vertebrae by using CT images. Paper presented at: Proceedings of the World Congress on Engineering; July 4–6, 2012; London, UK
30 Yao J, Burns JE, Sanoria V, Summers RM. Mixed spine metastasis detection through positron emission tomography/computed tomography synthesis and multiclassifier. J Med Imaging (Bellingham) 2017; 4 (02) 024504
31 Yao J, Burns JE, Summers RM. Sclerotic rib metastases detection on routine CT images. Paper presented at: 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI); May 2, 2012; Barcelona, Spain
32 Oh J, Kim G, Lee J. , et al. Automated detection of bone metastatic changes using serial CT scans. Comput Med Imaging Graph 2017; 58: 62-74
33 Sakamoto R, Yakami M, Fujimoto K. , et al. Temporal subtraction of serial CT images with large deformation diffeomorphic metric mapping in the identification of bone metastases. Radiology 2017; 285 (02) 629-639
34 Ueno M, Aoki T, Murakami S. , et al. CT temporal subtraction method for detection of sclerotic bone metastasis in the thoracolumbar spine. Eur J Radiol 2018; 107: 54-59
35 Martínez-Martínez F, Kybic J, Lambert L, Mecková Z. Fully automated classification of bone marrow infiltration in low-dose CT of patients with multiple myeloma based on probabilistic density model and supervised learning. Comput Biol Med 2016; 71: 57-66
36 Jerebko AK, Schmidt GP, Zhou X. , et al. Robust parametric modeling approach based on domain knowledge for computer aided detection of vertebrae column metastases in MRI. Inf Process Med Imaging 2007; 20: 713-724
37 Perk T, Bradshaw T, Chen S. , et al. Automated classification of benign and malignant lesions in ¹⁸F-NaF PET/CT images using machine learning. Phys Med Biol 2018; 63 (22) 225019
38 Bradshaw T, Perk T, Chen S. , et al. Deep learning for classification of benign and malignant bone lesions in [F-18] NaF PET/CT images. J Nucl Med 2018; 59 (Suppl. 01) 327-327
39 Xu L, Tetteh G, Lipkova J. , et al. Automated whole-body bone lesion detection for multiple myeloma on ⁶⁸Ga-pentixafor PET/CT imaging using deep learning methods. Contrast Media Mol Imaging 2018; 2018: 2391925
40 Greulich WW, Pyle SI. Radiographic Atlas of Skeletal Development of the Hand and Wrist. 2nd ed. Stanford, CA:: Stanford University Press;; 1959
41 Tanner JM, Whitehouse R, Cameron N, Marshall W, Healy M, Goldstein H. Assessment of Skeletal Maturity and Prediction of Adult Height (TW2 Method). Vol 16. London, UK:: Academic Press;; 1975
42 Tajmir SH, Lee H, Shailam R. , et al. Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skeletal Radiol 2019; 48 (02) 275-283
43 Tanner JM, Gibbons RD. A computerized image analysis system for estimating Tanner-Whitehouse 2 bone age. Horm Res 1994; 42 (06) 282-287
44 Michael DJ, Nelson AC. HANDX: a model-based system for automatic segmentation of bones from digital hand radiographs. IEEE Trans Med Imaging 1989; 8 (01) 64-69
45 Lee H, Tajmir S, Lee J. , et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017; 30 (04) 427-441
46 Pietka E, McNitt-Gray MF, Kuo ML, Huang HK. Computer-assisted phalangeal analysis in skeletal age assessment. IEEE Trans Med Imaging 1991; 10 (04) 616-620
47 Tanner JM, Gibbons RD. Automatic bone age measurement using computerized image analysis. J Pediatr Endocrinol Metab 1994; 7 (02) 141-145
48 Thodberg HH, Kreiborg S, Juul A, Pedersen KD. The BoneXpert method for automated determination of skeletal maturity. IEEE Trans Med Imaging 2009; 28 (01) 52-66
49 van Rijn RR, Thodberg HH. Bone age assessment: automated techniques coming of age?. Acta Radiol 2013; 54 (09) 1024-1029
50 Zhang J, Lin F, Ding X. Maturation disparity between hand-wrist bones in a chinese sample of normal children: an analysis based on automatic BoneXpert and manual Greulich and Pyle atlas assessment. Korean J Radiol 2016; 17 (03) 435-442
51 Halabi SS, Prevedello LM, Kalpathy-Cramer J. , et al. The RSNA pediatric bone age machine learning challenge. Radiology 2019; 290 (02) 498-503
52 Kim JR, Shim WH, Yoon HM. , et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol 2017; 209 (06) 1374-1380
53 Zhang A, Sayre JW, Vachon L, Liu BJ, Huang HK. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology 2009; 250 (01) 228-235
54 Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 2018; 73 (05) 439-445
55 Lindsey R, Daluiski A, Chopra S. , et al. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A 2018; 115 (45) 11591-11596
56 Cheng CT, Ho TY, Lee TY. , et al. Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 2019; 29 (10) 5469-5477
57 Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N. Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skeletal Radiol 2019; 48 (02) 239-244
58 Gale W, Oakden-Rayner L, Carneiro G, Bradley A, Palmer L. Detecting hip fractures with radiologist-level performance using deep neural networks. 2017 . Abstract available at: arxiv.org/abs/1711.06504
59 Kitamura G, Chung CY, Moore II BE. Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation. J Digit Imaging 2019; 32 (04) 672-677
60 Olczak J, Fahlberg N, Maki A. , et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop 2017; 88 (06) 581-586
61 Pranata YD, Wang KC, Wang JC. , et al. Deep learning and SURF for automated classification and detection of calcaneus fractures in CT images. Comput Methods Programs Biomed 2019; 171: 27-37
62 Kasai S, Li F, Shiraishi J, Doi K. Usefulness of computer-aided diagnosis schemes for vertebral fractures and lung nodules on chest radiographs. AJR Am J Roentgenol 2008; 191 (01) 260-265
63 Burns JE, Yao J, Summers RM. Vertebral body compression fractures and bone density: automated detection and classification on CT images. Radiology 2017; 284 (03) 788-797
64 Tomita N, Cheung YY, Hassanpour S. Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Comput Biol Med 2018; 98: 8-15
65 Mehta SD, Sebro R. Computer-aided detection of incidental lumbar spine fractures from routine dual-energy X-ray absorptiometry (DEXA) studies using a support vector machine (SVM) classifier. J Digit Imaging 2019 ; May 6 (Epub ahead of print)
66 Badgeley MA, Zech JR, Oakden-Rayner L. , et al. Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med 2019; 2: 31
67 Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med 2018; 15 (11) e1002683
68 Yi PH, Kim TK, Wei J. , et al. Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning. Pediatr Radiol 2019; 49 (08) 1066-1070
69 Jamaludin A, Lootus M, Kadir T. , et al; Genodisc Consortium. ISSLS PRIZE IN BIOENGINEERING SCIENCE 2017: Automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J 2017; 26 (05) 1374-1383
70 Xue Y, Zhang R, Deng Y, Chen K, Jiang T. A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis. PLoS One 2017; 12 (06) e0178992
71 Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S. Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 2018; 8 (01) 1727
72 Norman B, Pedoia V, Noworolski A, Link TM, Majumdar S. Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J Digit Imaging 2019; 32 (03) 471-477