Big Imaging Data: Klinische Bildanalyse mit Radiomics und Deep Learning

Zusammenfassung

Radiomics ist eine Methode der medizinischen Bildanalyse, bei der quantitative Merkmale aus Bilddaten extrahiert und mittels Machine Learning zu prädiktiven Modellen weiterverarbeitet werden. Ziel dieser Arbeit ist es, die technischen Grundlagen von Radiomics und mögliche klinische Anwendungen unter besonderer Berücksichtigung nuklearmedizinischer Daten zu erläutern. Dabei wird zunächst die klassische Radiomics-Methode besprochen, welche auf einer exakten Segmentierung der zu analysierenden Pathologie beruht und bei der die Features manuell definiert werden müssen. Anschließend wird auf das noch wenig verbreitete, allerdings vielversprechende Deep Learning basierte Radiomics eingegangen, dessen Vorteile darin liegen, dass ausschließlich datengetrieben gearbeitet wird und daher weder exakte Segmentierungen noch manuelle Definitionen der Features benötigt werden. Abschließend werden einige Anwendungen von Radiomics besprochen, die zukünftig im klinischen Alltag eine Rolle spielen könnten.

Abstract

Radiomics is a method of medical image analysis in which quantitative features are extracted from image data and processed into predictive models using machine learning. The aim of this work is to explain the technical basics of radiomics and possible clinical applications with special emphasis on nuclear medicine imaging data. First, the classical radiomics method is discussed, which is based on an exact segmentation of pathologies and where the features have to be defined manually. The advantages of this method lie in the fact that it is exclusively data-driven and therefore neither exact segmentations nor manual definitions of the features are required. Finally, some applications of Radiomics will be discussed that could play a role in the clinical routine in the future.

Schlüsselwörter

Radiomics - Machine Learning - Deep Learning - Big Data

Keywords

radiomics - machine learning - deep learning - big data

• Literatur

• 1 Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016; 278: 563-577
• 2 Lambin P, Rios-Velazquez E, Leijenaar R. et al. Radiomics: Extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012; 48: 441-446
• 3 Kumar V, Gu Y, Basu S. et al. Radiomics: the process and the challenges. Magn Reson Imaging 2012; 30: 1234-1248
• 4 Harlow CA, Dwyer SJ, Lodwick G. On radiographic image analysis. In: Rosenfeld A. (Ed.) Digital Picture Analysis. Berlin, Heidelberg: Springer; 1976: 65-150
• 5 Aerts HJWL, Velazquez ER, Leijenaar RTH. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5: 4006 Im Internet: http://www.nature.com/articles/ncomms5006
• 6 Lambin P, Leijenaar RTH, Deist TM. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017; 14: 749-762
• 7 Yip SSF, Parmar C, Kim J. et al. Impact of experimental design on PET radiomics in predicting somatic mutation status. Eur J Radiol 2017; 97: 8-15
• 8 Vezhnevets V, Konouchine V. GrowCut: Interactive multi-label ND image segmentation by cellular automata. In: proc. of Graphicon. Citeseer 2005: 150-156
• 9 Morar A, Moldoveanu F, Gröller E. Image segmentation based on active contours without edges. In: IEEE 8th International Conference on Intelligent Computer Communication and Processing 2012: 213-220
• 10 Erdi YE, Mawlawi O, Larson SM. et al. Segmentation of lung lesion volume by adaptive positron emission tomography image thresholding. Cancer 1997; 80: 2505-2509
• 11 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. ArXiv150504597 Cs 2015 Im Internet: http://arxiv.org/abs/1505.04597
• 12 Çiçek Ö, Abdulkadir A, Lienkamp SS. et al. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In: Ourselin S, Joskowicz L, Sabuncu MR. et al. (Hrsg.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Cham: Springer International Publishing; 2016: 424-432 Im Internet: http://link.springer.com/10.1007/978-3-319-46723-8_49
• 13 Isensee F, Kickingereder P, Wick W. et al. Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge. ArXiv180210508 Cs 2018 Im Internet: http://arxiv.org/abs/1802.10508
• 14 Zwanenburg A, Leger S, Vallières M. et al. Initiative for the IBS. Image biomarker standardisation initiative. ArXiv161207003 Cs 2016 Im Internet: http://arxiv.org/abs/1612.07003
• 15 Leijenaar RTH, Carvalho S, Velazquez ER. et al. Stability of FDG-PET Radiomics features: An integrated analysis of test-retest and inter-observer variability. Acta Oncol 2013; 52: 1391-1397
• 16 Galavis PE, Hollensen C, Jallow N. et al. Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters. Acta Oncol 2010; 49: 1012-1016
• 17 Tixier F, Hatt M, Le Rest CC. et al. Reproducibility of Tumor Uptake Heterogeneity Characterization Through Textural Feature Analysis in 18F-FDG PET. J Nucl Med 2012; 53: 693-700
• 18 Hatt M, Tixier F, Cheze Le Rest C. et al. Robustness of intratumour ¹⁸F-FDG PET uptake heterogeneity quantification for therapy response prediction in oesophageal carcinoma. Eur J Nucl Med Mol Imaging 2013; 40: 1662-1671
• 19 Leijenaar RTH, Nalbantov G, Carvalho S. et al. The effect of SUV discretization in quantitative FDG-PET Radiomics: the need for standardized methodology in tumor texture analysis. Sci Rep 2015; 5: 11075 Im Internet: http://www.nature.com/articles/srep11075
• 20 Orlhac F, Soussan M, Chouahnia K. et al. 18F-FDG PET-Derived Textural Indices Reflect Tissue-Specific Uptake Pattern in Non-Small Cell Lung Cancer. PLOS ONE 2015; 10: e0145063
• 21 Yan J, Chu-Shern JL, Loi HY. et al. Impact of Image Reconstruction Settings on Texture Features in 18F-FDG PET. J Nucl Med 2015; 56: 1667-1673
• 22 Doumou G, Siddique M, Tsoumpas C. et al. The precision of textural analysis in 18F-FDG-PET scans of oesophageal cancer. Eur Radiol 2015; 25: 2805-2812
• 23 Lu L, Lv W, Jiang J. et al. Robustness of Radiomic Features in [11C]Choline and [18F]FDG PET/CT Imaging of Nasopharyngeal Carcinoma: Impact of Segmentation and Discretization. Mol Imaging Biol 2016; 18: 935-945
• 24 Altazi BA, Zhang GG, Fernandez DC. et al. Reproducibility of F18-FDG PET radiomic features for different cervical tumor segmentation methods, gray-level discretization, and reconstruction algorithms. J Appl Clin Med Phys 2017; 18: 32-48
• 25 Reuzé S, Orlhac F, Chargari C. et al. Prediction of cervical cancer recurrence using textural features extracted from ¹⁸F-FDG PET images acquired with different scanners. Oncotarget 2017; 8: 43169-43179 Im Internet: http://www.oncotarget.com/fulltext/17856
• 26 Bellman R. Adaptive Control Processes: A Guided Tour. Princeton, N. J.: Princeton University Press; 1961
• 27 Hanchuan P, Fuhui L, Ding C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 2005; 27: 1226-1238
• 28 Guyon I, Elisseeff A. An Introduction to Variable and Feature Selection. Journal of machine learning research 2003; 3: 1157-1182
• 29 Guyon I, Weston J, Barnhill S. et al. Gene Selection for Cancer Classification using Support Vector Machines. Machine learning 2002; 46: 389-422
• 30 Meinshausen N, Bühlmann P. Stability selection. J R Stat Soc Ser B Stat Methodol 2010; 72: 417-473
• 31 Tibshirani R. Regression Shrinkage and Selection via the Lasso. J R Stat Soc Ser B Methodol 1996; 58: 267-288
• 32 Chen Y-W, Lin C-J. Combining SVMs with Various Feature Selection Strategies. In: Guyon I, Nikravesh M, Gunn S. et al. (Hrsg.) Feature Extraction. Berlin, Heidelberg: Springer; 2006: 315-324 Im Internet: http://link.springer.com/10.1007/978-3-540-35488-8_13
• 33 Haury A-C, Gestraud P, Vert J-P. The Influence of Feature Selection Methods on Accuracy, Stability and Interpretability of Molecular Signatures. PLOS ONE 2011; 6: e28210
• 34 Bair E, Hastie T, Paul D. et al. Prediction by Supervised Principal Components. J Am Stat Assoc 2006; 101: 119-137
• 35 Vincent P, Larochelle H, Lajoie I. et al. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of machine learning research 2010; 11: 3371-3408
• 36 Ishwaran H, Kogalur UB, Blackstone EH. et al. Random survival forests. Ann Appl Stat 2008; 2: 841-860
• 37 Cox DR. Regression Models and Life-Tables. J R Stat Soc Ser B Methodol 1972; 34: 187-220
• 38 Chawla NV, Bowyer KW, Hall LO. et al. SMOTE: Synthetic Minority Over-sampling Technique. J Artif Intell Res 2002; 16: 321-357
• 39 Colby JB. Radiomics Approach Fails to Outperform Null Classifier on Test Data. Am J Neuroradiol 2017; 38: E92-E93
• 40 Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: a comparison of resampling methods. Bioinformatics 2005; 21: 3301-3307
• 41 Subramanian J, Simon R. An evaluation of resampling methods for assessment of survival risk prediction in high-dimensional settings. Stat Med 2011; 30: 642-653
• 42 Venet D, Dumont JE, Detours V. Most Random Gene Expression Signatures Are Significantly Associated with Breast Cancer Outcome. PLoS Comput Biol 2011; 7: e1002240
• 43 Demšar J. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of machine learning research 2006; 7: 1-30
• 44 Holm S. A Simple Sequentially Rejective Multiple Test Procedure. Scand J Stat 1979; 6: 65-70
• 45 Rosenblatt F. The perceptron: A probabilistic model for information storage and organization in the brain. Psychol Rev 1958; 65: 386-408
• 46 McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. 1943. Bull Math Biol 1990; 52: 99-115 ; discussion 73-97
• 47 Rumelhart DE, Hinton GE, Williams RJ. Learning Internal Representation by Error Propagation. In: Rumelhart DE, McClelland JL. , and the PDP Research Group (Eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. Cambridge, MA, USA: MIT Press; 1986: 318-362 Im Internet: http://dl.acm.org/citation.cfm?id=104279.104293
• 48 Hinton GE, Osindero S, Teh Y-W. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput 2006; 18: 1527-1554
• 49 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444
• 50 Schmidhuber J. Deep Learning in Neural Networks: An Overview. Neural Netw 2015; 61: 85-117
• 51 Ravi D, Wong C, Deligianni F. et al. Deep Learning for Health Informatics. IEEE J Biomed Health Inform 2017; 21: 4-21
• 52 Lao J, Chen Y, Li Z-C. et al. A Deep Learning-Based Radiomics Model for Prediction of Survival in Glioblastoma Multiforme. Sci Rep 2017; 7: 10353 Im Internet: http://www.nature.com/articles/s41598-017-10649-8
• 53 Hubel DH, Wiesel TN. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 1962; 160: 106-154
• 54 Russakovsky O, Deng J, Su H. et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 2015; 115: 211-252
• 55 Selvaraju RR, Cogswell M, Das A. et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In: IEEE International Conference on Computer Vision (ICCV) Venice: IEEE; 2017: 618-626 Im Internet: http://ieeexplore.ieee.org/document/8237336/
• 56 Yune S, Lee H, Kim M. et al. Beyond Human Perception: Sexual Dimorphism in Hand and Wrist Radiographs Is Discernible by a Deep Learning Model. J Digit Imaging 2018; Im Internet: http://link.springer.com/10.1007/s10278-018-0148-x
• 57 Cheng H-T, Ispir M, Anil R. et al. Wide & Deep Learning for Recommender Systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems - DLRS 2016. Boston, MA, USA: ACM Press; 2016: 7-10 Im Internet: http://dl.acm.org/citation.cfm?doid=2988450.2988454
• 58 Emaminejad N, Qian W, Guan Y. et al. Fusion of Quantitative Image and Genomic Biomarkers to Improve Prognosis Assessment of Early Stage Lung Cancer Patients. IEEE Trans Biomed Eng 2016; 63: 1034-1043
• 59 Hu T, Wang S, Huang L. et al. A clinical-radiomics nomogram for the preoperative prediction of lung metastasis in colorectal cancer patients with indeterminate pulmonary nodules. Eur Radiol 2019; 29: 439-449
• 60 Zhang C, Ma Y. (Hrsg.) Ensemble Machine Learning. New York: Springer; 2012
• 61 Wu J, Aguilera T, Shultz D. et al. Early-Stage Non-Small Cell Lung Cancer: Quantitative Imaging Characteristics of ¹⁸F Fluorodeoxyglucose PET/CT Allow Prediction of Distant Metastasis. Radiology 2016; 281: 270-278
• 62 Oikonomou A, Khalvati F, Tyrrell PN. et al. Radiomics analysis at PET/CT contributes to prognosis of recurrence and survival in lung cancer treated with stereotactic body radiotherapy. Sci Rep 2018; 8: 4003 Im Internet: http://www.nature.com/articles/s41598-018-22357-y
• 63 Yoon HJ, Sohn I, Cho JH. et al. Decoding Tumor Phenotypes for ALK, ROS1, and RET Fusions in Lung Adenocarcinoma Using a Radiomics Approach. Medicine (Baltimore) 2015; 94: e1753
• 64 Vallières M, Freeman CR, Skamene SR. et al. A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 2015; 60: 5471-5496
• 65 Wang H, Zhou Z, Li Y. et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images. EJNMMI Res 2017; 7: 11 Im Internet: http://ejnmmires.springeropen.com/articles/10.1186/s13550-017-0260-9
• 66 Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira F, Burges CJC, Bottou L. et al. (Hrsg.) Advances in Neural Information Processing Systems 25. Curran Associates, Inc. 2012: 1097-1105 Im Internet: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
• 67 Kirienko M, Cozzi L, Antunovic L. et al. Prediction of disease-free survival by the PET/CT radiomic signature in non-small cell lung cancer patients undergoing surgery. Eur J Nucl Med Mol Imaging 2018; 45: 207-217
• 68 Milgrom SA, Elhalawani H, Lee J. et al. A PET Radiomics Model to Predict Refractory Mediastinal Hodgkin Lymphoma. Sci Rep 2019; 9: 1322 Im Internet: http://www.nature.com/articles/s41598-018-37197-z
• 69 Mu W, Chen Z, Liang Y. et al. Staging of cervical cancer based on tumor heterogeneity characterized by texture features on ¹⁸F-FDG PET images. Phys Med Biol 2015; 60: 5123-5139
• 70 Lucia F, Visvikis D, Desseroit M-C. et al. Prediction of outcome using pretreatment ¹⁸F-FDG PET/CT and MRI radiomics in locally advanced cervical cancer treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging 2018; 45: 768-786
• 71 Lucia F, Visvikis D, Vallières M. et al. External validation of a combined PET and MRI radiomics model for prediction of recurrence in cervical cancer patients treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging 2018; 46: 864-877 Im Internet: http://link.springer.com/10.1007/s00259-018-4231-9
• 72 Tsujikawa T, Rahman T, Yamamoto M. et al. ¹⁸F-FDG PET radiomics approaches: comparing and clustering features in cervical cancer. Ann Nucl Med 2017; 31: 678-685
• 73 Ha S, Park S, Bang J-I. et al. Metabolic Radiomics for Pretreatment ¹⁸F-FDG PET/CT to Characterize Locally Advanced Breast Cancer: Histopathologic Characteristics, Response to Neoadjuvant Chemotherapy, and Prognosis. Sci Rep 2017; 7: 1556 Im Internet: http://www.nature.com/articles/s41598-017-01524-7
• 74 Antunovic L, Gallivanone F, Sollini M. et al. [18F]FDG PET/CT features for the molecular characterization of primary breast tumors. Eur J Nucl Med Mol Imaging 2017; 44: 1945-1954
• 75 Zhou H, Jiang J, Lu J. et al. Dual-Model Radiomic Biomarkers Predict Development of Mild Cognitive Impairment Progression to Alzheimer’s Disease. Front Neurosci 2019; 12: 1045 Im Internet: https://www.frontiersin.org/article/10.3389/fnins.2018.01045/full
• 76 Spasov S, Passamonti L, Duggento A. et al. A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s disease. NeuroImage 2019; 189: 276-287
• 77 Jiang Y, Yuan Q, Lv W. et al. Radiomic signature of ¹⁸F fluorodeoxyglucose PET/CT for prediction of gastric cancer survival and chemotherapeutic benefits. Theranostics 2018; 8: 5915-5928
• 78 Vallières M, Kay-Rivest E, Perrin LJ. et al. Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. ArXiv170308516 Cs 2017 Im Internet: http://arxiv.org/abs/1703.08516
• 79 Chen KT, Gong E, de Carvalho Macruz FB. et al. Ultra–Low-Dose ¹⁸F-Florbetaben Amyloid PET Imaging Using Deep Learning with Multi-Contrast MRI Inputs. Radiology 2019; 290: 649-656
• 80 Schwyzer M, Ferraro DA, Muehlematter UJ. et al. Automated detection of lung cancer at ultralow dose PET/CT by deep neural networks – Initial results. Lung Cancer 2018; 126: 170-173
• 81 Zhao X, Li L, Lu W. et al. Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network. Phys Med Biol 2018; 64: 015011
• 82 Zhong Z, Kim Y, Plichta K. et al. Simultaneous Cosegmentation of Tumors in PET-CT Images Using Deep Fully Convolutional Networks. Med Phys 2019; 46: 619-633 Im Internet: https://onlinelibrary.wiley.com/doi/abs/10.1002/mp.13331
• 83 Baek S, He Y, Allen BG. et al. What does AI see? Deep segmentation networks discover biomarkers for lung cancer survival.. arXiv:1903.11593 2019 Im Internet: http://arxiv.org/abs/1903.11593