Artificial Intelligence in Oncological Hybrid Imaging

Benedikt Feuerecker; Maurice M. Heimer; Thomas Geyer; Matthias P Fabritius; Sijing Gu; Balthasar Schachtner; Leonie Beyer; Jens Ricke; Sergios Gatidis; Michael Ingrisch; Clemens C Cyran

doi:10.1055/a-1909-7013

RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren, Table of Contents

Rofo 2023; 195(02): 105-114
DOI: 10.1055/a-1909-7013

Review

Artificial Intelligence in Oncological Hybrid Imaging

Künstliche Intelligenz in der onkologischen Hybridbildgebung

Benedikt Feuerecker^*

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

²German Cancer Research Center (DKFZ), Partner site Munich, DKTK German Cancer Consortium, Munich, Germany

,

Maurice M. Heimer^*

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Thomas Geyer

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Matthias P Fabritius

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Sijing Gu

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Balthasar Schachtner

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Leonie Beyer

³Department of Nuclear Medicine, University Hospital, LMU Munich, Munich, Germany

,

Jens Ricke

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Sergios Gatidis

⁴Department of Radiology, University Hospital Tübingen, Tübingen, Germany

⁵MPI, Max Planck Institute for Intelligent Systems, Tübingen, Germany

,

Michael Ingrisch

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

,

Clemens C Cyran

¹Department of Radiology, University Hospital, LMU Munich, Munich, Germany

› Author Affiliations

Abstract

Background Artificial intelligence (AI) applications have become increasingly relevant across a broad spectrum of settings in medical imaging. Due to the large amount of imaging data that is generated in oncological hybrid imaging, AI applications are desirable for lesion detection and characterization in primary staging, therapy monitoring, and recurrence detection. Given the rapid developments in machine learning (ML) and deep learning (DL) methods, the role of AI will have significant impact on the imaging workflow and will eventually improve clinical decision making and outcomes.

Methods and Results The first part of this narrative review discusses current research with an introduction to artificial intelligence in oncological hybrid imaging and key concepts in data science. The second part reviews relevant examples with a focus on applications in oncology as well as discussion of challenges and current limitations.

Conclusion AI applications have the potential to leverage the diagnostic data stream with high efficiency and depth to facilitate automated lesion detection, characterization, and therapy monitoring to ultimately improve quality and efficiency throughout the medical imaging workflow. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based therapy guidance in oncology. However, significant challenges remain regarding application development, benchmarking, and clinical implementation.

Key Points:

Hybrid imaging generates a large amount of multimodality medical imaging data with high complexity and depth.
Advanced tools are required to enable fast and cost-efficient processing along the whole radiology value chain.
AI applications promise to facilitate the assessment of oncological disease in hybrid imaging with high quality and efficiency for lesion detection, characterization, and response assessment. The goal is to generate reproducible, structured, quantitative diagnostic data for evidence-based oncological therapy guidance.
Selected applications in three oncological entities (lung, prostate, and neuroendocrine tumors) demonstrate how AI algorithms may impact imaging-based tasks in hybrid imaging and potentially guide clinical decision making.

Citation Format

Feuerecker B, Heimer M, Geyer T et al. Artificial Intelligence in Oncological Hybrid Imaging. Fortschr Röntgenstr 2023; 195: 105 – 114

Zusammenfassung

Hintergrund Der Stellenwert künstlicher Intelligenz (KI) hat in der medizinischen Bildgebung in den letzten Jahren deutlich zugenommen. Aufgrund der enormen Datenmengen und strukturierbaren Aufgaben im diagnostischen Workflow hat die KI in der onkologischen Hybridbildgebung besonders vielversprechende Anwendungsgebiete für die Läsionsdetektion, die Läsionscharakterisierung und die Therapiebeurteilung. Vor dem Hintergrund rasanter Entwicklungen im Bereich des Machine Learning (ML) und des Deep Learning (DL) ist von einer zunehmenden Bedeutung in der onkologischen Hybridbildgebung auszugehen mit Potenzial, die klinische Therapiesteuerung und patientenrelevante Ergebnisse zu verbessern.

Methode und Ergebnisse Diese narrative Übersichtsarbeit fasst die Evidenz in verschiedenen aufgabenbezogenen Anwendungen der Bildanalyse von KI im Bereich der onkologischen Hybridbildgebung zusammen. Nach Einführung in das Thema der KI werden ausgewählte Beispiele exploriert, vor dem Hintergrund aktueller Herausforderungen und im Hinblick auf die klinische Relevanz in der Therapiesteuerung diskutiert.

Schlussfolgerung Der Einsatz von KI bietet vielversprechende Anwendungen der Detektion, der Charakterisierung und der longitudinalen Therapiebeurteilung im Bereich der onkologischen Hybridbildgebung. Schlüsselherausforderungen liegen in den Bereichen der Entwicklung von Algorithmen, der Validierung und der klinischen Implementierung.

Kernaussagen:

Mit der onkologischen Hybridbildgebung werden große Datenvolumen aus 2 bildgebenden Modalitäten erzeugt, deren strukturierte Analyse komplex ist.
Für die Datenanalyse werden neue Methoden benötigt, um eine schnelle und kosteneffiziente Beurteilung in allen Aspekten der diagnostischen Wertschöpfungskette zu ermöglichen.
KI verspricht, die diagnostische Auswertung der onkologischen Hybridbildgebung zu vereinfachen und wesentliche Verbesserungen in Qualität und Effizienz bei der Erkennung, Charakterisierung und dem longitudinalen Monitoring onkologischer Erkrankungen zu ermöglichen. Ziel ist reproduzierbare, strukturierte, quantitative diagnostische Daten für die evidenzbasierte onkologische Therapiesteuerung zu generieren.
Selektierte Anwendungsbeispiele in 3 ausgewählten Tumorentitäten (Lungenkarzinom, Prostatakarzinom, Neuroendokrine Tumore) zeigen wie KI-gestützte Applikationen einen wesentlichen Beitrag in der automatisierten Bildanalyse leisten und eine weitere Individualisierung von Therapien ermöglichen könnten.

Key words

artifical intelligence - oncological imaging - hybrid imaging

Full Text

References

References
1 Moore JH. et al. Preparing next-generation scientists for biomedical big data: artificial intelligence approaches. Per Med 2019; 16 (03) 247-257
2 Rueckel J. et al. Artificial Intelligence Algorithm Detecting Lung Infection in Supine Chest Radiographs of Critically Ill Patients With a Diagnostic Accuracy Similar to Board-Certified Radiologists. Crit Care Med 2020; 48 (07) e574-e583
3 Oosterhoff JHF, Doornberg JN, Doornberg C. Machine Learning, Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner’s hype cycle. EFORT Open Rev 2020; 5 (10) 593-603
4 Sollini M. et al. Artificial intelligence and hybrid imaging: the best match for personalized medicine in oncology. Eur J Hybrid Imaging 2020; 4 (01) 24
5 Capobianco E. High-dimensional role of AI and machine learning in cancer research. Br J Cancer 2022; 126 (04) 523-532
6 Shiyam Sundar LK. et al. Potenzials and caveats of AI in hybrid imaging. Methods 2021; 188: 4-19
7 Zaharchuk G. Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning. Eur J Nucl Med Mol Imaging 2019; 46 (13) 2700-2707
8 Aide N. et al. New PET technologies – embracing progress and pushing the limits. Eur J Nucl Med Mol Imaging 2021; 48 (09) 2711-2726
9 Hosny A. et al. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18 (08) 500-510
10 Pe’er D. et al. Tumor heterogeneity. Cancer Cell 2021; 39 (08) 1015-1017
11 Ju W. et al. Random Walk and Graph Cut for Co-Segmentation of Lung Tumor on PET-CT Images. IEEE Trans Image Process 2015; 24 (12) 5854-5867
12 Pfannenberg C. et al. Practice-based evidence for the clinical benefit of PET/CT-results of the first oncologic PET/CT registry in Germany. Eur J Nucl Med Mol Imaging 2019; 46 (01) 54-64
13 Mottet N. et al. EAU-EANM-ESTRO-ESUR-SIOG Guidelines on Prostate Cancer-2020 Update. Part 1: Screening, Diagnosis, and Local Treatment with Curative Intent. Eur Urol 2021; 79 (02) 243-262
14 AWMF. Leitlinienprogramm Onkologie (Deutsche Krebsgesellschaft, Deutsche Krebshilfe, AWMF): S3-Leitlinie Prostatakarzinom, Langversion 6.0, 2021, AWMF Registernummer: 043/022OL. (abgerufen am: 02.08.2021) http://www.leitlinienprogramm-onkologie.de/leitlinien/prostatakarzinom/
15 Zhao X. et al. Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network. Phys Med Biol 2018; 64 (01) 015011
16 Bradshaw TJ. et al. Nuclear Medicine and Artificial Intelligence: Best Practices for Algorithm Development. J Nucl Med 2021;
17 Pinto Dos Santos D, Dietzel M, Baessler B. A decade of radiomics research: are images really data or just patterns in the noise?. Eur Radiol 2021; 31 (01) 1-4
18 Gatta R. et al. Integrating radiomics into holomics for personalised oncology: from algorithms to bedside. Eur Radiol Exp 2020; 4 (01) 11
19 Holzinger A. et al. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip Rev Data Min Knowl Discov 2019; 9 (04) e1312
20 Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016; 278 (02) 563-577
21 Sepehri S. et al. Comparison and Fusion of Machine Learning Algorithms for Prospective Validation of PET/CT Radiomic Features Prognostic Value in Stage II-III Non-Small Cell Lung Cancer. Diagnostics (Basel) 2021; 11 (04)
22 Chartrand G. et al. Deep Learning: A Primer for Radiologists. Radiographics 2017; 37 (07) 2113-2131
23 Litjens G. et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88
24 Sanli Y. et al. Neuroendocrine Tumor Diagnosis and Management: (68)Ga-DOTATATE PET/CT. Am J Roentgenol 2018; 211 (02) 267-277
25 El-Galaly TC. et al. FDG-PET/CT in the management of lymphomas: current status and future directions. J Intern Med 2018; 284 (04) 358-376
26 Rakheja R. et al. Necrosis on FDG PET/CT correlates with prognosis and mortality in sarcomas. Am J Roentgenol 2013; 201 (01) 170-177
27 Bettinelli A. et al. A Novel Benchmarking Approach to Assess the Agreement among Radiomic Tools. Radiology 2022; 211604
28 van Timmeren JE. et al. Radiomics in medical imaging-“how-to” guide and critical reflection. Insights Imaging 2020; 11 (01) 91
29 Li K, Zhang R, Cai W. Deep learning convolutional neural network (DLCNN): unleashing the potential of (18)F-FDG PET/CT in lymphoma. Am J Nucl Med Mol Imaging 2021; 11 (04) 327-331
30 Mongan J, Moy L, Kahn Jr CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. Radiol Artif Intell 2020; 2 (02) e200029
31 Lambin P. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017; 14 (12) 749-762
32 Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021; 3 (03) e195-e203
33 Luchini C, Pea A, Scarpa A. Artificial intelligence in oncology: current applications and future perspectives. Br J Cancer 2022; 126 (01) 4-9
34 Sibille L. et al. (18)F-FDG PET/CT Uptake Classification in Lymphoma and Lung Cancer by Using Deep Convolutional Neural Networks. Radiology 2020; 294 (02) 445-452
35 Blanc-Durand P. et al. Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network. Eur J Nucl Med Mol Imaging 2021; 48 (05) 1362-1370
36 Qu YH. et al. The correlation of (18)F-FDG PET/CT metabolic parameters, clinicopathological factors, and prognosis in breast cancer. Clin Transl Oncol 2021; 23 (03) 620-627
37 Lohmann P. et al. PET/MRI Radiomics in Patients With Brain Metastases. Front Neurol 2020; 11: 1
38 Nakajo M. et al. Machine learning based evaluation of clinical and pretreatment (18)F-FDG-PET/CT radiomic features to predict prognosis of cervical cancer patients. Abdom Radiol (NY) 2022; 47 (02) 838-847
39 AWMF. Leitlinienprogramm Onkologie (Deutsche Krebsgesellschaft, Deutsche Krebshilfe, AWMF): Prävention, Diagnostik, Therapie und Nachsorge des Lungenkarzinoms, Langversion 1.0, 2018, AWMF-Registernummer: 020/007OL. 2018 (Zugriff am: 09.04.2022) http://leitlinienprogramm-onkologie.de/Lungenkarzinom.98.0.html
40 Kandathil A. et al. Role of FDG PET/CT in the Eighth Edition of TNM Staging of Non-Small Cell Lung Cancer. Radiographics 2018; 38 (07) 2134-2149
41 UyBico SJ. et al. Lung cancer staging essentials: the new TNM staging system and potential imaging pitfalls. Radiographics 2010; 30 (05) 1163-1181
42 Thakur SK, Singh DP, Choudhary J. Lung cancer identification: a review on detection and classification. Cancer Metastasis Rev 2020; 39 (03) 989-998
43 Venkadesh KV. et al. Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT. Radiology 2021; 300 (02) 438-447
44 Binczyk F. et al. Radiomics and artificial intelligence in lung cancer screening. Transl Lung Cancer Res 2021; 10 (02) 1186-1199
45 van Klaveren RJ. et al. Management of lung nodules detected by volume CT scanning. N Engl J Med 2009; 361 (23) 2221-2229
46 Werner S. et al. Accuracy and Reproducibility of a Software Prototype for Semi-Automated Computer-Aided Volumetry of the solid and subsolid Components of part-solid Pulmonary Nodules. Fortschr Röntgenstr 2022; 194 (03) 296-305
47 Borrelli P. et al. Freely available convolutional neural network-based quantification of PET/CT lesions is associated with survival in patients with lung cancer. EJNMMI Phys 2022; 9 (01) 6
48 Wallis D. et al. An [18F]FDG-PET/CT deep learning method for fully automated detection of pathological mediastinal lymph nodes in lung cancer patients. Eur J Nucl Med Mol Imaging 2022; 49 (03) 881-888
49 Mu W. et al. Non-invasive decision support for NSCLC treatment using PET/CT radiomics. Nat Commun 2020; 11 (01) 5228
50 Fendler WP. et al. (68)Ga-PSMA PET/CT: Joint EANM and SNMMI procedure guideline for prostate cancer imaging: version 1.0. Eur J Nucl Med Mol Imaging 2017; 44 (06) 1014-1024
51 Hofman MS. et al Prostate-specific membrane antigen PET-CT in patients with high-risk prostate cancer before curative-intent surgery or radiotherapy (proPSMA): a prospective, randomised, multicentre study. Lancet 2020; 395 10231 1208-1216
52 Calais J. et al. Potenzial Impact of (68)Ga-PSMA-11 PET/CT on the Planning of Definitive Radiation Therapy for Prostate Cancer. J Nucl Med 2018; 59 (11) 1714-1721
53 Grubmuller B. et al. PSMA Ligand PET/MRI for Primary Prostate Cancer: Staging Performance and Clinical Impact. Clin Cancer Res 2018; 24 (24) 6300-6307
54 Kostyszyn D. et al. Intraprostatic Tumor Segmentation on PSMA PET Images in Patients with Primary Prostate Cancer with a Convolutional Neural Network. J Nucl Med 2021; 62 (06) 823-828
55 Erle A. et al. Evaluating a Machine Learning Tool for the Classification of Pathological Uptake in Whole-Body PSMA-PET-CT Scans. Tomography 2021; 7 (03) 301-312
56 Gafita A. et al. qPSMA: Semiautomatic Software for Whole-Body Tumor Burden Assessment in Prostate Cancer Using (68)Ga-PSMA11 PET/CT. J Nucl Med 2019; 60 (09) 1277-1283
57 Gafita A. et al. Tumor sink effect in (68)Ga-PSMA-11 PET: Myth or Reality?. J Nucl Med 2021;
58 Moazemi S. et al. Decision-support for treatment with (177)Lu-PSMA: machine learning predicts response with high accuracy based on PSMA-PET/CT and clinical parameters. Ann Transl Med 2021; 9 (09) 818
59 Moazemi S. et al. Estimating the Potenzial of Radiomics Features and Radiomics Signature from Pretherapeutic PSMA-PET-CT Scans and Clinical Data for Prediction of Overall Survival When Treated with (177)Lu-PSMA. Diagnostics (Basel) 2021; 11 (02)
60 Papp L. et al. Supervised machine learning enables non-invasive lesion characterization in primary prostate cancer with [(68)Ga]Ga-PSMA-11 PET/MRI. Eur J Nucl Med Mol Imaging 2021; 48 (06) 1795-1805
61 Fehrenbach U. et al. Automatized Hepatic Tumor Volume Analysis of Neuroendocrine Liver Metastases by Gd-EOB MRI-A Deep-Learning Model to Support Multidisciplinary Cancer Conference Decision-Making. Cancers (Basel) 2021; 13 (11)
62 Hicks RJ. et al. ENETS standardized (synoptic) reporting for molecular imaging studies in neuroendocrine tumours. J Neuroendocrinol 2021; e13040
63 Dromain C. et al. ENETS standardized (synoptic) reporting for radiological imaging in neuroendocrine tumours. J Neuroendocrinol 2021; e13044
64 Ambrosini V. et al. Consensus on molecular imaging and theranostics in neuroendocrine neoplasms. Eur J Cancer 2021; 146: 56-73
65 Werner RA. et al. SSTR-RADS Version 1.0 as a Reporting System for SSTR PET Imaging and Selection of Potenzial PRRT Candidates: A Proposed Standardization Framework. J Nucl Med 2018; 59 (07) 1085-1091
66 Gao X, Wang X. Deep learning for World Health Organization grades of pancreatic neuroendocrine tumors on contrast-enhanced magnetic resonance images: a preliminary study. Int J Comput Assist Radiol Surg 2019; 14 (11) 1981-1991
67 Luo Y. et al. Preoperative Prediction of Pancreatic Neuroendocrine Neoplasms Grading Based on Enhanced Computed Tomography Imaging: Validation of Deep Learning with a Convolutional Neural Network. Neuroendocrinology 2020; 110 (05) 338-350
68 Atkinson C. et al. Radiomics-Based Texture Analysis of (68)Ga-DOTATATE Positron Emission Tomography and Computed Tomography Images as a Prognostic Biomarker in Adults With Neuroendocrine Cancers Treated With (177)Lu-DOTATATE. Front Oncol 2021; 11: 686235
69 Liberini V. et al. Impact of segmentation and discretization on radiomic features in (68)Ga-DOTA-TOC PET/CT images of neuroendocrine tumor. EJNMMI Phys 2021; 8 (01) 21
70 Liberini V. et al. The Challenge of Evaluating Response to Peptide Receptor Radionuclide Therapy in Gastroenteropancreatic Neuroendocrine Tumors: The Present and the Future. Diagnostics (Basel) 2020; 10 (12)
71 Onner H, Abdulrezzak U, Tutus A. Could the skewness and kurtosis texture parameters of lesions obtained from pretreatment Ga-68 DOTA-TATE PET/CT images predict receptor radionuclide therapy response in patients with gastroenteropancreatic neuroendocrine tumors?. Nucl Med Commun 2020; 41 (10) 1034-1039
72 Wehrend J. et al. Automated liver lesion detection in (68)Ga DOTATATE PET/CT using a deep fully convolutional neural network. EJNMMI Res 2021; 11 (01) 98
73 Sonni I. et al. Impact of (68)Ga-PSMA-11 PET/CT on Staging and Management of Prostate Cancer Patients in Various Clinical Settings: A Prospective Single-Center Study. J Nucl Med 2020; 61 (08) 1153-1160
74 Barrio M. et al. The Impact of Somatostatin Receptor-Directed PET/CT on the Management of Patients with Neuroendocrine Tumor: A Systematic Review and Meta-Analysis. J Nucl Med 2017; 58 (05) 756-761
75 Yin Q. et al. Associations between Tumor Vascularity, Vascular Endothelial Growth Factor Expression and PET/MRI Radiomic Signatures in Primary Clear-Cell-Renal-Cell-Carcinoma: Proof-of-Concept Study. Sci Rep 2017; 7: 43356
76 Srimathi S, Yamuna G, Nanmaran R. An Efficient Cancer Classification Model for CT/MRI/PET Fused Images. Curr Med Imaging 2021; 17 (03) 319-330