Nuklearmedizin - NuclearMedicine, Table of Contents

CC BY-NC-ND 4.0 · Nuklearmedizin 2023; 62(06): 334-342
DOI: 10.1055/a-2198-0358

Review

Artificial Intelligence and Deep Learning for Advancing PET Image Reconstruction: State-of-the-Art and Future Directions

Künstliche Intelligenz und Deep Learning für die Weiterentwicklung der PET-Bildrekonstruktion: Stand der Technik und zukünftige Perspektiven

Dirk Hellwig

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

²Partner Site Regensburg, Bavarian Center for Cancer Research (BZKF), Regensburg, Germany

³Medical Data Integration Center (MEDIZUKR), University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

,

Nils Constantin Hellwig

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

³Medical Data Integration Center (MEDIZUKR), University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

,

Steven Boehner

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

²Partner Site Regensburg, Bavarian Center for Cancer Research (BZKF), Regensburg, Germany

³Medical Data Integration Center (MEDIZUKR), University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

,

Timo Fuchs

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

²Partner Site Regensburg, Bavarian Center for Cancer Research (BZKF), Regensburg, Germany

³Medical Data Integration Center (MEDIZUKR), University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

,

Regina Fischer

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

²Partner Site Regensburg, Bavarian Center for Cancer Research (BZKF), Regensburg, Germany

³Medical Data Integration Center (MEDIZUKR), University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

,

Daniel Schmidt

¹Department of Nuclear Medicine, University Hospital Regensburg, Regensburg, Germany (Ringgold ID: RIN39070)

› Author Affiliations

Abstract

Positron emission tomography (PET) is vital for diagnosing diseases and monitoring treatments. Conventional image reconstruction (IR) techniques like filtered backprojection and iterative algorithms are powerful but face limitations. PET IR can be seen as an image-to-image translation. Artificial intelligence (AI) and deep learning (DL) using multilayer neural networks enable a new approach to this computer vision task. This review aims to provide mutual understanding for nuclear medicine professionals and AI researchers. We outline fundamentals of PET imaging as well as state-of-the-art in AI-based PET IR with its typical algorithms and DL architectures. Advances improve resolution and contrast recovery, reduce noise, and remove artifacts via inferred attenuation and scatter correction, sinogram inpainting, denoising, and super-resolution refinement. Kernel-priors support list-mode reconstruction, motion correction, and parametric imaging. Hybrid approaches combine AI with conventional IR. Challenges of AI-assisted PET IR include availability of training data, cross-scanner compatibility, and the risk of hallucinated lesions. The need for rigorous evaluations, including quantitative phantom validation and visual comparison of diagnostic accuracy against conventional IR, is highlighted along with regulatory issues. First approved AI-based applications are clinically available, and its impact is foreseeable. Emerging trends, such as the integration of multimodal imaging and the use of data from previous imaging visits, highlight future potentials. Continued collaborative research promises significant improvements in image quality, quantitative accuracy, and diagnostic performance, ultimately leading to the integration of AI-based IR into routine PET imaging protocols.

Keywords

Keywords

PET image reconstruction - Artificial intelligence - Deep learning - Medical imaging - Quantitative accuracy

Literature
1 Hoeschen C, Rafecas M, Aspelmeier T. Algorithms for Image Reconstruction. In: Cantone MC, Hoeschen C. Radiation Physics for Nuclear Medicine. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011: 211-232
2 Radon J. Über die Bestimmung von Funktionen durch ihre Integralwerte längs gewisser Mannigfaltigkeiten. Berichte Über Verhandlungen K-Sächs Ges Wiss Zu Leipz Math-Phys Kl 1917; 69: 262-277
3 Bracewell R. Strip Integration in Radio Astronomy. Aust J Phys 1956; 9: 198
4 López-Montes A, Galve P, Udias JM. et al. Real-Time 3D PET Image with Pseudoinverse Reconstruction. Appl Sci 2020; 10: 2829
5 Bares R, Klever P, Hellwig D. et al. Pancreatic cancer detected by positron emission tomography with 18F-labelled deoxyglucose: method and first results. Nucl Med Commun 1993; 14: 596-601
6 Hudson HM, Larkin RS. Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans Med Imaging 1994; 13: 601-609
7 Levitan E, Herman GT. A Maximum a Posteriori Probability Expectation Maximization Algorithm for Image Reconstruction in Emission Tomography. IEEE Trans Med Imaging 1987; 6: 185-192
8 Rogasch JM, Hofheinz F, Lougovski A. et al. The influence of different signal-to-background ratios on spatial resolution and F18-FDG-PET quantification using point spread function and time-of-flight reconstruction. EJNMMI Phys 2014; 1: 12
9 Lange K, Bahn M, Little R. A Theoretical Study of Some Maximum Likelihood Algorithms for Emission and Transmission Tomography. IEEE Trans Med Imaging 1987; 6: 106-114
10 Lipinski B, Herzog H, Rota Kops E. et al. Expectation maximization reconstruction of positron emission tomography images using anatomical magnetic resonance information. IEEE Trans Med Imaging 1997; 16: 129-136
11 Bland J, Mehranian A, Belzunce MA. et al. MR-Guided Kernel EM Reconstruction for Reduced Dose PET Imaging. IEEE Trans Radiat Plasma Med Sci 2018; 2: 235-243
12 Bishop CM. Pattern Recognition and Machine Learning. Softcover reprint of the original 1st edition 2006 (corrected at 8th printing 2009). New York, NY: Springer New York; 2016
13 McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 1943; 5: 115-133
14 Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural Netw 1989; 2: 359-366
15 Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. 2017.
16 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444
17 He K, Zhang X, Ren S. et al. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). In: . Las Vegas, NV, USA: IEEE; 2016: 770-778
18 Tieleman T, Hinton G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA Neural Netw Mach Learn 2012; 4: 26-31
19 Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017; 60: 84-90
20 Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on Computational learning theory. In: . Pittsburgh Pennsylvania USA: ACM; 1992: 144-152
21 De A, Horwitz GD. Spatial receptive field structure of double-opponent cells in macaque V1. J Neurophysiol 2021; 125: 843-857
22 Turner MR. Texture discrimination by Gabor functions. Biol Cybern 1986; 55: 71-82
23 Ronneberger O, Fischer P, Brox T. et al. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing; 2015: 234-241
24 Goodfellow I, Pouget-Abadie J, Mirza M. et al. Generative Adversarial Nets. Ghahramani Z, Welling M, Cortes C. Curran Associates, Inc.; 2014.
25 Zhu B, Liu JZ, Cauley SF. et al. Image reconstruction by domain-transform manifold learning. Nature 2018; 555: 487-492
26 Häggström I, Schmidtlein CR, Campanella G. et al. DeepPET: A deep encoder–decoder network for directly solving the PET image reconstruction inverse problem. Med Image Anal 2019; 54: 253-262
27 Liu Z, Chen H, Liu H. et al. Deep Learning Based Framework for Direct Reconstruction of PET Images. In: Shen D, Liu T, Peters TM. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Cham: Springer International Publishing; 2019: 48-56
28 Whiteley W, Luk WK, Gregor J. DirectPET: full-size neural network PET reconstruction from sinogram data. J Med Imaging 2020; 7: 1
29 ComtatBataille F, Michel C. et al. OSEM-3D Reconstruction Strategies for the ECAT HRRT. In: IEEE Symposium Conference Record Nuclear Science 2004. Rome, Italy: IEEE; 2004: 3492-3496
30 Hu Z, Xue H, Zhang Q. et al. DPIR-Net: Direct PET Image Reconstruction Based on the Wasserstein Generative Adversarial Network. IEEE Trans Radiat Plasma Med Sci 2021; 5: 35-43
31 Ma R, Hu J, Sari H. et al. An encoder-decoder network for direct image reconstruction on sinograms of a long axial field of view PET. Eur J Nucl Med Mol Imaging 2022; 49: 4464-4477
32 Gong K, Guan J, Kim K. et al. Iterative PET Image Reconstruction Using Convolutional Neural Network Representation. IEEE Trans Med Imaging 2019; 38: 675-685
33 Ulyanov D, Vedaldi A, Lempitsky V. Deep Image Prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018
34 Gong K, Catana C, Qi J. et al. Direct Reconstruction of Linear Parametric Images From Dynamic PET Using Nonlocal Deep Image Prior. IEEE Trans Med Imaging 2022; 41: 680-689
35 Yokota T, Kawai K, Sakata M. et al. Dynamic PET Image Reconstruction Using Nonnegative Matrix Factorization Incorporated With Deep Image Prior. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2019
36 Ote K, Hashimoto F, Onishi Y. et al. List-Mode PET Image Reconstruction Using Deep Image Prior. IEEE Trans Med Imaging 2023; 42: 1822-1834
37 Fessler JA. Optimization Methods for Magnetic Resonance Image Reconstruction: Key Models and Optimization Algorithms. IEEE Signal Process Mag 2020; 37: 33-40
38 Gong K, Catana C, Qi J. et al. PET Image Reconstruction Using Deep Image Prior. IEEE Trans Med Imaging 2019; 38: 1655-1665
39 Chun Y, Fessler JA. Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery. In: 2018 IEEE 13th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP). Zagorochoria: IEEE; 2018: 1-5
40 Gong K, Wu D, Kim K. et al. MAPEM-Net: an unrolled neural network for Fully 3D PET image reconstruction. In: Matej S, Metzler SD. 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine. Philadelphia, United States: SPIE; 2019: 102
41 Mehranian A, Reader AJ. Model-Based Deep Learning PET Image Reconstruction Using Forward–Backward Splitting Expectation–Maximization. IEEE Trans Radiat Plasma Med Sci 2021; 5: 54-64
42 Corda-D’Incan G, Schnabel JA, Hammers A. et al. Single-Modality Supervised Joint PET-MR Image Reconstruction. IEEE Trans Radiat Plasma Med Sci 2023; 1-1
43 Whiteley W, Gregor J. CNN-based PET sinogram repair to mitigate defective block detectors. Phys Med Biol 2019; 64: 235017
44 Kaplan S, Zhu YM. Full-Dose PET Image Estimation from Low-Dose PET Image Using Deep Learning: a Pilot Study. J Digit Imaging 2019; 32: 773-778
45 Wang Y, Zhou L, Yu B. et al. 3D Auto-Context-Based Locality Adaptive Multi-Modality GANs for PET Synthesis. IEEE Trans Med Imaging 2019; 38: 1328-1339
46 Lu W, Onofrey JA, Lu Y. et al. An investigation of quantitative accuracy for deep learning based denoising in oncological PET. Phys Med Biol 2019; 64: 165019
47 Zhou L, Schaefferkoetter JD, Tham IWK. et al. Supervised learning with cyclegan for low-dose FDG PET image denoising. Med Image Anal 2020; 65: 101770
48 Matej S, Surti S, Jayanthi S. et al. Efficient 3-D TOF PET Reconstruction Using View-Grouped Histo-Images: DIRECT—Direct Image Reconstruction for TOF. IEEE Trans Med Imaging 2009; 28: 739-751
49 Whiteley W, Panin V, Zhou C. et al. FastPET: Near Real-Time Reconstruction of PET Histo-Image Data Using a Neural Network. IEEE Trans Radiat Plasma Med Sci 2021; 5: 65-77
50 Shiri I, Arabi H, Geramifar P. et al. Deep-JASC: joint attenuation and scatter correction in whole-body 18F-FDG PET using a deep residual network. Eur J Nucl Med Mol Imaging 2020; 47: 2533-2548
51 Li T, Zhang M, Qi W. et al. Motion correction of respiratory-gated PET images using deep learning based image registration framework. Phys Med Biol 2020; 65: 155003
52 Ledig C, Theis L, Huszar F. et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE; 2017: 105-114
53 YangJianchaoWright J, Huang TS. et al. Image Super-Resolution Via Sparse Representation. IEEE Trans Image Process 2010; 19: 2861-2873
54 Jin W, Li X, Fatehi M. et al. Guidelines and evaluation of clinical explainable AI in medical image analysis. Med Image Anal 2023; 84: 102684
55 Ribeiro MT, Singh S, Guestrin C. „Why Should I Trust You?“: Explaining the Predictions of Any Classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco California USA: ACM; 2016
56 Herington J, McCradden MD, Creel K. et al. Ethical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance. J Nucl Med 2023;
57 Langlotz CP, Allen B, Erickson BJ. et al. A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791