Einsatz von erweiterten Realitäten (XR) in der Thoraxchirurgie

Philipp Feodorovici; Jan Arensmeyer; Philipp Schnorr; Joachim Schmidt

doi:10.1055/a-2121-6478

Zentralblatt für Chirurgie - Zeitschrift für Allgemeine, Viszeral-, Thorax- und Gefäßchirurgie, Inhaltsverzeichnis

Zentralbl Chir 2023; 148(04): 367-375
DOI: 10.1055/a-2121-6478

Thoraxchirurgie Übersichtsarbeit

Einsatz von erweiterten Realitäten (XR) in der Thoraxchirurgie

Extended Reality (XR) – Applications in Thoracic Surgery

Philipp Feodorovici‡

¹Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland (Ringgold ID: RIN39062)

,

Jan Arensmeyer ‡

¹Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland (Ringgold ID: RIN39062)

,

Philipp Schnorr

²Klinik für Thoraxchirurgie, Helios Klinikum Bonn/Rhein-Sieg, Bonn, Deutschland (Ringgold ID: RIN155843)

,

Joachim Schmidt

¹Sektion Thoraxchirurgie, Klinik und Poliklinik für Allgemein-, Viszeral-, Thorax- und Gefäßchirurgie, Universitätsklinikum Bonn, Bonn, Deutschland (Ringgold ID: RIN39062)

²Klinik für Thoraxchirurgie, Helios Klinikum Bonn/Rhein-Sieg, Bonn, Deutschland (Ringgold ID: RIN155843)

› Institutsangaben

Abstract

Zusammenfassung

Erweiterte Realitäten (XR) mit den Unterbegriffen Virtual Reality (VR), Augmented Reality (AR) und Mixed Reality (MR) beschreiben interaktive und immersive Technologien, welche die reelle Welt durch digitale Elemente ersetzen oder diese um solche nahtlos erweitern. Die XR bieten somit ein sehr breites mögliches Anwendungsspektrum in der Medizin. In der Chirurgie und insbesondere in der Thoraxchirurgie lassen sich XR-Technologien für Anwendungen der Behandlungsplanung, Navigation, Ausbildung und Patienteninformationen nutzbar machen. Solche Anwendungsfälle befinden sich zunehmend in der Erprobung und müssen hinsichtlich ihrer Wertigkeit evaluiert werden. Wir geben einen Überblick über den Status quo der technischen Entwicklung, aktuelle chirurgische Einsatzgebiete der XR und blicken in die Zukunft der medizinischen XR-Landschaft mit Integration von künstlicher Intelligenz (KI).

Abstract

Extended reality (XR) includes the sub-terms of virtual reality (VR), augmented reality (AR) and mixed reality (MR) and describes interactive and immersive technologies that replace the real world with digital elements or seamlessly extend it with such approaches. XR thus offers a very wide range of possible applications in medicine. In surgery, and thoracic surgery in particular, XR technologies can be harnessed for treatment planning, navigation, training, and patient information. Such applications are increasingly being tested and need to be evaluated. We provide an overview of the status quo of technical development, current surgical applications of XR, and look into the future of the medical XR landscape with integration of artificial intelligence (AI).

Schlüsselwörter

erweiterte Realität - virtuelle Realität - Navigation - künstliche Intelligenz - Simulation - chirurgische Lehre

Keywords

extended reality - virtual reality - augmented reality - mixed reality - surgical planning - image guided navigation

Volltext

Referenzen

Literatur
1 Azuma RT. A survey of augmented reality. Presence: Teleoperators & Virtual Environments 1997; 6: 355-385
2 Kuhn S, Huettl F, Deutsch K. et al. Chirurgische Ausbildung im digitalen Zeitalter – Virtual Reality, Augmented Reality und Robotik im Medizinstudium. Zentralbl Chir 2021; 146: 37-43
3 Alaker M, Wynn GR, Arulampalam T. Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis. Int J Surg 2016; 29: 85-94
4 Omlor A, Schwärzel L, Bewarder M. et al. Comparison of immersive and non-immersive virtual reality videos as substitute for in-hospital teaching during coronavirus lockdown: a survey with graduate medical students in Germany. Med Educ Online 2022; 27: 2101417
5 Barsom E, Graafland M, Schijven M. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 2016; 30: 4174-4183
6 Reinschluessel A, Muender T, Salzmann D. et al. Virtual Reality for Surgical Planning – Evaluation Based on Two Liver Tumor Resections. Front Surg 2022; 9: 821060
7 Uppot R, Laguna B, McCarthy C. et al. Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care. Radiology 2019; 291: 182210
8 Rahman R, Wood ME, Qian L. et al. Head-Mounted Display Use in Surgery: A Systematic Review. Surgical Innovation 2020; 27: 88-100
9 Torres K, Denisow-Pietrzyk M, Pietrzyk Ł. et al. Does simulation-based training facilitate the integration of human anatomy with surgery? A report of a novel Surgical Anatomy Course. Folia Morphol (Warsz) 2018; 77: 279-285
10 Bakhuis W, Sadeghi AH, Moes I. et al. Essential Surgical Plan Modifications After Virtual Reality Planning in 50 Consecutive Segmentectomies. Ann Thorac Surg 2023; 115: 1247-1255
11 Marr B. What is extended reality technology? A simple explanation for anyone. Forbes; 2022. Zugriff am 10. April 2023 unter: https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/?sh=6738782a7249
12 Marre P, Villet R. Anatomy theaters in the history and teaching of surgery. J Visc Surg 2020; 157 (Suppl. 02) S73-S76
13 Kemp M. Style and non-style in anatomical illustration: From Renaissance Humanism to Henry Gray. J Anat 2010; 216: 192-208
14 Azer S, Azer S. 3D Anatomy Models and Impact on Learning: A Review of the Quality of the Literature. Health Professions Education 2016; 2
15 Kockro R, Amaxopoulou C, Killeen T. et al. Stereoscopic neuroanatomy lectures using a three-dimensional virtual reality environment. Ann Anat 2015; 201: 91-98
16 Tang YM, Chau KY, Kwok APK. et al. A systematic review of immersive technology applications for medical practice and education – Trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev 2022; 35: 100429
17 Carpinello A, Vezzetti E, Ramieri G. et al. Evaluation of HMDs by QFD for Augmented Reality Applications in the Maxillofacial Surgery Domain. Appl Sci 2021; 11: 11053
18 Kolla S, Elgawly M, Gaughan J. et al. Medical Student Perception of a Virtual Reality Training Module for Anatomy Education. Med Sci Educ 2020; 30: 1201-1210
19 Ma M, Fallavollita P, Seelbach I. et al. Personalized augmented reality for anatomy education. Clin Anat 2016; 29: 446-453
20 Nakai K, Terada S, Takahara A. et al. Anatomy education for medical students in a virtual reality workspace: A pilot study. Clin Anat 2022; 35: 40-44
21 Feodorovici P, Bergedieck B, Sommer N. et al. Entwicklung und Implementierung eines Systems zur Virtual Reality (VR) gestützten Chirurgischen Lehre und Thoraxchirurgischen Weiterbildung am Universitätsklinikum Bonn. Zentralbl Chir 2022; 147 (Suppl. 01) S65
22 Sader J, Clavien C, Korris J. et al. Serious game training in medical education: potential to mitigate cognitive biases of healthcare professionals. Diagnosis (Berl) 2021; 8: 536-537
23 Graafland M, Schraagen JM, Schijven M. Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012; 99: 1322-1330
24 Jensen K, Bjerrum F, Hansen HJ. et al. A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 2015; 21: 420-426
25 Spethmann S, Köhler F. Telemedizin bei chronischer Herzinsuffizienz – von klinischen Studien zur Regelversorgung. Internist (Berl) 2022; 63: 266-273
26 Kremers M. Teleradiologie und Telemedizin. MKG-Chirurg 2020; 13: 248-259
27 Ali S, AbdullahArmand TPT. et al. Metaverse in Healthcare Integrated with Explainable AI and Blockchain: Enabling Immersiveness, Ensuring Trust, and Providing Patient Data Security. Sensors 2023; 23
28 Fréchette E, Deslauriers J. Surgical Anatomy of the Bronchial Tree and Pulmonary Artery. Sem Thorac Cardiovasc Surg 2006; 18: 77-84
29 Saji H, Okada M, Tsuboi M. et al. Segmentectomy versus lobectomy in small-sized peripheral non-small-cell lung cancer (JCOG0802/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet 2022; 399: 1607-1617
30 Zhang L, Li M, Li Z. et al. Three-dimensional printing of navigational template in localization of pulmonary nodule: A pilot study. J Thorac Cardiovasc Surg 2017; 154: 2113-2119.e7
31 Kwok J, Lau R, Zhao ZR. et al. Multi-dimensional printing in thoracic surgery: Current and future applications. J Thorac Dis 2018; 10 (Suppl. 06) S756-S763
32 Xu W, Zhang X, Ke T. et al. 3D printing-assisted preoperative plan of pedicle screw placement for middle-upper thoracic trauma: a cohort study. BMC Musculoskelet Disord 2017; 18: 348
33 Bartella AK, Kamal M, Scholl I. et al. Virtual reality in preoperative imaging in maxillofacial surgery: implementation of “the next level”?. Br J Oral Maxillofac Surg 2019; 57: 644-648
34 Dharmarajan H, Anderson J, Kim S. et al. Transition to a Virtual Multidisciplinary Tumor Board during the COVID-19 Pandemic: The University of Pittsburgh Experience. Head Neck 2020; 42: 1310-1316
35 Blasi L, Bordonaro R, Serretta V. et al. Virtual Clinical and Precision Medicine Tumor Boards, a Cloud-based Platform-mediated Implementation of Multidisciplinary Reviews among Oncology Centers in the Covid-19 Era: an Observational Study Protocol. (Preprint). JMIR Research Protocols 2020; 10: e26220
36 Park CH, Han K, Hur J. et al. Comparative Effectiveness and Safety of Preoperative Lung Localization for Pulmonary Nodules: A Systematic Review and Meta-analysis. Chest 2017; 151: 316-328
37 Fang HY, Chang KW, Chao YK. Hybrid operating room for the intraoperative CT-guided localization of pulmonary nodules. Ann Transl Med 2019; 7: 34
38 Reisenauer J, Simoff MJ, Pritchett MA. et al. Ion: Technology and Techniques for Shape-sensing Robotic-assisted Bronchoscopy. Ann Thorac Surg 2022; 113: 308-315
39 Ng CS, Ong BH, Chao YK. et al. Use of Indocyanine Green Fluorescence Imaging in Thoracic and Esophageal Surgery. Ann Thorac Surg 2023; 115: 1068-1076
40 Hennig S, Jansen-Winkeln B, Köhler H. et al. Novel Intraoperative Imaging of Gastric Tube Perfusion during Oncologic Esophagectomy – A Pilot Study Comparing Hyperspectral Imaging (HSI) and Fluorescence Imaging (FI) with Indocyanine Green (ICG). Cancers (Basel) 2021; 14: 97
41 Lee YJ, van den Berg NS, Orosco RK. et al. A narrative review of fluorescence imaging in robotic-assisted surgery. Laparosc Surg 2021; 5: 31
42 Soyer P, Mosnier H, Choti MA. et al. Intraoperative and laparoscopic sonography of the liver. Eur Radiol 1997; 7: 1296-1302
43 Khundam C, Vorachart V, Preeyawongsakul P. et al. A comparative study of interaction time and usability of using controllers and hand tracking in virtual reality training. Informatics 2021; 8: 60
44 Carter C, Graham R. Experimental comparison of manual and voice controls for the operation of in-vehicle systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2000; 44: 3–286-3–289
45 Greener R. What does CHATGPT mean for enterprise-grade XR?. XR Today; 2023. Zugriff am 10. April 2023 unter: https://www.xrtoday.com/virtual-reality/what-does-chat-gpt-mean-for-enterprise-grade-xr/
46 Lo S, Fowers S, Darko K. et al. Participatory Development of a 3D Telemedicine system during Covid: the future of remote consultations. J Plast Reconstruct Aesthet Surg 2022;
47 Microsoft Corporation, Microsoft Research. 3D telemedicine. Bringing specialized healthcare to rural and underserved communities with live 3D communication. 2023 Zugriff am 10. April 2023 unter: https://www.microsoft.com/en-us/research/project/3d-telemedicine/