Different Evaluation Strategies of Oncological CT Examinations with Regard to Professional Experience: A Clinical Study Using Eye-tracking

Abstract

Purpose

Contrast-enhanced CT is the standard imaging technique in oncological objectives. Rates of missed pathologies depend on work experience of the respective radiologists. Thus the aim of this study is to analyze the eye movements of professionals while reading CT images in order to evaluate whether the eye-fixation patterns and search strategies of experienced radiologists could explain higher detection rates of pathologies and whether such patterns can be learned.

Materials and Methods

Anonymized images of 10 patients were presented to three medical students and six radiologists with different levels of work experience. During image analysis, ocular fixation positions were recorded using an eye-tracking software tool. The CT scans were analyzed retrospectively, considering the individual course of disease with the issue of successful detection of all pathologies. Visual attention and dwell time of ocular fixation on clinically important abnormalities or areas with pathological findings, general search patterns, and time efficiency were assessed. For statistical analysis, interobserver variability and accuracy of lesion detection were evaluated taking into account individual experience.

Results

The results revealed that observer sensitivity depends on work experience due to a more systematic order of inspection and a well-known course of disease, e.g. in case of metastatic spread. The areas of missed pathologies mostly included secondary findings. Inexperienced readers changed the stratification considerably more often and required more time for reporting or detecting pathologies.

Conclusions

Our results suggest that experienced radiological physicians reduce their amount of missed findings by looking more systematically at images and by applying a more targeted inspection of clinically important regions.

Key Points

• CT interpretation by radiology residents is faster and less error-prone compared to postgraduate residents

• systematic image analysis is trainable

• engrams tend to be acquired through experience

Citation Format

• Kluge S, Schülke C, Ites HC et al. Different Evaluation Strategies of Oncological CT Examinations with Regard to Professional Experience: A Clinical Study Using Eye-tracking. Rofo 2025; 197: 1077–1087

Zusammenfassung

Ziel

Die kontrastverstärkte CT ist das Standardbildgebungsverfahren bei onkologischen Zielsetzungen. Die Rate der übersehenen Pathologien hängt von der Berufserfahrung der jeweiligen Radiologen ab. Ziel dieser Studie ist es daher, die Augenbewegungen von Fachleuten beim Lesen von CT-Bildern zu analysieren, um festzustellen, ob die Augenfixationsmuster und Suchstrategien erfahrener Radiologen höhere Entdeckungsraten von Pathologien erklären können und ob solche Muster erlernt werden können.

Materialien und Methoden

Anonymisierte Bilder von 10 Patienten wurden 3 Medizinstudenten und 6 Radiologen mit unterschiedlicher Berufserfahrung vorgelegt. Während der Bildanalyse wurden die Fixationspositionen der Augen mit einer Eyetracking-Software aufgezeichnet. Die CT-Scans wurden retrospektiv unter Berücksichtigung des individuellen Krankheitsverlaufs mit der Fragestellung der erfolgreichen Erkennung aller Pathologien analysiert. Die visuelle Aufmerksamkeit und die Verweildauer der Augenfixation auf klinisch wichtige Anomalien oder Bereiche mit pathologischen Befunden, allgemeine Suchmuster und Zeiteffizienz wurden bewertet. Für die statistische Analyse wurden die Interobserver-Variabilität und die Genauigkeit der Läsionserkennung unter Berücksichtigung der individuellen Erfahrung bewertet.

Ergebnisse

Die Ergebnisse zeigten, dass die Sensitivität des Beobachters von der Berufserfahrung abhängt, da die Inspektion systematischer abläuft und der Krankheitsverlauf bekannt ist, z.B. im Falle einer metastatischen Ausbreitung. Zu den übersehenen Pathologien gehörten meist Sekundärbefunde. Unerfahrene Beobachter änderten die Schichtebenen wesentlich häufiger und benötigten mehr Zeit für die Meldung oder Entdeckung von Pathologien.

Schlussfolgerungen

Unsere Ergebnisse deuten darauf hin, dass erfahrene Radiologen die Zahl der übersehenen Befunde verringern, indem sie die Bilder systematischer betrachten und eine gezieltere Inspektion der klinisch wichtigen Regionen vornehmen.

Kernaussagen

• schnellere und weniger fehleranfälligere CT-Interpretation durch radiologische Fachärzte gegenüber Weiterbildungs-Assistenzärzten

• systematische Bildanalyse trainierbar

• Engramme eher erworben durch Erfahrung

Keywords

abdomen - diagnostic radiology - education - observer performance - eye tracking

Publication History

Received: 10 April 2024

Accepted after revision: 21 October 2024

Article published online:
12 November 2024

• Literature

• 1 Purves DAG, Fitzpatrick D. Associates SMS, editor. et al. Types of Eye Movements and Their Functions. In: Neuroscience. 2001
• 2 Duchowski A. Eye tracking methodology: Theory and practice: Springer Science & Business Media; 2007.
• 3 Kammerer S, Schülke C, Leclaire MD. et al. Impact of Working Experience on Image Perception and Image Evaluation Approaches in Stroke Imaging: Results of an Eye-Tracking Study. Fortschr Röntgenstr 2019; 191 (09) 836-844
• 4 Krupinski EA. The importance of perception research in medical imaging. Radiation medicine 2000; 18 (06) 329-334
• 5 Hundt W, Rust F, Stabler A. et al. Dose reduction in multislice computed tomography. Journal of computer assisted tomography 2005; 29 (01) 140-147
• 6 Iball GR, Brettle DS, Moore AC. Assessment of tube current modulation in pelvic CT. The British journal of radiology 2006; 79 (937) 62-70
• 7 Kammerer S, Höink AJ, Wessling J. et al. Abdominal and pelvic CT: is positive enteric contrast still necessary? Results of a retrospective observational study. European radiology 2015; 25 (03) 669-678
• 8 Bertram R, Helle L, Kaakinen JK. et al. The Effect of Expertise on Eye Movement Behaviour in Medical Image Perception. PloS one 2013; 8 (06) e66169
• 9 Mann DT, Williams AM, Ward P. et al. Perceptual-cognitive expertise in sport: a meta-analysis. J Sport Exerc Psychol 2007; 29 (04) 457-478
• 10 Rayner K. Eye movements in reading and information processing: 20 years of research. Psychol Bull 1998; 124 (03) 372-422
• 11 Karargyris A, Kashyap S, Lourentzou I. et al. Creation and validation of a chest X-ray dataset with eye-tracking and report dictation for AI development. Scientific Data 2021; 8 (01) 92
• 12 MacKenzie C, Law B, Atkins MS. et al. (2004) Eye gaze patterns differentiate novices and experts in a virtual laparoscopic surgery training environment. Eye Tracking Research and Applications, Proceedings ACM ETRA Conference, March 22–24, 2004, San Antonio, TX, 41 – 472004.
• 13 Wilson M, McGrath J, Vine S. et al. Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts. Surg Endosc 2010; 24 (10) 2458-64
• 14 Kocak E, Ober J, Berme N. et al. Eye motion parameters correlate with level of experience in video-assisted surgery: objective testing of three tasks. J Laparoendosc Adv Surg Tech A 2005; 15 (06) 575-580
• 15 Mallett S, Phillips P, Fanshawe TR. et al. Tracking Eye Gaze during Interpretation of Endoluminal Three-dimensional CT Colonography: Visual Perception of Experienced and Inexperienced Readers. Radiology 2014; 273: 783-792
• 16 Hsieh SS, Inoue A, Yalon M. et al. Targeted Training Reduces Search Errors but Not Classification Errors for Hepatic Metastasis Detection at Contrast-Enhanced CT. Academic radiology 2024; 31 (02) 448-456
• 17 Hsieh SS, Inoue A, Yalon M. et al. A training program to reduce reader search errors for liver metastasis detection in CT. Proc SPIE Int Soc Opt Eng 2023; 12467
• 18 Gong H, Hsieh SS, Holmes DR. et al. An interactive eye-tracking system for measuring radiologists' visual fixations in volumetric CT images: Implementation and initial eye-tracking accuracy validation. Med Phys 2021; 48 (11) 6710-6723
• 19 Kundel HL, Nodine CF, Conant EF. et al. Holistic component of image perception in mammogram interpretation: gaze-tracking study. Radiology 2007; 242 (02) 396-402
• 20 Swensson RG. A two-stage detection model applied to skilled visual search by radiologists. Perception & Psychophysics 1980; 27 (01) 11-16
• 21 Drew T, Evans K, Võ MLH. et al. Informatics in Radiology: What Can You See in a Single Glance and How Might This Guide Visual Search in Medical Images?. Radiographics : a review publication of the Radiological Society of North America, Inc 2013; 33 (01) 263-274
• 22 Samuel S, Kundel HL, Nodine CF. et al. Mechanism of satisfaction of search: eye position recordings in the reading of chest radiographs. Radiology 1995; 194 (03) 895-902
• 23 Berbaum KS, Franken Jr. EA, Dorfman DD. et al. Satisfaction of search in diagnostic radiology. Investigative radiology 1990; 25 (02) 133-140
• 24 Rogers LF. Keep looking: satisfaction of search. AJR American journal of roentgenology 2000; 175 (02) 287
• 25 Berbaum KS, Franken Jr. EA, Dorfman DD. et al. Time course of satisfaction of search. Investigative radiology 1991; 26 (07) 640-648
• 26 Berbaum KS, Franken Jr. EA, Dorfman DD. et al. Can order of report prevent satisfaction of search in abdominal contrast studies?. Academic radiology 2005; 12 (01) 74-84
• 27 Diaz I, Schmidt S, Verdun FR. et al. Eye-tracking of nodule detection in lung CT volumetric data. Med Phys 2015; 42 (06) 2925-2932
• 28 Ellis SM, Hu X, Dempere-Marco L. et al. Thin-section CT of the lungs: eye-tracking analysis of the visual approach to reading tiled and stacked display formats. European journal of radiology 2006; 59 (02) 257-264