Evaluation of image quality in five different handheld ultrasound devices and analysis of various impact factors

Abstract

Purpose

Handheld ultrasound (HHUS) devices are increasingly used in clinical practice due to their portability and cost-effectiveness. However, HHUS image quality remains variable, and a standardized expert-based evaluation protocol is lacking. This study compares the image quality of B-mode recordings of five HHUS models with a high-end reference system (HEUS) using a blinded assessment to identify differences, covariates, offering a standardized evaluation framework.

Materials and Methods

In this blinded, multi-center study, video sequences of the liver, pancreas, and sigmoid colon were recorded in ten volunteers using 5 HHUS devices – Vscan Air, Butterfly iQ+, Philips Lumify, Clarius C3, and Clarius C3 HD3 – and one HEUS (GE Logiq E10). Twenty-two physicians with many years of abdominal ultrasound experience rated blinded clips for 5 image quality parameters.

Results

The HEUS consistently received the highest ratings. Image quality differed significantly between HHUS devices. Clarius systems showed the smallest deviation from the reference, followed by Vscan Air and Philips Lumify. All image parameters significantly correlated with the overall impression, without a single dominant factor. The patients’ BMI had no relevant influence. Examiner experience was less impactful than device familiarity and personal preference.

Conclusion

The image quality of HEUS remained superior to all tested HHUS devices. Further subdivision of image quality aspects did not provide additional information. Furthermore, we could not identify significant covariates in our setup. These findings could justify simplified setups with reduced questionnaires and requirements for examiners.

Zusammenfassung

Zielsetzung

Portable Ultraschallsysteme (HHUS) finden aufgrund ihrer Mobilität und Kosteneffizienz zunehmend Anwendung in der Praxis. Ihre Bildqualität variiert jedoch, und ein standardisiertes, expertenbasiertes Bewertungsprotokoll fehlt bislang. Ziel dieser Studie ist der Vergleich von 5 HHUS-Modellen mit einem High-End-Referenzsystem (HEUS) in einem verblindeten Design sowie die Analyse relevanter Kovariablen, die zur Entwicklung eines standardisierten Bewertungsverfahrens beitragen können.

Material und Methoden

In dieser multizentrischen, verblindeten Studie wurden Videos von Leber, Pankreas und Sigma bei 10 Probanden mit 5 HHUS-Geräten (Vscan Air, Butterfly iQ+, Philips Lumify, Clarius C3 und C3 HD3) sowie einem HEUS (Ge Logiq E10) aufgezeichnet. Die verblindeten Videos wurden von 22 Experten mit langjähriger Ultraschall-Erfahrung nach verschiedenen Kriterien zur Bildqualität bewertet.

Ergebnisse

Das HEUS erzielte die besten Bewertungen, die HHUS unterschieden sich signifikant. Clarius-Systeme wichen am wenigsten vom HEUS ab, gefolgt von Vscan Air und Philips Lumify. Alle Bildparameter korrelierten signifikant mit dem Gesamteindruck, ohne dass ein dominanter Parameter identifizierbar war. Der BMI der Patienten hatte keinen relevanten Einfluss. Die Erfahrung der Gutachter war weniger entscheidend als die Geräte-Vertrautheit und persönliche Präferenz.

Schlussfolgerung

Die Bildqualität des HEUS war allen getesteten HHUS überlegen. Eine weitere Unterteilung in einzelne Aspekte der Bildqualität lieferte keine zusätzlichen Informationen. Es wurden keine relevanten Kovariablen identifiziert, was für die Etablierung vereinfachter Bewertungsprotokolle mit reduzierten Fragebögen spricht.

Keywords

HHUS - Handheld ultrasound - image quality - comparison - ultrasound

• References

• 1 Del Medico M, Altieri A, Carnevale-Maffè G. et al. Pocket-size ultrasound device in cholelithiasis: diagnostic accuracy and efficacy of short-term training. Intern Emerg Med 2018; 13: 1121-1126
• 2 Leung SK, Dancel R, Soni RN. et al. Comparison of Six Handheld Ultrasound Devices by Pediatric Point of Care Ultrasound (POCUS) Experts. POCUS J 2025; 10: 141-156
• 3 Perez-Sanchez A, Johnson G, Pucks N. et al. Comparison of 6 handheld ultrasound devices by point-of-care ultrasound experts: a cross-sectional study. Ultrasound J 2024; 16: 45
• 4 Gibbons RC, Jaeger DJ, Berger M. et al. Diagnostic Accuracy of a Handheld Ultrasound vs a Cart-based Model: A Randomized Clinical Trial. West J Emerg Med 2024; 25: 268-274
• 5 Fröhlich E, Beller K, Muller R. et al. Point of Care Ultrasound in Geriatric Patients: Prospective Evaluation of a Portable Handheld Ultrasound Device. Ultraschall in Med 2020; 41: 308-316
• 6 Merkel D, Züllich TF, Schneider C. et al. Prospective Comparison of Handheld Ultrasound Devices from Different Manufacturers with Respect to B-Scan Quality and Clinical Significance for Various Abdominal Sonography Questions. Diagnostics (Basel) 2023; 13: 3622
• 7 Lo H, Frauendorf V, Wischke S. et al. Ambulatory Use of Handheld Point-of-Care Ultrasound (HH-POCUS) in Rural Brandenburg – A Pilot Study. Ultraschall in Med 2022; 43: 584-591
• 8 Ienghong K, Cheung LW, Tiamkao S. et al. The Utilization of Handheld Ultrasound Devices in a Prehospital Setting. Prehosp Disaster Med 2022; 37: 355-359
• 9 Blaivas M, Brannam L, Theodoro D. Ultrasound Image Quality Comparison between an Inexpensive Handheld Emergency Department (ED) Ultrasound Machine and a Large Mobile ED Ultrasound System. Academic Emergency Medicine 2004; 11: 778-781
• 10 Andrea S, Giovanna L, Pietro C. et al. Teaching echoscopy for the early diagnosis of ascites in cirrhosis: assessment of an objective structured clinical examination (OSCE). J Ultrasound 2017; 20: 123-126
• 11 Weimer JM, Beer D, Schneider C. et al. Inter-System Variability of Eight Different Handheld Ultrasound (HHUS) Devices-A Prospective Comparison of B-Scan Quality and Clinical Significance in Intensive Care. Diagnostics (Basel) 2023; 14: 54
• 12 Karlas T, Blank V, Trenker C. et al. Ultrasound systems for abdominal diagnostics – current methods, clinical applications and new technologies. Z Gastroenterol 2023; 61: 1235-1245
• 13 Malik AN, Rowland J, Haber BD. et al. The Use of Handheld Ultrasound Devices in Emergency Medicine. Curr Emerg Hosp Med Rep 2021; 9: 73-81
• 14 Dietrich CF, Bulla P, Dudwiesus H. et al. Entwicklungen, Herausforderung und Perspektiven des Handheld-Ultraschalls (HHUS). Z Gastroenterol 2023; 61: 836-851
• 15 Barreiros AP, Cui XW, Ignee A. et al. Echoscopy in scanning abdominal diseases: initial clinical experience. Z Gastroenterol 2014; 52: 269-275
• 16 Barreiros AP, Dong Y, Ignee A. et al. EchoScopy in scanning abdominal diseases; a prospective single center study. Med Ultrason 2019; 21: 8-15
• 17 Miles DA, Levi CS, Uhanova J. et al. Pocket-Sized Versus Conventional Ultrasound for Detecting Fatty Infiltration of the Liver. Dig Dis Sci 2020; 65: 82-85
• 18 Merkel D, Lueders C, Schneider C. et al. Prospective Comparison of Nine Different Handheld Ultrasound (HHUS) Devices by Ultrasound Experts with Regard to B-Scan Quality, Device Handling and Software in Abdominal Sonography. Diagnostics 2024; 14: 1913
• 19 Gabriel T, Chaghaneh O, Kober J. et al. Enhancing Liver Steatosis Classification: H-Scan Analysis of Handheld Ultrasound Data. IEEE Open Journal of Ultrasonics, Ferroelectrics, and Frequency Control 2025; 5: 62-66
• 20 Dewar ZE, Wu J, Hughes H. et al. A comparison of handheld ultrasound versus traditional ultrasound for acquisition of RUSH views in healthy volunteers. J Am Coll Emerg Physicians Open 2020; 1: 1320-1325
• 21 Stock KF, Klein B, Steubl D. et al. Comparison of a pocket-size ultrasound device with a premium ultrasound machine: diagnostic value and time required in bedside ultrasound examination. Abdom Imaging 2015; 40: 2861-2866
• 22 Seitz K, Vasilakis D, Ziegler M. Efficiency of a portable B-scan ultrasound device in comparison to a high-end machine in abdominal ultrasound. Results of a pilot study. Ultraschall in Med 2003; 24: 96-100
• 23 Le M-PT, Voigt L, Nathanson R. et al. Comparison of four handheld point-of-care ultrasound devices by expert users. Ultrasound J 2022; 14: 27
• 24 Herzog M, Arsova M, Matthes K. et al. Technical assessment of resolution of handheld ultrasound devices and clinical implications. Ultraschall in Med 2024;
• 25 Nylund K, Maconi G, Hollerweger A. et al. EFSUMB Recommendations and Guidelines for Gastrointestinal Ultrasound. Ultraschall in Med 2017; 38: e1-e15
• 26 Harris PA, Taylor R, Minor BL. et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208
• 27 Faul F, Erdfelder E, Buchner A. et al. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods 2009; 41: 1149-1160
• 28 Rykkje A, Carlsen JF, Nielsen MB. Hand-Held Ultrasound Devices Compared with High-End Ultrasound Systems: A Systematic Review. Diagnostics (Basel) 2019; 9: 61
• 29 Browne JE, Watson AJ, Gibson NM. et al. Objective measurements of image quality. Ultrasound in Medicine & Biology 2004; 30: 229-237
• 30 Mannila V, Sipilä O. Phantom-based quality assurance measurements in B-mode ultrasound. Acta Radiologica Short Reports 2013; 2: 2047981613511967