User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial

Abstract

Objectives To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia.

Methods Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on Fast Healthcare Interoperability Resources (FHIR) web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and postdeployment summative evaluation.

Results Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitated enrollment, randomization, model visualization, data capture, and reporting for trial purposes.

Conclusion The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Keywords

FHIR - user-centered design - electronic health record - interoperability - research informatics

• References

• 1 Williams DJ, Zhu Y, Grijalva CG. et al. Predicting severe pneumonia outcomes in children. Pediatrics 2016; 138 (04) e20161019
• 2 Antoon JW, Nian H, Ampofo K. et al. Validation of childhood pneumonia prognostic models for use in emergency care settings. J Pediatric Infect Dis Soc 2023; 12 (08) 451-458
• 3 Van Belle V, Van Calster B. Visualizing risk prediction models. PLoS One 2015; 10 (07) e0132614-e0132614
• 4 Van Belle V, Van Calster B, Van Huffel S, Suykens JAK, Lisboa P. Explaining support vector machines: a color based nomogram. PLoS One 2016; 11 (10) e0164568
• 5 Garvin JH, Ducom J, Matheny M. et al. Descriptive usability study of CirrODS: clinical decision and workflow support tool for management of patients with cirrhosis. JMIR Med Inform 2019; 7 (03) e13627
• 6 Slagle JM, Gordon JS, Harris CE. et al. MyMediHealth - designing a next generation system for child-centered medication management. J Biomed Inform 2010; 43 (05) S27-S31
• 7 Tejani N, Dresselhaus TR, Weinger MB. Development of a hand-held computer platform for real-time behavioral assessment of physicians and nurses. J Biomed Inform 2010; 43 (01) 75-80
• 8 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 23 (05) 899-908
• 9 ANSI/AAMI. Human Factors Engineering - Design of Medical Devices. (HE75:2009 R2018). 2018
• 10 Bilan N, Dastranji A, Ghalehgolab Behbahani A. Comparison of the spo2/fio2 ratio and the PaO2/FiO2 ratio in patients with acute lung injury or acute respiratory distress syndrome. J Cardiovasc Thorac Res 2015; 7 (01) 28-31
• 11 Muniraman HK, Song AY, Ramanathan R. et al. Evaluation of oxygen saturation index compared with oxygenation index in neonates with hypoxemic respiratory failure. JAMA Netw Open 2019; 2 (03) e191179
• 12 Garcia JA, Gardner D, Vines D, Shelledy D, Wettstein R, Peters J. The oxygen concentrations delivered by different oxygen therapy systems. Chest 2005; 128 (04) 389S-389S
• 13 Sung V, Massie J, Hochmann MA, Carlin JB, Jamsen K, Robertson CF. Estimating inspired oxygen concentration delivered by nasal prongs in children with bronchiolitis. J Paediatr Child Health 2008; 44 (1-2) 14-18
• 14 Harrell FE. Regression Modeling Strategies. Springer International Publishing; 2015
• 15 Holtzblatt K, Wendell JB, Wood S. Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design (Interactive Technologies). Morgan Kaufmann; 2005
• 16 Snyder C. Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces. Morgan Kaufmann; 2003
• 17 Smith JC, Spann A, McCoy AB. et al. Natural language processing and machine learning to enable clinical decision support for treatment of pediatric pneumonia. AMIA Annu Symp Proc 2021; 2020: 1130-1139
• 18 Williams DJ, Martin JM, Nian H. et al. Antibiotic clinical decision support for pneumonia in the ED: a randomized trial. J Hosp Med 2023; 18 (06) 491-501
• 19 Osheroff JA, Teich JM, Levick D. et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. HIMSS; 2012
• 20 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
• 21 Pascarella G, Rossi M, Montella E. et al. Risk analysis in healthcare organizations: methodological framework and critical variables. Risk Manag Healthc Policy 2021; 14: 2897-2911
• 22 Dorr DA, D'Autremont C, Pizzimenti C. et al. Assessing data adequacy for high blood pressure clinical decision support: a quantitative analysis. Appl Clin Inform 2021; 12 (04) 710-720
• 23 Lobach DF, Boxwala A, Kashyap N. et al. Integrating a patient engagement app into an electronic health record-enabled workflow using interoperability standards. Appl Clin Inform 2022; 13 (05) 1163-1171
• 24 US Food and Drug Administration. Clinical decision support software: guidance for industry and Food and Drug Administration staff. 2022 . Accessed 5 March 2024 at: https://www.fda.gov/media/109618/download