Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2025; 16(02): 377-392
DOI: 10.1055/a-2505-7743
Research Article

“Be Really Careful about That”: Clinicians' Perceptions of an Intelligence Augmentation Tool for In-Hospital Deterioration Detection

Jorie M. Butler
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
2   Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Geriatrics Research, Education, and Clinical Center (GRECC), VA Salt Lake City Health Care System, Salt Lake City, Utah, United States
,
Alyssa Doubleday
3   Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, Idaho, United States
,
Usman Sattar
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Mary Nies
3   Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, Idaho, United States
,
Amanda Jeppesen
4   Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Meridian, Idaho, United States
,
Melanie Wright
5   Tunnell Government Services, Inc., Bethesda, Maryland, United States
,
Thomas Reese
6   Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, United States
,
Kensaku Kawamoto
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Guilherme Del Fiol
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Karl Madaras-Kelly
4   Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Meridian, Idaho, United States
› Author Affiliations Funding This study was funded by the U.S. Department of Health and Human Services, National Institutes of Health (grant no.: 1R01GM137083-01).
› Further Information
Also available at
   Permissions and Reprints

Abstract

Objective This study aimed to explore clinicians' perceptions and preferences of prototype intelligence augmentation (IA)-based visualization displays of in-hospital deterioration risk scores to inform future user interface design and implementation in clinical care.

Methods Prototype visualization displays incorporating an IA-based early warning score (EWS) for in-hospital deterioration were developed using cognitive theory and user-centered design principles. The displays featured variations of EWS and clinical data arranged in multipatient and single-patient views. Physician and nurse participants with at least 5 years of clinical experience were recruited to participate in semistructured qualitative interviews focused on understanding their experiences with IA and thoughts and preferences about the prototype displays. A thematic analysis was performed on these data.

Results Six themes were identified: (1) clinicians perceive IA as valuable with some caveats related to function and context; (2) individual differences among users influence preferences for customizability; (3) EWS are particularly useful for patient triage; (4) need for patient-centered contextual information to complement EWS; (5) perspectives related to understanding the EWS composition; and (6) design preferences that focus on clarity for interpretation of information.

Conclusion This study demonstrates clinicians' interest in and reservations about IA tools for clinical deterioration. The findings underscore the importance of understanding clinicians' cognitive needs and framing IA-generated tools as complementary to support them. A clinician focuses on high-level pattern matching information, and clinician's comments related to the power of consistency with typical views (e.g., this is “how I usually see things”), and questions regarding support of score interpretation (e.g., age of the data, questions about what the model “knows”) suggest some of the challenges of IA implementation. The findings also identify design implications including the need for contextualizing the EWS for the patient's specific situation, incorporating trend information, and explaining the display purpose for clinical use.

Keywords

intelligence augmentation - in-hospital deterioration - patient-centered - visualization displays - artificial intelligence

Protection of Human and Animal Subjects

This project was reviewed and approved by the Institutional Review Board at Idaho State and University of Utah. Participants provided verbal consent for participation at the beginning of each session.


Note

The views expressed in this paper are those of the authors and do not necessarily represent the position or policy of the U.S. Department of Veterans Affairs or the United States Government.


Supplementary Material



Publication History

Received: 08 August 2024

Accepted: 16 December 2024

Article published online:
30 April 2025

© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany

 

  • References

  • 1 Sadiku MNO, Ashaaola TJ, Ajayi-Majebi A, Musa SM. Augmented intelligence. Int J Sci Adv 2021; 2 (05) 772-776
    Search in Google Scholar
  • 2 Wan YJ, Wright MC, McFarland MM. et al. Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review. J Am Med Inform Assoc 2023; 31 (01) 256-273
    Search in Google Scholar
  • 3 Korach ZT, Cato KD, Collins SA. et al. Unsupervised machine learning of topics documented by nurses about hospitalized patients prior to a rapid-response event. Appl Clin Inform 2019; 10 (05) 952-963
    Search in Google Scholar
  • 4 Croskerry P, Campbell SG, Petrie DA. The challenge of cognitive science for medical diagnosis. Cogn Res Princ Implic 2023; 8 (01) 13
    Search in Google Scholar
  • 5 Patel VL, Kaufman DR, Arocha JF. Emerging paradigms of cognition in medical decision-making. J Biomed Inform 2002; 35 (01) 52-75
    Search in Google Scholar
  • 6 Corazza GR, Lenti MV, Howdle PD. Diagnostic reasoning in internal medicine: a practical reappraisal. Intern Emerg Med 2021; 16 (02) 273-279
    Search in Google Scholar
  • 7 Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform 2015; 53: 3-14
    Search in Google Scholar
  • 8 Peelen RV, Eddahchouri Y, Koeneman M, van de Belt TH, van Goor H, Bredie SJ. Algorithms for prediction of clinical deterioration on the general wards: a scoping review. J Hosp Med 2021; 16 (10) 612-619
    Search in Google Scholar
  • 9 Jahandideh S, Ozavci G, Sahle BW, Kouzani AZ, Magrabi F, Bucknall T. Evaluation of machine learning-based models for prediction of clinical deterioration: a systematic literature review. Int J Med Inform 2023; 175: 105084
    Search in Google Scholar
  • 10 Mann D, Hess R, McGinn T. et al. Adaptive design of a clinical decision support tool: what the impact on utilization rates means for future CDS research. Digit Health 2019; 5: 2055207619827716
    Search in Google Scholar
  • 11 Petersen JA, Rasmussen LS, Rydahl-Hansen S. Barriers and facilitating factors related to use of early warning score among acute care nurses: a qualitative study. BMC Emerg Med 2017; 17 (01) 36
    Search in Google Scholar
  • 12 Baig MM, GholamHosseini H, Afifi S, Lindén M. A systematic review of rapid response applications based on early warning score for early detection of inpatient deterioration. Inform Health Soc Care 2021; 46 (02) 148-157
    Search in Google Scholar
  • 13 Wood C, Chaboyer W, Carr P. How do nurses use early warning scoring systems to detect and act on patient deterioration to ensure patient safety? A scoping review. Int J Nurs Stud 2019; 94: 166-178
    Search in Google Scholar
  • 14 Chua WL, Wee LC, Lim JYG. et al. Automated rapid response system activation-Impact on nurses' attitudes and perceptions towards recognising and responding to clinical deterioration: mixed-methods study. J Clin Nurs 2023; 32 (17-18): 6322-6338
    Search in Google Scholar
  • 15 Baxter SL, Bass JS, Sitapati AM. Barriers to implementing an artificial intelligence model for unplanned readmissions. ACI Open 2020; 4 (02) e108-e113
    Search in Google Scholar
  • 16 Plana D, Shung DL, Grimshaw AA, Saraf A, Sung JJY, Kann BH. Randomized clinical trials of machine learning interventions in health care: a systematic review. JAMA Netw Open 2022; 5 (09) e2233946
    Search in Google Scholar
  • 17 Mello MM, Shah NH, Char DS. President Biden's executive order on artificial intelligence-implications for health care organizations. JAMA 2024; 331 (01) 17-18
    Search in Google Scholar
  • 18 Coiera E. The last mile: where artificial intelligence meets reality. J Med Internet Res 2019; 21 (11) e16323
    Search in Google Scholar
  • 19 Salwei ME, Carayon P. A sociotechnical systems framework for the application of artificial intelligence in health care delivery. J Cogn Eng Decis Mak 2022; 16 (04) 194-206
    Search in Google Scholar
  • 20 Lazar S, Nelson A. AI safety on whose terms?. Science 2023; 381 (6654) 138
    Search in Google Scholar
  • 21 Shields C, Cunningham SG, Wake DJ. et al. User-centered design of a novel risk prediction behavior change tool augmented with an artificial intelligence engine (MyDiabetesIQ): a sociotechnical systems approach. JMIR Hum Factors 2022; 9 (01) e29973
    Search in Google Scholar
  • 22 Wickens CD, Helton WS, Hollands JG, Banbury S. Engineering Psychology and Human Performance. Routledge; 2021
    Search in Google Scholar
  • 23 Wright MC. Chapter 14 - Information visualization and integration. In: Greenes RA, Del Fiol G. eds. Clinical Decision Support and Beyond (Third Edition). Oxford: Academic Press; 2023: 435-463
    Search in Google Scholar
  • 24 Kahneman D, Klein G. Conditions for intuitive expertise: a failure to disagree. Am Psychol 2009; 64 (06) 515-526
    Search in Google Scholar
  • 25 Croskerry P. A universal model of diagnostic reasoning. Acad Med 2009; 84 (08) 1022-1028
    Search in Google Scholar
  • 26 Weir CR, Rubin MA, Nebeker J, Samore M. Modeling the mind: how do we design effective decision-support?. J Biomed Inform 2017; 71S: S1-S5
    Search in Google Scholar
  • 27 Hegarty M. The cognitive science of visual-spatial displays: implications for design. Top Cogn Sci 2011; 3 (03) 446-474
    Search in Google Scholar
  • 28 Klein GA. Sources of power: How People Make Decisions. MIT press; 2017
    Search in Google Scholar
  • 29 Churpek MM, Yuen TC, Winslow C. et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med 2014; 190 (06) 649-655
    Search in Google Scholar
  • 30 Wright MC, Borbolla D, Waller RG. et al. Critical care information display approaches and design frameworks: a systematic review and meta-analysis. J Biomed Inform 2019; 100S: 100041
    Search in Google Scholar
  • 31 Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual Res Sport Exerc Health 2021; 13 (02) 201-216
    Search in Google Scholar
  • 32 Braun V, Clarke V. Conceptual and design thinking for thematic analysis. Qual Psychol 2022; 9 (01) 3-26
    Search in Google Scholar
  • 33 Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res 2016; 26 (13) 1753-1760
    Search in Google Scholar
  • 34 Ancker JS, Benda NC, Reddy M, Unertl KM, Veinot T. Guidance for publishing qualitative research in informatics. J Am Med Inform Assoc 2021; 28 (12) 2743-2748
    Search in Google Scholar
  • 35 QIP Ltd. QSR International 2020. NVivo (released in March 2020).
  • 36 Braun V, Clarke V, Hayfield N, Terry G. Thematic Analysis. In: Liamputtong P. ed. Handbook of Research Methods in Health Social Sciences. Singapore: Springer; 2019
    Search in Google Scholar
  • 37 Braun V, Clarke V. What can “thematic analysis” offer health and wellbeing researchers?. Int J Qual Stud Health Well-being 2014; 9: 26152
    Search in Google Scholar
  • 38 Schütze D, Holtz S, Neff MC. et al. Requirements analysis for an AI-based clinical decision support system for general practitioners: a user-centered design process. BMC Med Inform Decis Mak 2023; 23 (01) 144
    Search in Google Scholar
  • 39 Helman S, Terry MA, Pellathy T. et al. Engaging clinicians early during the development of a graphical user display of an intelligent alerting system at the bedside. Int J Med Inform 2022; 159: 104643
    Search in Google Scholar
  • 40 Reese TJ, Del Fiol G, Tonna JE. et al. Impact of integrated graphical display on expert and novice diagnostic performance in critical care. J Am Med Inform Assoc 2020; 27 (08) 1287-1292
    Search in Google Scholar
  • 41 Reese TJ, Segall N, Del Fiol G. et al. Iterative heuristic design of temporal graphic displays with clinical domain experts. J Clin Monit Comput 2021; 35 (05) 1119-1131
    Search in Google Scholar
  • 42 Hwang J, Lee T, Lee H, Byun S. A clinical decision support system for sleep staging tasks with explanations from artificial intelligence: user-centered design and evaluation study. J Med Internet Res 2022; 24 (01) e28659
    Search in Google Scholar
  • 43 Zhang Z, Citardi D, Wang D, Genc Y, Shan J, Fan X. Patients' perceptions of using artificial intelligence (AI)-based technology to comprehend radiology imaging data. Health Informatics J 2021 ;27(2):14604582211011215
    Search in Google Scholar
  • 44 Cheng L, Senathirajah Y. Using clinical data visualizations in electronic health record user interfaces to enhance medical student diagnostic reasoning: randomized experiment. JMIR Hum Factors 2023; 10: e38941
    Search in Google Scholar
  • 45 Klein GA. A recognition-primed decision (RPD) model of rapid decision making. In: Decision making in action. Ablex; 1993. . Vol. 5(4), pp. 138-147
    Search in Google Scholar
  • 46 Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005; 38 (01) 75-87
    Search in Google Scholar
  • 47 Patel S, Pierce L, Jones M. et al. Using participatory design to engage physicians in the development of a provider-level performance dashboard and feedback system. Jt Comm J Qual Patient Saf 2022; 48 (03) 165-172
    Search in Google Scholar
  • 48 Miller K, Kowalski R, Capan M, Wu P, Mosby D, Arnold R. Assessment of nursing response to a real-time alerting tool for sepsis: a provider survey. Am J Hosp Med 2017; 1 (03) 2017.021
    Search in Google Scholar
  • 49 Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021; 12 (04) 888-896
    Search in Google Scholar