Download PDF
Appl Clin Inform 2026; 17(01): 046-051
DOI: 10.1055/a-2793-0977
Special Issue on CDS Failures

Leveraging 10 Days of Alert Malfunction to Improve Mature Organizational Clinical Decision Support Processes

Authors

  • Daria F. Ferro 

    1   Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    2   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    3   Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

  • Marc Tobias

    4   Phrase Health, Inc., Philadelphia, Pennsylvania, United States

  • Leah H. Carr

    1   Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    2   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    3   Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

  • Pamela Wentz

    5   Center for Healthcare Quality and Analytics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

  • Melissa Rodriguez

    6   Digital and Technology Services, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

  • Casey Pitts

    1   Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    2   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    3   Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

  • Emily Kane

    1   Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    3   Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States

  • Eric Shelov

    1   Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    2   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
    3   Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States
Further Information
Also available at
   Permissions and Reprints

Abstract

Background

Interruptive clinical decision support (CDS) alerts are intended to standardize patient care and prevent harm. However, failures can occur even in organizations with mature CDS governance and advanced analytics. These breakdowns, marked by excessive firings, workflow disruption, and clinician dissatisfaction, can provide insights into systemic weaknesses in CDS design, testing, and monitoring processes.

Objectives

This study aimed to examine a CDS alert malfunction as a lens for identifying system-level gaps and propose strategies to strengthen resilience in CDS operations.

Methods

A retrospective analysis was conducted on an interruptive alert that was developed through a phased, multistakeholder, committee-driven process, but was removed within 10 days due to poor performance, revealing gaps that persisted despite established governance.

Results

The alert fired 1,866 times in 5 days, with a 91% dismissal rate and reports of workflow disruption. Feedback indicated provider frustration and concern for malfunction. Analysis revealed gaps in end-user engagement, testing rigor, committee reviews, and monitoring practices.

Conclusion

CDS failures can serve as catalysts for system improvement. This case highlights actionable lessons, such as operationalizing user-centered design, clarifying testing expectations, and distributing monitoring responsibilities, to enhance CDS reliability. Even well-established governance structures must be continuously evaluated and adapted to keep pace with evolving CDS technologies, and such investments position organizations to maintain responsive, sustainable systems aligned with high-quality care.

Keywords

clinical decision support - alerting - organizational change management - system improvement - process improvement - clinical information systems - sociotechnical aspects of information technology - workflows and human interactions

Protection of Human and Animal Subjects

This case review was undertaken as an improvement initiative and as such does not constitute human subjects research.


Declaration of GenAI Use

During the writing process of this paper, the authors used ChatGPT (OpenAI) in order to improving text flow, reducing redundancy, and enhancing overall readability. The author(s) reviewed and edited the text and take(s) full responsibility for the content of the paper.




Publication History

Received: 27 March 2025

Accepted: 20 January 2026

Accepted Manuscript online:
21 January 2026

Article published online:
05 February 2026

© 2026. Thieme. All rights reserved.

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany

 

  • References

  • 1 Osheroff JA, Teich JM, Levick D. et al. Improving Outcomes with Clinical Decision Support: An Implementer's Guide. 2nd ed.. HIMSS; 2012
    Search in Google Scholar
  • 2 Berner ES. ed. Clinical Decision Support Systems: Theory and Practice. 2nd ed.. Springer; 2007
    Search in Google Scholar
  • 3 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
    Search in Google Scholar
  • 4 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494) 765
    Search in Google Scholar
  • 5 Bright TJ, Wong A, Dhurjati R. et al. Effect of clinical decision-support systems: A systematic review. Ann Intern Med 2012; 157 (01) 29-43
    Search in Google Scholar
  • 6 Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: Impact on national ambulatory care quality. Arch Intern Med 2011; 171 (10) 897-903
    Search in Google Scholar
  • 7 Orenstein EW, Muthu N, Weitkamp AO. et al. Towards a maturity model for clinical decision support operations. Appl Clin Inform 2019; 10 (05) 810-819
    Search in Google Scholar
  • 8 Wright A, Sittig DF, Ash JS. et al. Governance for clinical decision support: Case studies and recommended practices from leading institutions. J Am Med Inform Assoc 2009; 16 (03) 232-240
    Search in Google Scholar
  • 9 Phansalkar S, van der Sijs H, Tucker AD. et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
    Search in Google Scholar
  • 10 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: State of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
    Search in Google Scholar
  • 11 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
    Search in Google Scholar
  • 12 Magrabi F, Ong MS, Runciman W. et al. Patient safety problems associated with healthcare information technology: An analysis of adverse event reports. J Am Med Inform Assoc 2012; 19 (06) 1111-1116
    Search in Google Scholar
  • 13 Wright A, Nelson S, Rubins D, Schreiber R, Sittig DF. Clinical decision support malfunctions related to medication routes: A case series. J Am Med Inform Assoc 2022; 29 (11) 1972-1975
    Search in Google Scholar
  • 14 Guzman-Karlsson MCG, Hess LM, Jeppesen AL, Fortunov RM. Redesigning clinical decision support for retinopathy of prematurity screening after alert failure. Appl Clin Inform 2025; 16 (04) 1024-1030
    Search in Google Scholar
  • 15 Clarke T, Kotarski T, Tobias M. The burden of a highly targeted alert. Appl Clin Inform 2025; 16 (03) 732-735
    Search in Google Scholar
  • 16 Plattner AS, Lockowitz CR, Same RG. et al. A rash decision: Implementing an EHR-integrated penicillin allergy delabeling protocol without adequate clinician support. Appl Clin Inform 2025; 16 (04) 1095-1103
    Search in Google Scholar
  • 17 Children's Hospital of Philadelphia. Clinical Pathways Program. Accessed August 8, 2025 at: https://www.chop.edu/pathways-library
  • 18 Kandaswamy S, Williams H, Thompson SA, Dawson TE, Muthu N, Orenstein EW. Realizing the full potential of clinical decision support: Translating usability testing into routine practice in health care operations. Appl Clin Inform 2024; 15 (05) 1039-1048
    Search in Google Scholar
  • 19 Aaron S, McEvoy DS, Ray S, Hickman TT, Wright A. Cranky comments: Detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 26 (01) 37-43
    Search in Google Scholar
  • 20 McCoy AB, Russo EM, Johnson KB. et al. Clinician collaboration to improve clinical decision support: The Clickbusters initiative. J Am Med Inform Assoc 2022; 29 (06) 1050-1059
    Search in Google Scholar
  • 21 Newton N, Bamgboje-Ayodele A, Forsyth R. et al. Opportunities and challenges associated with the pilot implementation of CDS systems: A qualitative study. Appl Clin Inform 2025; 16: 777-785
    Search in Google Scholar
  • 22 Butler JM, Doubleday A, Sattar U. et al. “Be really careful about that”: Clinicians' perceptions of an intelligence augmentation tool for in-hospital deterioration detection. Appl Clin Inform 2025; 16 (02) 377-392
    Search in Google Scholar
  • 23 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (Suppl. 03) i68-i74
    Search in Google Scholar