Appl Clin Inform 2020; 11(04): 589-597
DOI: 10.1055/s-0040-1715893
Research Article

Considerations for Designing EHR-Embedded Clinical Decision Support Systems for Antimicrobial Stewardship in Pediatric Emergency Departments

Mustafa Ozkaynak
1   College of Nursing, University of Colorado-Denver, Anschutz Medical Campus, Aurora, Colorado, United States
Noel Metcalf
1   College of Nursing, University of Colorado-Denver, Anschutz Medical Campus, Aurora, Colorado, United States
Daniel M. Cohen
2   Division of Emergency Medicine, Nationwide Children's Hospital, Columbus, Ohio, United States
Larissa S. May
3   Department of Emergency Medicine, UC Davis Health, Davis, California, United States
Peter S. Dayan
4   Division of Pediatric Emergency Medicine, Department of Emergency Medicine, Columbia University College of Physicians and Surgeons, New York, New York, United States
Rakesh D. Mistry
5   Department of Pediatrics and Emergency Medicine, Section of Emergency Medicine, School of Medicine, University of Colorado, Anschutz Medical Campus, Aurora, Colorado, United States
› Author Affiliations
Funding This study was supported by National Institute of Allergy and Infectious Diseases (NIAID) under award number R21AI139839 (principal investigator: R.D.M.).


Objective This study was aimed to explore the intersection between organizational environment, workflow, and technology in pediatric emergency departments (EDs) and how these factors impact antibiotic prescribing decisions.

Methods Semistructured interviews with 17 providers (1 fellow and 16 attending faculty), and observations of 21 providers (1 physician assistant, 5 residents, 3 fellows, and 12 attendings) were conducted at three EDs in the United States. We analyzed interview transcripts and observation notes using thematic analysis.

Results Seven themes relating to antibiotic prescribing decisions emerged as follows: (1) professional judgement, (2) cognition as a critical individual resource, (3) decision support as a critical organizational resource, (4) patient management with imperfect information, (5) information-seeking as a primary task, (6) time management, and (7) broad process boundaries of antibiotic prescribing.

Discussion The emerging interrelated themes identified in this study can be used as a blueprint to design, implement, and evaluate clinical decision support (CDS) systems that support antibiotic prescribing in EDs. The process boundaries of antibiotic prescribing are broader than the current boundaries covered by existing CDS systems. Incongruities between process boundaries and CDS can under-support clinicians and lead to suboptimal decisions. We identified two incongruities: (1) the lack of acknowledgment that the process boundaries go beyond the physical boundaries of the ED and (2) the lack of integration of information sources (e.g., accessibility to prior cultures on an individual patient outside of the organization).

Conclusion Significant opportunities exist to improve appropriateness of antibiotic prescribing by considering process boundaries in the design, implementation, and evaluation of CDS systems.

Protection of Human and Animal Subjects

The study protocol was approved by the Colorado Multiple Institutional Review Board.

Supplementary Material

Publication History

Received: 24 March 2020

Accepted: 21 July 2020

Article published online:
09 September 2020

Georg Thieme Verlag KG
Stuttgart · New York

  • References

  • 1 Spellberg B, Blaser M, Guidos RJ. Infectious Diseases Society of America (IDSA). , et al; Combating antimicrobial resistance: policy recommendations to save lives. Clin Infect Dis 2011; 52 (Suppl. 05) S397-S428
  • 2 May L, Cosgrove S, L'Archeveque M. , et al. A call to action for antimicrobial stewardship in the emergency department: approaches and strategies. Ann Emerg Med 2013; 62 (01) 69-77
  • 3 Shlaes DM, Gerding DN, John Jr. JF. , et al. Society for Healthcare Epidemiology of America and Infectious Diseases Society of America Joint Committee on the prevention of antimicrobial resistance: guidelines for the prevention of antimicrobial resistance in hospitals. Clin Infect Dis 1997; 25 (03) 584-599
  • 4 Centers for Disease Control and Prevention. Antibiotic/antimicrobial resistance (AR/AMR). . Accessed August 5, 2020 at:
  • 5 Mistry RD, Dayan PS, Kuppermann N. The battle against antimicrobial resistance: time for the emergency department to join the fight. JAMA Pediatr 2015; 169 (05) 421-422
  • 6 Donnelly JP, Baddley JW, Wang HE. Antibiotic utilization for acute respiratory tract infections in U.S. emergency departments. Antimicrob Agents Chemother 2014; 58 (03) 1451-1457
  • 7 Kroening-Roche JC, Soroudi A, Castillo EM, Vilke GM. Antibiotic and bronchodilator prescribing for acute bronchitis in the emergency department. J Emerg Med 2012; 43 (02) 221-227
  • 8 Paul IM, Maselli JH, Hersh AL, Boushey HA, Nielson DW, Cabana MD. Antibiotic prescribing during pediatric ambulatory care visits for asthma. Pediatrics 2011; 127 (06) 1014-1021
  • 9 Society for Healthcare Epidemiology of America; Infectious Diseases Society of America; Pediatric Infectious Diseases Society. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Infect Control Hosp Epidemiol 2012; 33 (04) 322-327
  • 10 Avdic E, Cushinotto LA, Hughes AH. , et al. Impact of an antimicrobial stewardship intervention on shortening the duration of therapy for community-acquired pneumonia. Clin Infect Dis 2012; 54 (11) 1581-1587
  • 11 Fishman N. Antimicrobial stewardship. Am J Infect Control 2006; 34 (05) (Suppl. 01) S55-S63
  • 12 Ohl CA, Luther VP. Antimicrobial stewardship for inpatient facilities. J Hosp Med 2011; 6 (Suppl. 01) S4-S15
  • 13 Percival KM, Valenti KM, Schmittling SE, Strader BD, Lopez RR, Bergman SJ. Impact of an antimicrobial stewardship intervention on urinary tract infection treatment in the ED. Am J Emerg Med 2015; 33 (09) 1129-1133
  • 14 Ansari F, Gray K, Nathwani D. , et al. Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J Antimicrob Chemother 2003; 52 (05) 842-848
  • 15 Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med 2002; 9 (11) 1184-1204
  • 16 Goldberg RM, Kuhn G, Andrew LB, Thomas Jr HA. Coping with medical mistakes and errors in judgment. Ann Emerg Med 2002; 39 (03) 287-292
  • 17 Grenier C, Pépin J, Nault V. , et al. Impact of guideline-consistent therapy on outcome of patients with healthcare-associated and community-acquired pneumonia. J Antimicrob Chemother 2011; 66 (07) 1617-1624
  • 18 Ozkaynak M, Wu DTY, Hannah K, Dayan PS, Mistry RD. Examining workflow in a pediatric emergency department to develop a clinical decision support for an antimicrobial stewardship program. Appl Clin Inform 2018; 9 (02) 248-260
  • 19 Shaw B, Cheater F, Baker R. , et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2005; (03) CD005470
  • 20 Agwu AL, Lee CK, Jain SK. , et al. A world wide web-based antimicrobial stewardship program improves efficiency, communication, and user satisfaction and reduces cost in a tertiary care pediatric medical center. Clin Infect Dis 2008; 47 (06) 747-753
  • 21 Evans RS, Olson JA, Stenehjem E. , et al. Use of computer decision support in an antimicrobial stewardship program (ASP). Appl Clin Inform 2015; 6 (01) 120-135
  • 22 Sick AC, Lehmann CU, Tamma PD, Lee CK, Agwu AL. Sustained savings from a longitudinal cost analysis of an internet-based preapproval antimicrobial stewardship program. Infect Control Hosp Epidemiol 2013; 34 (06) 573-580
  • 23 Venugopal V, Lehmann CU, Diener-West M, Agwu AL. Longitudinal evaluation of a World Wide Web-based antimicrobial stewardship program: assessing factors associated with approval patterns and trends over time. Am J Infect Control 2014; 42 (02) 100-105
  • 24 Webber EC, Warhurst HM, Smith SS, Cox EG, Crumby AS, Nichols KR. Conversion of a single-facility pediatric antimicrobial stewardship program to multi-facility application with computerized provider order entry and clinical decision support. Appl Clin Inform 2013; 4 (04) 556-568
  • 25 Mistry RD, Newland JG, Gerber JS. , et al. Current state of antimicrobial stewardship in children's hospital emergency departments. Infect Control Hosp Epidemiol 2017; 38 (04) 469-475
  • 26 Simpao AF, Ahumada LM, Larru Martinez B. , et al. Design and implementation of a visual analytics electronic antibiogram within an electronic health record system at a tertiary pediatric hospital. Appl Clin Inform 2018; 9 (01) 37-45
  • 27 Sanchez GV, Fleming-Dutra KE, Roberts RM, Hicks LA. Core elements of outpatient antibiotic stewardship. MMWR Recomm Rep 2016; 65 (06) 1-12
  • 28 Bates DW, Kuperman GJ, Wang S. , et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530
  • 29 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
  • 30 Kawamoto K, Del Fiol G, Lobach DF, Jenders RA. Standards for scalable clinical decision support: need, current and emerging standards, gaps, and proposal for progress. Open Med Inform J 2010; 4: 235-244
  • 31 Gerber JS, Prasad PA, Fiks AG. , et al. Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback. JAMA 2014; 312 (23) 2569-2570
  • 32 Kullar R, Goff DA. Transformation of antimicrobial stewardship programs through technology and informatics. Infect Dis Clin North Am 2014; 28 (02) 291-300
  • 33 Kullar R, Goff DA, Schulz LT, Fox BC, Rose WE. The “epic” challenge of optimizing antimicrobial stewardship: the role of electronic medical records and technology. Clin Infect Dis 2013; 57 (07) 1005-1013
  • 34 Liberati EG, Ruggiero F, Galuppo L. , et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 2017; 12 (01) 113
  • 35 Rawson TM, Moore LSP, Hernandez B. , et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately?. Clin Microbiol Infect 2017; 23 (08) 524-532
  • 36 Wadhwa R, Fridsma DB, Saul MI. , et al. Analysis of a failed clinical decision support system for management of congestive heart failure. AMIA Annu Symp Proc 2008; 2008: 773-777
  • 37 Karsh BT, Alper SJ. Work system analysis: the key to understanding health care systems. In: Henriksen K, Battles JB, Marks ES, Lewin DI. , eds. Advances in Patient Safety: From Research to Implementation (Volume 2: Concepts and Methodology). Rockville, MD: AHRQ Publications; 2005: 337-348
  • 38 Ozkaynak M, Ponnala S, Werner NE. Patient-oriented workflow approach. In: Zheng K, Westbrook J, Kannampallil TG, Patel VL. , eds. Cognitive Informatics: Reengineering Clinical Workflow for Safer and More Efficient Care. Cham: Springer International Publishing; 2019: 149-64
  • 39 Ozkaynak M, Valdez R, Holden RJ, Weiss J. Infinicare framework for integrated understanding of health-related activities in clinical and daily-living contexts. Health Syst (Basingstoke) 2017; 7 (01) 66-78
  • 40 Walker JM, Carayon P. From tasks to processes: the case for changing health information technology to improve health care. Health Aff (Millwood) 2009; 28 (02) 467-477
  • 41 Creswell JW. Research Design Qualitative, Quantitative, and Mixed Methods Approaches. 3rd ed. Thousand Oaks, CA: Sage Publications; 2009
  • 42 Robson C. Real World Research. 3rd ed. Chichester, United Kingdom: John Wiley & Sons; 2011
  • 43 Ozkaynak M, Reeder B, Drake C, Ferrarone P, Trautner B, Wald H. Characterizing Workflow to Inform Clinical Decision Support Systems in Nursing Homes. Gerontologist 2019; 59 (06) 1024-1033
  • 44 Lavrakas PJ. , ed. Encyclopedia of Survey Research Methods. Thousand Oaks, CA: Sage Publications; 2008
  • 45 Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: Sage Publications; 1998
  • 46 Chung P, Scandlyn J, Dayan PS, Mistry RD. Working at the intersection of context, culture, and technology: provider perspectives on antimicrobial stewardship in the emergency department using electronic health record clinical decision support. Am J Infect Control 2017; 45 (11) 1198-1202
  • 47 Georgiou A, Prgomet M, Paoloni R. , et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med 2013; 61 (06) 644-653.e16
  • 48 May L, Gudger G, Armstrong P. , et al. Multisite exploration of clinical decision making for antibiotic use by emergency medicine providers using quantitative and qualitative methods. Infect Control Hosp Epidemiol 2014; 35 (09) 1114-1125
  • 49 McCaffery KJ, Smith SK, Wolf M. The challenge of shared decision making among patients with lower literacy: a framework for research and development. Med Decis Making 2010; 30 (01) 35-44
  • 50 Masterson Creber RM, Dayan PS, Kuppermann N. Pediatric Emergency Care Applied Research Network (PECARN) and the Clinical Research on Emergency Services and Treatments (CREST) Network. , et al; Applying the RE-AIM framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study. Appl Clin Inform 2018; 9 (03) 693-703
  • 51 Harle CA, Apathy NC, Cook RL. Information Needs and Requirements for Decision Support in Primary Care: An Analysis of Chronic Pain Care. AMIA Annual Symposium proceedings AMIA Symposium. 2018 ;2018: 527-34
  • 52 Pauwen NY, Louis E, Siegel C, Colombel J-F, Macq J. Integrated care for crohn's disease: a plea for the development of clinical decision support systems. J Crohn's Colitis 2018; 12 (12) 1499-1504
  • 53 Shenoy ES, Macy E, Rowe T, Blumenthal KG. Evaluation and management of penicillin allergy: a review. JAMA 2019; 321 (02) 188-199
  • 54 Vyles D, Chiu A, Routes J. , et al. Antibiotic use after removal of penicillin allergy label. Pediatrics 2018; 141 (05) e20173466
  • 55 Chin-Yee B, Upshur R. Clinical judgement in the era of big data and predictive analytics. J Eval Clin Pract 2018; 24 (03) 638-645
  • 56 Fox J, Patkar V, Chronakis I, Begent R. From practice guidelines to clinical decision support: closing the loop. J R Soc Med 2009; 102 (11) 464-473
  • 57 Horsky J, Aarts J, Verheul L, Seger DL, van der Sijs H, Bates DW. Clinical reasoning in the context of active decision support during medication prescribing. Int J Med Inform 2017; 97: 1-11
  • 58 Marcial LH, Johnston DS, Shapiro MR, Jacobs SR, Blumenfeld B, Rojas Smith L. A qualitative framework-based evaluation of radiology clinical decision support initiatives: eliciting key factors to physician adoption in implementation. JAMIA Open 2019; 2 (01) 187-196
  • 59 Miller A, Koola JD, Matheny ME. , et al. Application of contextual design methods to inform targeted clinical decision support interventions in sub-specialty care environments. Int J Med Inform 2018; 117: 55-65
  • 60 Richardson JE, Middleton B, Platt JE, Blumenfeld BH. Building and maintaining trust in clinical decision support: recommendations from the patient-centered CDS learning network. Learn Health Syst 2019; 4 (02) e10208
  • 61 Altmann EM, Trafton J. , eds. Task Interruption: Resumption Lag and the Role of Cues. Proceedings of the Twenty-Sixth Annual Conference of the Cognitive Science Society. Annual Meeting of the Cognitive Science Society; 2005: 43-48
  • 62 Gillie T, Broadbent D. What makes interruptions disruptive? A study of length, similarity, and complexity. Psychol Res 1989; 50 (04) 243-250
  • 63 Relihan E, O'Brien V, O'Hara S, Silke B. The impact of a set of interventions to reduce interruptions and distractions to nurses during medication administration. Qual Saf Health Care 2010; 19 (05) e52
  • 64 Rivera-Rodriguez AJ, Karsh B-T. Interruptions and distractions in healthcare: review and reappraisal. Qual Saf Health Care 2010; 19 (04) 304-312