Design, Implementation, and Validation of a Pediatric ICU Sepsis Prediction Tool as Clinical Decision SupportFunding None.
02 November 2019
22 January 2020
25 March 2020 (online)
Background Sepsis is an uncontrolled inflammatory reaction caused by infection. Clinicians in the pediatric intensive care unit (PICU) developed a paper-based tool to identify patients at risk of sepsis. To improve the utilization of the tool, the PICU team integrated the paper-based tool as a real-time clinical decision support (CDS) intervention in the electronic health record (EHR).
Objective This study aimed to improve identification of PICU patients with sepsis through an automated EHR-based CDS intervention.
Methods A prospective cohort study of all patients admitted to the PICU from May 2017 to May 2019. A CDS intervention was implemented in May 2018. The CDS intervention screened patients for nonspecific sepsis criteria, temperature dysregulation and a blood culture within 6 hours. Following the screening, an interruptive alert prompted nursing staff to complete a perfusion screen to assess for clinical signs of sepsis. The primary alert performance outcomes included sensitivity, specificity, and positive and negative predictive value. The secondary clinical outcome was completion of sepsis management tasks.
Results During the 1-year post implementation period, there were 45.0 sepsis events per 1,000 patient days over 10,805 patient days. The sepsis alert identified 392 of the 436 sepsis episodes accurately with sensitivity of 92.5%, specificity of 95.6%, positive predictive value of 46.0%, and negative predictive value of 99.7%. Examining only patients with severe sepsis confirmed by chart review, test characteristics fell to a sensitivity of 73.3%, a specificity of 92.5%. Prior to the initiation of the alert, 18.6% (13/70) of severe sepsis patients received recommended sepsis interventions. Following the implementation, 34% (27/80) received these interventions in the time recommended, p = 0.04.
Conclusion An EHR CDS intervention demonstrated strong performance characteristics and improved completion of recommended sepsis interventions.
Protection of Human and Animal Subjects
The purpose of this work was to measure and improve the quality of existing care practices so per our institutional review board's (IRB's) existing guidance regarding projects of this nature, it did not require review via the IRB.
- 1 Despond O, Proulx F, Carcillo JA, Lacroix J. Pediatric sepsis and multiple organ dysfunction syndrome. Curr Opin Pediatr 2001; 13 (03) 247-253
- 2 DuPont HL, Spink WW. Infections due to gram-negative organisms: an analysis of 860 patients with bacteremia at the University of Minnesota Medical Center, 1958-1966. Medicine (Baltimore) 1969; 48 (04) 307-332
- 3 Hartman ME, Linde-Zwirble WT, Angus DC, Watson RS. Trends in the epidemiology of pediatric severe sepsis*. Pediatr Crit Care Med 2013; 14 (07) 686-693
- 4 Carlton EF, Barbaro RP, Iwashyna TJ, Prescott HC. Cost of Pediatric Severe Sepsis Hospitalizations. JAMA Pediatr 2019 ;(August) Doi: 10.1001/jamapediatrics.2019.2570
- 5 Han YY, Carcillo JA, Dragotta MA. , et al. Early reversal of pediatric-neonatal septic shock by community physicians is associated with improved outcome. Pediatrics 2003; 112 (04) 793-799
- 6 Carcillo JA, Kuch BA, Han YY. , et al. Mortality and functional morbidity after use of PALS/APLS by community physicians. Pediatrics 2009; 124 (02) 500-508
- 7 Oliveira CF, Nogueira de Sá FR, Oliveira DSF. , et al. Time- and fluid-sensitive resuscitation for hemodynamic support of children in septic shock: barriers to the implementation of the American College of Critical Care Medicine/Pediatric Advanced Life Support Guidelines in a pediatric intensive care unit in a developing world. Pediatr Emerg Care 2008; 24 (12) 810-815
- 8 Davis AL, Carcillo JA, Aneja RK. , et al. American College of Critical Care Medicine Clinical Practice Parameters for Hemodynamic Support of Pediatric and Neonatal Septic Shock. Crit Care Med 2017; 45 (06) 1061-1093
- 9 Sepsis Collaborative. Available at: https://www.childrenshospitals.org/Programs-and-Services/Quality-Improvement-and-Measurement/Collaboratives/Sepsis . Accessed August 5, 2019
- 10 Cruz AT, Perry AM, Williams EA, Graf JM, Wuestner ER, Patel B. Implementation of goal-directed therapy for children with suspected sepsis in the emergency department. Pediatrics 2011; 127 (03) e758-e766
- 11 Larsen GY, Mecham N, Greenberg R. An emergency department septic shock protocol and care guideline for children initiated at triage. Pediatrics 2011; 127 (06) e1585-e1592
- 12 Paul R, Melendez E, Stack A, Capraro A, Monuteaux M, Neuman MI. Improving adherence to PALS septic shock guidelines. Pediatrics 2014; 133 (05) e1358-e1366
- 13 Lane RD, Funai T, Reeder R, Larsen GY. High reliability pediatric septic shock quality improvement initiative and decreasing mortality. Pediatrics 2016; 138 (04) e20154153
- 14 Lloyd JK, Ahrens EA, Clark D, Dachenhaus T, Nuss KE. Automating a manual sepsis screening tool in a pediatric emergency department. Appl Clin Inform 2018; 9 (04) 803-808
- 15 Westra BL, Landman S, Yadav P, Steinbach M. Secondary analysis of an electronic surveillance system combined with multi-focal interventions for early detection of sepsis. Appl Clin Inform 2017; 8 (01) 47-66
- 16 Semler MW, Weavind L, Hooper MH. , et al. An electronic tool for the evaluation and treatment of sepsis in the ICU: a randomized controlled trial. Crit Care Med 2015; 43 (08) 1595-1602
- 17 Sawyer AM, Deal EN, Labelle AJ. , et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med 2011; 39 (03) 469-473
- 18 Warttig S, Alderson P, Evans DJ, Lewis SR, Kourbeti IS, Smith AF. Automated monitoring compared to standard care for the early detection of sepsis in critically ill patients. Cochrane Database Syst Rev 2018; 6: CD012404
- 19 Despins LA. Automated detection of sepsis using electronic medical record data: a systematic review. J Healthc Qual 2017; 39 (06) 322-333
- 20 Kamaleswaran R, Akbilgic O, Hallman MA, West AN, Davis RL, Shah SH. Applying artificial intelligence to identify physiomarkers predicting severe sepsis in the PICU. Pediatr Crit Care Med 2018; 19 (10) e495-e503
- 21 Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med 2018; 46 (04) 547-553
- 22 Schlapbach LJ, MacLaren G, Festa M. , et al; Australian & New Zealand Intensive Care Society (ANZICS) Centre for Outcomes & Resource Evaluation (CORE) and Australian & New Zealand Intensive Care Society (ANZICS) Paediatric Study Group. Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intensive Care Med 2017; 43 (08) 1085-1096
- 23 Rolnick J, Downing NL, Shepard J. , et al. Validation of test performance and clinical time zero for an electronic health record embedded severe sepsis alert. Appl Clin Inform 2016; 7 (02) 560-572
- 24 Weiss SL, Fitzgerald JC, Pappachan J. , et al; Sepsis Prevalence, Outcomes, and Therapies (SPROUT) Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med 2015; 191 (10) 1147-1157
- 25 Goldstein B, Giroir B, Randolph A. ; International Consensus Conference on Pediatric Sepsis. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med 2005; 6 (01) 2-8
- 26 Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care 2015; 19 (01) 285
- 27 Balamuth F, Alpern ER, Grundmeier RW. , et al. Comparison of two sepsis recognition methods in a pediatric emergency department. Acad Emerg Med 2015; 22 (11) 1298-1306
- 28 Balamuth F, Alpern ER, Abbadessa MK. , et al. Improving recognition of pediatric severe sepsis in the emergency department: contributions of a vital sign-based electronic alert and bedside clinician identification. Ann Emerg Med 2017; 70 (06) 759-768
- 29 Cruz AT, Williams EA, Graf JM. , et al. Test characteristics of an automated age- and temperature-adjusted tachycardia alert in pediatric septic shock. Pediatr Emerg Care 2012; 28 (09) 889-894
- 30 Gyang E, Shieh L, Forsey L, Maggio P. A nurse-driven screening tool for the early identification of sepsis in an intermediate care unit setting. J Hosp Med 2015; 10 (02) 97-103
- 31 Wright A, Phansalkar S, Bloomrosen M. , et al. Best practices in clinical decision support: the case of preventive care reminders. Appl Clin Inform 2010; 1 (03) 331-345
- 32 Bates DW, Kuperman GJ, Wang S. , et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530
- 33 Ruth A, McCracken CE, Fortenberry JD, Hall M, Simon HK, Hebbar KB. Pediatric severe sepsis: current trends and outcomes from the pediatric health information systems database. Pediatr Crit Care Med 2014; 15 (09) 828-838
- 34 Dewan M, Sanchez-Pinto LN. Crystal balls and magic eight balls: the art of developing and implementing automated algorithms in acute care pediatrics. Pediatr Crit Care Med 2019; 20 (12) 1197-1199