Design and Implementation of a Pediatric ICU Acuity Scoring Tool as Clinical Decision Support

Abstract

Background and Objective Pediatric in-hospital cardiac arrest most commonly occurs in the pediatric intensive care unit (PICU) and is frequently preceded by early warning signs of clinical deterioration. In this study, we describe the implementation and evaluation of criteria to identify high-risk patients from a paper-based checklist into a clinical decision support (CDS) tool in the electronic health record (EHR).

Materials and Methods The validated paper-based tool was first adapted by PICU clinicians and clinical informaticians and then integrated into clinical workflow following best practices for CDS design. A vendor-based rule engine was utilized. Littenberg's assessment framework helped guide the overall evaluation. Preliminary testing took place in EHR development environments with more rigorous evaluation, testing, and feedback completed in the live production environment. To verify data quality of the CDS rule engine, a retrospective Structured Query Language (SQL) data query was also created. As a process metric, preparedness was measured in pre- and postimplementation surveys.

Results The system was deployed, evaluating approximately 340 unique patients monthly across 4 clinical teams. The verification against retrospective SQL of 15-minute intervals over a 30-day period revealed no missing triggered intervals and demonstrated 99.3% concordance of positive triggers. Preparedness showed improvements across multiple domains to our a priori goal of 90%.

Conclusion We describe the successful adaptation and implementation of a real-time CDS tool to identify PICU patients at risk of deterioration. Prospective multicenter evaluation of the tool's effectiveness on clinical outcomes is necessary before broader implementation can be recommended.

Keywords

decision support systems - cardiopulmonary arrest - pediatrics

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was approved by the Institutional Review Board of CHOP with a waiver of consent.

• References

• 1 Berg RA, Sutton RM, Holubkov R. , et al; Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network and for the American Heart Association's Get With the Guidelines-Resuscitation (formerly the National Registry of Cardiopulmonary Resuscitation) Investigators. Ratio of PICU versus ward cardiopulmonary resuscitation events is increasing. Crit Care Med 2013; 41 (10) 2292-2297
• 2 Knudson JD, Neish SR, Cabrera AG. , et al. Prevalence and outcomes of pediatric in-hospital cardiopulmonary resuscitation in the United States: an analysis of the Kids' Inpatient Database. Crit Care Med 2012; 40 (11) 2940-2944
• 3 Morrison LJ, Neumar RW, Zimmerman JL. , et al; American Heart Association Emergency Cardiovascular Care Committee, Council on Cardiopulmonary, Critical Care, Perioperative and Resuscitation, Council on Cardiovascular and Stroke Nursing, Council on Clinical Cardiology, and Council on P. Strategies for improving survival after in-hospital cardiac arrest in the United States: 2013 consensus recommendations: a consensus statement from the American Heart Association. Circulation 2013; 127 (14) 1538-1563
• 4 Parshuram CS, Hutchison J, Middaugh K. Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care 2009; 13 (04) R135
• 5 Bonafide CP, Holmes JH, Nadkarni VM, Lin R, Landis JR, Keren R. Development of a score to predict clinical deterioration in hospitalized children. J Hosp Med 2012; 7 (04) 345-349
• 6 Duncan H, Hutchison J, Parshuram CS. The Pediatric Early Warning System score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care 2006; 21 (03) 271-278
• 7 Parshuram CS, Duncan HP, Joffe AR. , et al. Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Crit Care 2011; 15 (04) R184
• 8 Haines C, Perrott M, Weir P. Promoting care for acutely ill children-development and evaluation of a paediatric early warning tool. Intensive Crit Care Nurs 2006; 22 (02) 73-81
• 9 Tucker KM, Brewer TL, Baker RB, Demeritt B, Vossmeyer MT. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs 2009; 14 (02) 79-85
• 10 Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med 1996; 24 (05) 743-752
• 11 Edwards ED, Powell CVE, Mason BW, Oliver A. Prospective cohort study to test the predictability of the Cardiff and Vale paediatric early warning system. Arch Dis Child 2009; 94 (08) 602-606
• 12 Sharek PJ, Parast LM, Leong K. , et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a Children's Hospital. JAMA 2007; 298 (19) 2267-2274
• 13 DeVita MA, Smith GB, Adam SK. , et al. “Identifying the hospitalised patient in crisis”—a consensus conference on the afferent limb of rapid response systems. Resuscitation 2010; 81 (04) 375-382
• 14 Bonafide CP, Localio AR, Roberts KE, Nadkarni VM, Weirich CM, Keren R. Impact of rapid response system implementation on critical deterioration events in children. JAMA Pediatr 2014; 168 (01) 25-33
• 15 Sutton RM, Niles D, Meaney PA. , et al. Low-dose, high-frequency CPR training improves skill retention of in-hospital pediatric providers. Pediatrics 2011; 128 (01) e145-e151
• 16 Niles D, Sutton RM, Donoghue A. , et al. “Rolling Refreshers”: a novel approach to maintain CPR psychomotor skill competence. Resuscitation 2009; 80 (08) 909-912
• 17 Sutton RM, Niles D, Meaney PA. , et al. “Booster” training: evaluation of instructor-led bedside cardiopulmonary resuscitation skill training and automated corrective feedback to improve cardiopulmonary resuscitation compliance of Pediatric Basic Life Support providers during simulated cardiac arrest. Pediatr Crit Care Med 2011; 12 (03) e116-e121
• 18 Wolfe H, Zebuhr C, Topjian AA. , et al. Interdisciplinary ICU cardiac arrest debriefing improves survival outcomes*. Crit Care Med 2014; 42 (07) 1688-1695
• 19 Niles DE, Dewan M, Zebuhr C. , et al. A pragmatic checklist to identify pediatric ICU patients at risk for cardiac arrest or code bell activation. Resuscitation 2016; 99: 33-37
• 20 Goldenhar LM, Brady PW, Sutcliffe KM, Muething SE. Huddling for high reliability and situation awareness. BMJ Qual Saf 2013; 22 (11) 899-906
• 21 Endsley MR. Measurement of situation awareness in dynamic systems. Hum Factors J Hum Factors Ergon Soc 1995; 37: 65-84
• 22 Romero-Brufau S, Huddleston JM, Naessens JM. , et al. Widely used track and trigger scores: are they ready for automation in practice?. Resuscitation 2014; 85 (04) 549-552
• 23 Wright MC, Dunbar S, Macpherson BC. , et al. Toward designing information display to support critical care. A qualitative contextual evaluation and visioning effort. Appl Clin Inform 2016; 7 (04) 912-929
• 24 Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45 (06) 1202-1216
• 25 Wright A, Aaron S, Sittig DF. Testing electronic health records in the “production” environment: an essential step in the journey to a safe and effective health care system. J Am Med Inform Assoc 2017; 24 (01) 188-192
• 26 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
• 27 Littenberg B. Technology assessment in medicine. Acad Med 1992; 67 (07) 424-428
• 28 Rothman MJ, Tepas III JJ, Nowalk AJ. , et al. Development and validation of a continuously age-adjusted measure of patient condition for hospitalized children using the electronic medical record. J Biomed Inform 2017; 66: 180-193
• 29 Escobar GJ, Turk BJ, Ragins A. , et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med 2016; 11 (Suppl. 01) S18-S24
• 30 Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM 2001; 94 (10) 521-526
• 31 Finlay GD, Rothman MJ, Smith RA. Measuring the modified early warning score and the Rothman index: advantages of utilizing the electronic medical record in an early warning system. J Hosp Med 2014; 9 (02) 116-119
• 32 Parshuram CS, Dryden-Palmer K, Farrell C. , et al; Canadian Critical Care Trials Group and the EPOCH Investigators. Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: the epoch randomized clinical trial. JAMA 2018; 319 (10) 1002-1012
• 33 Kirkland LL, Malinchoc M, O'Byrne M. , et al. A clinical deterioration prediction tool for internal medicine patients. Am J Med Qual 2013; 28 (02) 135-142
• 34 Byrne CB, Sherry DS, Mercincavage L. , et al. Key Lessons in Clinical Decision Support Implementation. Westat Tech Report; 2010