Applying the RE-AIM Framework for the Evaluation of a Clinical Decision Support Tool for Pediatric Head Trauma: A Mixed-Methods Study

Ruth M. Masterson Creber; Peter S. Dayan; Nathan Kuppermann; Dustin W. Ballard; Leah Tzimenatos; Evaline Alessandrini; Rakesh D. Mistry; Jeffrey Hoffman; David R. Vinson; Suzanne Bakken; for the Pediatric Emergency Care Applied Research Network (PECARN) and the Clinical Research on Emergency Services and Treatments (CREST) Network

doi:10.1055/s-0038-1669460

Applied Clinical Informatics, Table of Contents

CC BY-NC-ND 4.0 · Appl Clin Inform 2018; 09(03): 693-703
DOI: 10.1055/s-0038-1669460

Research Article

Georg Thieme Verlag KG Stuttgart · New York

Applying the RE-AIM Framework for the Evaluation of a Clinical Decision Support Tool for Pediatric Head Trauma: A Mixed-Methods Study

Ruth M. Masterson Creber

¹Division of Health Informatics, Department of Healthcare Policy & Research, Weill Cornell Medicine, New York, New York, United States

,

Peter S. Dayan

²Division of Pediatric Emergency Medicine, Columbia University College of Physicians and Surgeons, New York, New York, United States

,

Nathan Kuppermann

³Davis School of Medicine, University of California, Sacramento, California, United States

,

Dustin W. Ballard

⁴Kaiser Permanente, San Rafael Medical Center, San Rafael, California, United States

⁵Kaiser Permanente Division of Research, Oakland, California, United States

,

Leah Tzimenatos

³Davis School of Medicine, University of California, Sacramento, California, United States

,

Evaline Alessandrini

⁶Department of Pediatrics, James M. Anderson Center for Health Systems Excellence and Emergency Medicine, Cincinnati Children's Hospital and Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, United States

,

Rakesh D. Mistry

⁷Department of Pediatrics, Children's Hospital Colorado, University of Colorado Denver, Aurora, Colorado, United States

,

Jeffrey Hoffman

⁸Division of Emergency Medicine, Department of Pediatrics, Nationwide Children's Hospital, Columbus, Ohio, United States

,

David R. Vinson

⁵Kaiser Permanente Division of Research, Oakland, California, United States

⁹Kaiser Permanente, Roseville Medical Center, Roseville, California, United States

,

Suzanne Bakken

¹⁰School of Nursing, Columbia University, New York, New York, United States

¹¹Department of Biomedical Informatics, Columbia University, New York, New York, United States

,

for the Pediatric Emergency Care Applied Research Network (PECARN) and the Clinical Research on Emergency Services and Treatments (CREST) Network

› Author Affiliations

Abstract

Background The overuse of cranial computed tomography (CT) to diagnose potential traumatic brain injuries (TBIs) exposes children with minor blunt head trauma to unnecessary ionizing radiation. The Pediatric Emergency Care Applied Research Network and the Clinical Research on Emergency Services and Treatments Network implemented TBI prediction rules via electronic health record (EHR) clinical decision support (CDS) to decrease use of CTs in children with minor blunt head trauma.

Objective This article aims to facilitate implementation and dissemination of a CDS alert into emergency departments around the country.

Methods We evaluated the EHR CT CDS tool through a mixed-methods analysis of 38 audio-recorded interviews with health care stakeholders and quantitative data sources, using the Reach, Efficacy, Adoption, Implementation, and Maintenance framework.

Results Reach —The demographics of participants enrolled in the clinical trial were consistent with national estimates of TBI prevalence. Efficacy—There was a variable and modest reduction in CT rates for the 8,067 children with minor head trauma whose clinicians received CDS. Adoption —The EHR CT CDS tool was well matched with the organizational mission, values, and priorities of the implementation sites. Implementation— The most important predisposing factors for successful implementation were the presence of an approachable clinical champion at each site and belief that the tool was a relevant, reusable knowledge asset. Enabling factors included an effective integration within the clinical workflow, organizational investment in user training, and ease of use. Maintenance — Reinforcing factors for the EHR CT CDS tool included a close fit with the institutional culture, belief that it was useful for providers and families, and a good educational and informational tool. As such, the EHR CT CDS tool was maintained in clinical practice long after study completion.

Conclusion Data from this mixed-methods study complement findings from the efficacy trial and provide critical components for consideration prior to integration and subsequent dissemination of the EHR CT CDS tool.

Trial Registration NCT01453621, Registered September 27, 2011

Keywords

clinical decision support - clinical trial - blunt head trauma - traumatic brain injury - implementation - child

Full Text

References

References
1 Centers for Disease Control and Prevention. Rates of TBI-related emergency department visits, hospitalizations, and deaths — United States, 2001–2010. Secondary Rates of TBI-related emergency department visits, hospitalizations, and deaths — United States, 2001–2010; 2016 . Available at: https://www.cdc.gov/traumaticbraininjury/data/rates.html . Accessed August 13, 2018
2 Center for Disease Control and Prevention. Rates of TBI-related emergency department visits by age group — United States, 2001–2010. Secondary Rates of TBI-related emergency department visits by age group — United States, 2001–2010; 2016 . Available at: https://www.cdc.gov/traumaticbraininjury/data/rates_ed_byage.html . Accessed August 13, 2018
3 Miglioretti DL, Johnson E, Williams A. , et al. The use of computed tomography in pediatrics and the associated radiation exposure and estimated cancer risk. JAMA Pediatr 2013; 167 (08) 700-707
4 Marin JR, Weaver MD, Barnato AE, Yabes JG, Yealy DM, Roberts MS. Variation in emergency department head computed tomography use for pediatric head trauma. Acad Emerg Med 2014; 21 (09) 987-995
5 Kuppermann N, Holmes JF, Dayan PS. , et al; Pediatric Emergency Care Applied Research Network (PECARN). Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet 2009; 374 (9696): 1160-1170
6 Dayan PS, Ballard DW, Tham E. , et al; Pediatric Emergency Care Applied Research Network (PECARN); Clinical Research on Emergency Services and Treatment (CREST) Network; and Partners Healthcare; Traumatic Brain Injury-Knowledge Translation Study Group. Use of traumatic brain injury prediction rules with clinical decision support. Pediatrics 2017; 139 (04) e20162709
7 Tham E, Swietlik M, Deakyne S. , et al; Pediatric Emergency Care Applied Research Network (PECARN). Clinical decision support for a multicenter trial of pediatric head trauma. Appl Clin Inform 2016; 7 (02) 534-542
8 Goldberg HS, Paterno MD, Grundmeier RW. , et al. Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma. Int J Med Inform 2016; 87: 101-110
9 Deakyne SJ, Bajaj L, Hoffman J. , et al; Pediatric Emergency Care Applied Research Network (PECARN). Development, evaluation and implementation of chief complaint groupings to activate data collection: a multi-center study of clinical decision support for children with head trauma. Appl Clin Inform 2015; 6 (03) 521-535
10 Sheehan B, Nigrovic LE, Dayan PS. , et al; Pediatric Emergency Care Applied Research Network (PECARN). Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis. J Biomed Inform 2013; 46 (05) 905-913
11 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
12 Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res 2006; 21 (05) 688-694
13 Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help?. J Am Med Inform Assoc 2009; 16 (06) 889-897
14 Green LW, Kreuter MW. Health Promotion Planning: An Educational and Ecological Approach. 4th ed. Boston, MA: McGraw-Hill; 2005
15 Holmes-Rovner M, Valade D, Orlowski C, Draus C, Nabozny-Valerio B, Keiser S. Implementing shared decision-making in routine practice: barriers and opportunities. Health Expect 2000; 3 (03) 182-191
16 Aronsky D, Chan KJ, Haug PJ. Evaluation of a computerized diagnostic decision support system for patients with pneumonia: study design considerations. J Am Med Inform Assoc 2001; 8 (05) 473-485
17 Faul MXL, Wald MM, Coronado VG. . Traumatic Brain Injury in the United States: Emergency Department Visits, Hospitalizations and Deaths 2002–2006. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; 2010
18 EMR 2016: The Market for Electronic Medical Records by Kalorama Information. Available at: https://www.kaloramainformation.com/EMR-Electronic-Medical-10009693/ . Accessed August 27, 2018
19 Monegain B. . Kalorama: Cerner, McKesson earn EHR marketshare lead, Epic and Allscripts follow. Secondary Kalorama: Cerner, McKesson earn EHR marketshare lead, Epic and Allscripts follow; 2016 . Available at: http://www.healthcareitnews.com/news/kalorama-cerner-mckesson-earn-ehr-marketshare-lead-epic-and-allscripts-follow . Accessed August 13, 2018
20 Epic software. Epic fact sheet. In: Communications ET, ed.; 2017
21 Sittig DF, Wright A, Osheroff JA. , et al. Grand challenges in clinical decision support. J Biomed Inform 2008; 41 (02) 387-392
22 Miller A, Moon B, Anders S, Walden R, Brown S, Montella D. Integrating computerized clinical decision support systems into clinical work: a meta-synthesis of qualitative research. Int J Med Inform 2015; 84 (12) 1009-1018
23 Hess EP, Wyatt KD, Kharbanda AB. , et al. Effectiveness of the head CT choice decision aid in parents of children with minor head trauma: study protocol for a multicenter randomized trial. Trials 2014; 15: 253
24 Bressan S, Romanato S, Mion T, Zanconato S, Da Dalt L. Implementation of adapted PECARN decision rule for children with minor head injury in the pediatric emergency department. Acad Emerg Med 2012; 19 (07) 801-807
25 Ballard DW, Vemula R, Chettipally UK. , et al; KP CREST Network Investigators. Optimizing clinical decision support in the electronic health record. clinical characteristics associated with the use of a decision tool for disposition of ED patients with pulmonary embolism. Appl Clin Inform 2016; 7 (03) 883-898
26 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
27 Finlayson M, Cattaneo D, Cameron M. , et al. Applying the RE-AIM framework to inform the development of a multiple sclerosis falls-prevention intervention. Int J MS Care 2014; 16 (04) 192-197
28 Meeker D, Linder JA, Fox CR. , et al. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA 2016; 315 (06) 562-570
29 Ivers N, Jamtvedt G, Flottorp S. , et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; (06) CD000259
30 Le Grand Rogers R, Narvaez Y, Venkatesh AK. , et al. Improving emergency physician performance using audit and feedback: a systematic review. Am J Emerg Med 2015; 33 (10) 1505-1514