Pseudorandomized Testing of a Discharge Medication Alert to Reduce Free-Text Prescribing

Abstract

Background Pseudorandomized testing can be applied to perform rigorous yet practical evaluations of clinical decision support tools. We apply this methodology to an interruptive alert aimed at reducing free-text prescriptions. Using free-text instead of structured computerized provider order entry elements can cause medication errors and inequity in care by bypassing medication-based clinical decision support tools and hindering automated translation of prescription instructions.

Objective The objective of this study is to evaluate the effectiveness of an interruptive alert at reducing free-text prescriptions via pseudorandomized testing using native electronic health records (EHR) functionality.

Methods Two versions of an EHR alert triggered when a provider attempted to sign a discharge free-text prescription. The visible version displayed an interruptive alert to the user, and a silent version triggered in the background, serving as a control. Providers were assigned to the visible and silent arms based on even/odd EHR provider IDs. The proportion of encounters with a free-text prescription was calculated across the groups. Alert trigger rates were compared in process control charts. Free-text prescriptions were analyzed to identify prescribing patterns.

Results Over the 28-week study period, 143 providers triggered 695 alerts (345 visible and 350 silent). The proportions of encounters with free-text prescriptions were 83% (266/320) and 90% (273/303) in the intervention and control groups, respectively (p = 0.01). For the active alert, median time to action was 31 seconds. Alert trigger rates between groups were similar over time. Ibuprofen, oxycodone, steroid tapers, and oncology-related prescriptions accounted for most free-text prescriptions. A majority of these prescriptions originated from user preference lists.

Conclusion An interruptive alert was associated with a modest reduction in free-text prescriptions. Furthermore, the majority of these prescriptions could have been reproduced using structured order entry fields. Targeting user preference lists shows promise for future intervention.

Keywords

clinical decision support - medication safety - randomized controlled trials - pediatrics - quality improvement

Protection of Human and Animal Subjects

The preceding work was performed as part of a quality improvement effort at our institution and does not qualify as human subjects research.

Publikationsverlauf

Eingereicht: 11. November 2022

Angenommen: 03. April 2023

Accepted Manuscript online:
04. April 2023

Artikel online veröffentlicht:
14. Juni 2023

• References

• 1 Horwitz LI, Kuznetsova M, Jones SA. Creating a learning health system through rapid-cycle, randomized testing. N Engl J Med 2019; 381 (12) 1175-1179
• 2 Austrian J, Mendoza F, Szerencsy A. et al. Applying A/B testing to clinical decision support: rapid randomized controlled trials. J Med Internet Res 2021; 23 (04) e16651
• 3 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
• 4 Finkelstein A. A strategy for improving U.S. health care delivery - conducting more randomized, controlled trials. N Engl J Med 2020; 382 (16) 1485-1488
• 5 Zhou L, Mahoney LM, Shakurova A. et al. How many medication orders are entered through free-text in EHRs? A study on hypoglycemic agents. AMIA Annu Symp Proc 2012; 2012: 1079-1088
• 6 Morse KE, Chadwick WA, Paul W, Haaland W, Pageler NM, Tarrago R. Quantifying discharge medication reconciliation errors at 2 pediatric hospitals. Pediatr Qual Saf 2021; 6 (04) e436
• 7 Kandaswamy S, Pruitt Z, Kazi S. et al. Clinician perceptions on the use of free-text communication orders. Appl Clin Inform 2021; 12 (03) 484-494
• 8 McDaniel RB, Burlison JD, Baker DK. et al. Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts. J Am Med Inform Assoc 2016; 23 (e1): e138-e141
• 9 Arthur J. Control Chart White Paper [Internet]. 2021 Accessed September 1, 2022 at: https://www.qimacros.com/pdf/control-chart-whitepaper.pdf
• 10 Provost LP, Murray SK. The Health Care Data Guide: Learning from Data for Improvement. Hoboken, New Jersey: John Wiley & Sons; 2022: 656
• 11 Kwan JL, Lo L, Sampson M, Shojania KG. Medication reconciliation during transitions of care as a patient safety strategy: a systematic review. Ann Intern Med 2013; 158 (5 Pt 2): 397-403
• 12 Coleman EA, Berenson RA. Lost in transition: challenges and opportunities for improving the quality of transitional care. Ann Intern Med 2004; 141 (07) 533-536
• 13 Cornish PL, Knowles SR, Marchesano R. et al. Unintended medication discrepancies at the time of hospital admission. Arch Intern Med 2005; 165 (04) 424-429
• 14 Coleman EA, Smith JD, Raha D, Min SJ. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med 2005; 165 (16) 1842-1847
• 15 Bell CM, Brener SS, Gunraj N. et al. Association of ICU or hospital admission with unintentional discontinuation of medications for chronic diseases. JAMA 2011; 306 (08) 840-847
• 16 Huynh C, Wong ICK, Tomlin S. et al. Medication discrepancies at transitions in pediatrics: a review of the literature. Paediatr Drugs 2013; 15 (03) 203-215
• 17 Gattari TB, Krieger LN, Hu HM, Mychaliska KP. Medication discrepancies at pediatric hospital discharge. Hosp Pediatr 2015; 5 (08) 439-445
• 18 Hron JD, Manzi S, Dionne R. et al. Electronic medication reconciliation and medication errors. Int J Qual Health Care 2015; 27 (04) 314-319
• 19 Stockton KR, Wickham ME, Lai S. et al. Incidence of clinically relevant medication errors in the era of electronically prepopulated medication reconciliation forms: a retrospective chart review. CMAJ Open 2017; 5 (02) E345-E353
• 20 Rinke ML, Bundy DG, Velasquez CA. et al. Interventions to reduce pediatric medication errors: a systematic review. Pediatrics 2014; 134 (02) 338-360
• 21 Marien S, Krug B, Spinewine A. Electronic tools to support medication reconciliation: a systematic review. J Am Med Inform Assoc 2017; 24 (01) 227-240
• 22 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
• 23 Nanji KC, Seger DL, Slight SP. et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc 2018; 25 (05) 476-481
• 24 Tolley CL, Slight SP, Husband AK, Watson N, Bates DW. Improving medication-related clinical decision support. Am J Health Syst Pharm 2018; 75 (04) 239-246
• 25 Westbrook JI, Coiera E, Dunsmuir WTM. et al. The impact of interruptions on clinical task completion. Qual Saf Health Care 2010; 19 (04) 284-289
• 26 Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions with an increased risk and severity of medication administration errors. Arch Intern Med 2010; 170 (08) 683-690
• 27 Bonafide CP, Miller JM, Localio AR. et al. Association between mobile telephone interruptions and medication administration errors in a pediatric intensive care unit. JAMA Pediatr 2020; 174 (02) 162-169
• 28 Orenstein EW, Kandaswamy S, Muthu N. et al. Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics. J Am Med Inform Assoc 2021; 28 (12) 2654-2660
• 29 Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (01) 46-58
• 30 Chaparro JD, Beus JM, Dziorny AC. et al. Clinical decision support stewardship: best practices and techniques to monitor and improve interruptive alerts. Appl Clin Inform 2022; 13 (03) 560-568
• 31 Downing NL, Rolnick J, Poole SF. et al. Electronic health record-based clinical decision support alert for severe sepsis: a randomised evaluation. BMJ Qual Saf 2019; 28 (09) 762-768