Evaluating the Impact of Interruptive Alerts within a Health System: Use, Response Time, and Cumulative Time Burden

Abstract

Background Health systems often employ interruptive alerts through the electronic health record to improve patient care. However, concerns of “alert fatigue” have been raised, highlighting the importance of understanding the time burden and impact of these alerts on providers.

Objectives Our main objective was to determine the total time providers spent on interruptive alerts in both inpatient and outpatient settings. Our secondary objectives were to analyze dwell time for individual alerts and examine both provider and alert-related factors associated with dwell time variance.

Methods We retrospectively evaluated use and response to the 75 most common interruptive (“popup”) alerts between June 1st, 2015 and November 1st, 2016 in a large academic health care system. Alert “dwell time” was calculated as the time between the alert appearing on a provider's screen until it was closed. The total number of alerts and dwell times per provider per month was calculated for inpatient and outpatient alerts and compared across alert type.

Results The median number of alerts seen by a provider was 12 per month (IQR 4–34). Overall, 67% of inpatient and 39% of outpatient alerts were closed in under 3 seconds. Alerts related to patient safety and those requiring more than a single click to proceed had significantly longer median dwell times of 5.2 and 6.7 seconds, respectively. The median total monthly time spent by providers viewing alerts was 49 seconds on inpatient alerts and 28 seconds on outpatient alerts.

Conclusion Most alerts were closed in under 3 seconds and a provider's total time spent on alerts was less than 1 min/mo. Alert fatigue may lie in their interruptive and noncritical nature rather than time burden. Monitoring alert interaction time can function as a valuable metric to assess the impact of alerts on workflow and potentially identify routinely ignored alerts.

Keywords

electronic health records - workflow - decision support systems - patient care - medical order entry systems

Protection of Human and Animal Subjects

No human and/or animal subjects were included in the project. The research was limited to a retrospective review of de-identified data, and no personally identifiable information was collected.

• References

• 1 Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc 2012; 19 (e1): e145-e148
• 2 Bates DW, Leape LL, Cullen DJ. , et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15) 1311-1316
• 3 McDaniel RB, Burlison JD, Baker DK. , et al. Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts. J Am Med Inform Assoc 2016; 23 (e1): e138-e141
• 4 Slight SP, Seger DL, Nanji KC. , et al. Are we heeding the warning signs? Examining providers' overrides of computerized drug-drug interaction alerts in primary care. PLoS One 2013; 8 (12) e85071
• 5 Baysari MT, Tariq A, Day RO, Westbrook JI. Alert override as a habitual behavior—a new perspective on a persistent problem. J Am Med Inform Assoc 2017; 24 (02) 409-412
• 6 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
• 7 Nanji KC, Slight SP, Seger DL. , et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
• 8 Meslin SMM, Zheng WY, Day RO, Tay EMY, Baysari MT. Evaluation of clinical relevance of drug-drug interaction alerts prior to implementation. Appl Clin Inform 2018; 9 (04) 849-855
• 9 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
• 10 Payne TH, Hines LE, Chan RC. , et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22 (06) 1243-1250
• 11 Ojeleye O, Avery A, Gupta V, Boyd M. The evidence for the effectiveness of safety alerts in electronic patient medication record systems at the point of pharmacy order entry: a systematic review. BMC Med Inform Decis Mak 2013; 13 (01) 69
• 12 Weingart SN, Simchowitz B, Shiman L. , et al. Clinicians' assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009; 169 (17) 1627-1632
• 13 Phansalkar S, van der Sijs H, Tucker AD. , et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
• 14 Taylor LK, Tamblyn R. Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform 2004; 107 (Pt 2): 1101-1105
• 15 Dexheimer JW, Kirkendall ES, Kouril M. , et al. The effects of medication alerts on prescriber response in a pediatric hospital. Appl Clin Inform 2017; 8 (02) 491-501
• 16 Makam AN, Lanham HJ, Batchelor K. , et al. Use and satisfaction with key functions of a common commercial electronic health record: a survey of primary care providers. BMC Med Inform Decis Mak 2013; 13 (01) 86
• 17 Varghese P, Wright A, Andersen JM, Yoshida EI, Bates DW. Clinical decision support: the experience at Brigham and women's hospital/partners healthcare. In: Clinical Decision Support Systems. New York, NY: Springer Nature; 2016: 227-244
• 18 Kuperman GJ, Bobb A, Payne TH. , et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14 (01) 29-40
• 19 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
• 20 Murphy DR, Meyer AN, Russo E, Sittig DF, Wei L, Singh H. Burden of inbox notifications in commercial electronic health records. JAMA Intern Med 2016; 176 (04) 559-560
• 21 Dombrowski W, Clay B, O'Brien JD. Electronic health records quantify previously existing phenomenon. JAMA Intern Med 2016; 176 (08) 1234-1235