Evaluation of Clinical Relevance of Drug–Drug Interaction Alerts Prior to ImplementationFunding This study was funded by National Health and Medical Research Council Program Grant APP1054146.
17 July 2018
12 October 2018
28 November 2018 (online)
Introduction Drug–drug interaction (DDI) alerts are often implemented in the hospital computerized provider order entry (CPOE) systems with limited evaluation. This increases the risk of prescribers experiencing too many irrelevant alerts, resulting in alert fatigue. In this study, we aimed to evaluate clinical relevance of alerts prior to implementation in CPOE using two common approaches: compendia and expert panel review.
Methods After generating a list of hypothetical DDI alerts, that is, alerts that would have been triggered if DDI alerts were operational in the CPOE, we calculated the agreement between multiple drug interaction compendia with regards to the severity of these alerts. A subset of DDI alerts (n = 13), with associated patient information, were presented to an expert panel to reach a consensus on whether each alert should be included in the CPOE.
Results There was poor agreement between compendia in their classifications of DDI severity (Krippendorff's α: 0.03; 95% confidence interval: –0.07 to 0.14). Only 10% of DDI alerts were classed as severe by all compendia. On the other hand, the panel reached consensus on 12 of the 13 alerts that were presented to them regarding whether they should be included in the CPOE.
Conclusion Using an expert panel and allowing them to discuss their views openly likely resulted in high agreement on what alerts should be included in a CPOE system. Presenting alerts in the context of patient cases allowed panelists to identify the conditions under which alerts were clinically relevant. The poor agreement between compendia suggests that this methodology may not be ideal for the evaluation of DDI alerts. Performing preimplementation review of DDI alerts before they are enabled provides an opportunity to minimize the risk of alert fatigue before prescribers are exposed to false-positive alerts.
Keywordsmedical order entry systems - drug interactions - alert fatigue - alert systems - clinical decision support
Protection of Human and Animal Subjects
Ethics approval was obtained by the local hospital's ethics board.
- 1 Day RO, Snowden L, McLachlan AJ. Life-threatening drug interactions: what the physician needs to know. Intern Med J 2017; 47 (05) 501-512
- 2 Magro L, Moretti U, Leone R. Epidemiology and characteristics of adverse drug reactions caused by drug-drug interactions. Expert Opin Drug Saf 2012; 11 (01) 83-94
- 3 Schachter M. The epidemiology of medication errors: how many, how serious?. Br J Clin Pharmacol 2009; 67 (06) 621-623
- 4 Beeler PE, Bates DW, Hug BL. Clinical decision support systems. Swiss Med Wkly 2014; 144: w14073
- 5 Zheng WY, Richardson LC, Li L, Day RO, Westbrook JI, Baysari MT. Drug-drug interactions and their harmful effects in hospitalised patients: a systematic review and meta-analysis. Eur J Clin Pharmacol 2018; 74 (01) 15-27
- 6 Peterson JF, Bates DW. Preventable medication errors: identifying and eliminating serious drug interactions. J Am Pharm Assoc (Wash) 2001; 41 (02) 159-160
- 7 Slight SP, Seger DL, Nanji KC. , et al. Are we heeding the warning signs? Examining providers' overrides of computerized drug-drug interaction alerts in primary care. PLoS One 2013; 8 (12) e85071
- 8 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
- 9 Fung KW, Kapusnik-Uner J, Cunningham J, Higby-Baker S, Bodenreider O. Comparison of three commercial knowledge bases for detection of drug-drug interactions in clinical decision support. J Am Med Inform Assoc 2017; 24 (04) 806-812
- 10 Payne TH, Nichol WP, Hoey P. , et al. Characteristics and override rates of order checks in a practitioner order entry system. Proceedings of AMIA Symposium; 2002 :602–606
- 11 van der Sijs H, Aarts J, van Gelder T, Berg M, Vulto A. Turning off frequently overridden drug alerts: limited opportunities for doing it safely. J Am Med Inform Assoc 2008; 15 (04) 439-448
- 12 Paterno MD, Maviglia SM, Gorman PN. , et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16 (01) 40-46
- 13 Beccaro MAD, Villanueva R, Knudson KM, Harvey EM, Langle JM, Paul W. Decision support alerts for medication ordering in a computerized provider order entry (CPOE) system: a systematic approach to decrease alerts. Appl Clin Inform 2010; 1 (03) 346-362
- 14 HiMSS Analytics. Electronic Medical Record Adoption Model; 2017 . Available at: https://www.himssanalytics.org/emram . Accessed October 31, 2018
- 15 Medicare and Medicaid Services. Electronic Health Record Incentive Program: Centers for Medicare and Medicaid Services; 2010 . Available at: https://www.federalregister.gov/documents/2010/07/28/2010-17207/medicare-and-medicaid-programs-electronic-health-record-incentive-program . Accessed October 31, 2018
- 16 Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the Meaningful Use era: no evidence of progress. Appl Clin Inform 2014; 5 (03) 802-813
- 17 Baysari MT, Westbrook JI, Egan B, Day RO. Identification of strategies to reduce computerized alerts in an electronic prescribing system using a Delphi approach. Stud Health Technol Inform 2013; 192: 8-12
- 18 Abarca J, Malone DC, Armstrong EP. , et al. Concordance of severity ratings provided in four drug interaction compendia. J Am Pharm Assoc (2003) 2004; 44 (02) 136-141
- 19 van Leeuwen RW, Jansman FG, van den Bemt PM. , et al. Drug-drug interactions in patients treated for cancer: a prospective study on clinical interventions. Ann Oncol 2015; 26 (05) 992-997
- 20 Scheife RT, Hines LE, Boyce RD. , et al. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf 2015; 38 (02) 197-206
- 21 Phansalkar S, van der Sijs H, Tucker AD. , et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
- 22 Strasberg HR, Chan A, Sklar SJ. Inter-Rater Agreement among Physicians on the Clinical Significance of Drug-Drug Interactions. AMIA Annual Symposium Proceedings. 2013 11/16; 2013 :1325–1328
- 23 Smithburger PL, Kane-Gill SL, Benedict NJ, Falcione BA, Seybert AL. Grading the severity of drug-drug interactions in the intensive care unit: a comparison between clinician assessment and proprietary database severity rankings. Ann Pharmacother 2010; 44 (11) 1718-1724
- 24 Seidling HM, Klein U, Schaier M. , et al. What, if all alerts were specific - estimating the potential impact on drug interaction alert burden. Int J Med Inform 2014; 83 (04) 285-291
- 25 MedChart. MedChart Electronic Medication Management: DXC.technology; 2017 . Available at: http://www.dxc.technology/providers/offerings/139499/140202-medchart_electronic_medication_management . Accessed October 31, 2018
- 26 MIMS. Decision Support Overview. Available at: http://www.mims.com.au/index.php/products/emims2017 . Accessed June 5, 2017
- 27 Vonbach P, Dubied A, Krähenbühl S, Beer JH. Evaluation of frequently used drug interaction screening programs. Pharm World Sci 2008; 30 (04) 367-374
- 28 Muhič N, Mrhar A, Brvar M. Comparative analysis of three drug-drug interaction screening systems against probable clinically relevant drug-drug interactions: a prospective cohort study. Eur J Clin Pharmacol 2017; 73 (07) 875-882
- 29 Reis AMM, Cassiani SHDB. Evaluation of three brands of drug interaction software for use in intensive care units. Pharm World Sci 2010; 32 (06) 822-828
- 30 Hayes AF, Krippendorff K. Answering the call for a standard reliability measure for coding data. Commun Methods Meas 2007; 1 (01) 77-89
- 31 Conde-Estévez D, Echeverría-Esnal D, Tusquets I, Albanell J. Potential clinical relevant drug-drug interactions: comparison between different compendia, do we have a validated method?. Ann Oncol 2015; 26 (06) 1272-1272
- 32 Saverno KR, Hines LE, Warholak TL. , et al. Ability of pharmacy clinical decision-support software to alert users about clinically important drug-drug interactions. J Am Med Inform Assoc 2011; 18 (01) 32-37