Comparison of Overridden Medication-related Clinical Decision Support in the Intensive Care Unit between a Commercial System and a Legacy System

Summary

Background: Electronic health records (EHRs) with clinical decision support (CDS) have shown to be effective at improving patient safety. Despite this, alerts delivered as part of CDS are overridden frequently, which is of concern in the critical care population as this group may have an increased risk of harm. Our organization recently transitioned from an internally-developed EHR to a commercial system. Data comparing various EHR systems, especially after transitions between EHRs, are needed to identify areas for improvement.

Objectives: To compare the two systems and identify areas for potential improvement with the new commercial system at a single institution.

Methods: Overridden medication-related CDS alerts were included from October to December of the systems’ respective years (legacy, 2011; commercial, 2015), restricted to three intensive care units. The two systems were compared with regards to CDS presentation and override rates for four types of CDS: drug-allergy, drug-drug interaction (DDI), geriatric and renal alerts. A post hoc analysis to evaluate for adverse drug events (ADEs) potentially resulting from overridden alerts was performed for ‘contraindicated’ DDIs via chart review.

Results: There was a significant increase in provider exposure to alerts and alert overrides in the commercial system (commercial: n=5,535; legacy: n=1,030). Rates of overrides were higher for the allergy and DDI alerts (p<0.001) in the commercial system. Geriatric and renal alerts were significantly different in incidence and presentation between the two systems. No ADEs were identified in an analysis of 43 overridden contraindicated DDI alerts.

Conclusions: The vendor system had much higher rates of both alerts and overrides, although we did not find evidence of harm in a review of DDIs which were overridden. We propose recommendations for improving our current system which may be helpful to other similar institutions; improving both alert presentation and the underlying knowledge base appear important.

Citation: Wong A, Wright A, Seger DL, Amato MG, Fiskio JM, Bates D. Comparison of Overridden Medication-related Clinical Decision Support in the Intensive Care Unit between a Commercial System and a Legacy System. Appl Clin Inform 2017; 8: 866–879 https://doi.org/10.4338/ACI-2017-04-RA-0059

Keywords

Adverse drug event - Clinical decision support - Critical care - Informatics - Patient safety

Clinical Relevance Statement

In this comparison of a legacy EHR and commercial EHR system in regards to CDS, we found significant differences in alert characteristics and frequency. We discuss recommendations to improve both systems, which may be extrapolated to other institutions. Recommendations include the timing of presentation of alerts to time of order instead of signing, increasing specificity by including patient-specific factors, and suggested doses based on a patient’s presentation.

Human Subjects Protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the Partners Institutional Review Board.

• References

• 1 Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, 1. Seger DL, Lee R, Mekala A, Song J, Komaroff AL, Bates DW. Guided medication dosing for inpatients with renal insufficiency. JAMA 2001; 286: 2839-44.
• 2 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163: 1409-16.
• 3 Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs, and issues. Ann Intern Med 2003; 139: 31-9.
• 4 Lin CP, Payne TH, Nichol P, Hoey PJ, Anderson CL, Gennari JH. Evaluating clinical decision support systems: monitoring CPOE order check override rates in the Department of Veterans Affairs’ Computerized Patient Record System. J Am Med Inform Assoc 2008; 15: 620-6.
• 5 Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med 1997; 25: 1289-97.
• 6 Bertsche T, Pfaff J, Schiller P, Kaltschmidt J, Pruszydlo MG, Stremmel W, Walter-Sack I, Haefeli WE, Encke J. Prevention of adverse drug reactions in intensive care patients by personal intervention based on an electronic clinical decision support system. Intensive Care Med 2010; 36: 665-72.
• 7 Phansalkar S, Zachariah M, Seidling HM, Mendes C, Volk L, Bates DW. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc 2014; 21: e332-40.
• 8 Phansalkar S, Wright A, Kuperman GJ, Vaida AJ, Bobb AM, Jenders RA, Payne TH, Halamka J, Bloom-rosen M, Bates DW. Towards meaningful medication-related clinical decision support: recommendations for an initial implementation. Appl Clin Inf 2011; 2: 50-62.
• 9 Shah NR, Seger AC, Seger DL. et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006; 13: 5-11.
• 10 Peterson JF, Kuperman GJ, Shek C, Patel M, Avorn J, Bates DW. Guided prescription of psychotropic medications for geriatric inpatients. Arch Intern Med 2005; 165: 802-7.
• 11 Sittig DF, Wright A, Meltzer S, Simonaitis L, Evans RS, Nichol WP, Ash JS, Middleton B. Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records. BMC Med Inform Decis Mak 2011; 11: 13.
• 12 Teich JM, Glaser JP, Beckley RF, Aranow M, Bates DW, Kuperman GJ, Ward ME, Spurr CD. The Brigham Integrated Computing System (BICS): advanced clinical systems in an academic hospital environment. Int J Med Inform 1999; 54: 197-208.
• 13 Wright A, Sittig DF, Ash JS, Feblowitz J, Meltzer S, McMullen C, Guappone K, Carpenter J, Richardson J, Simonaitis L, Evans RS, Nichol WP, Middleton B. Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems. J Am Med Inform Assoc 2011; 18: 232-42.
• 14 Wong A, Amato MG, Seger DL, Slight SP, Beeler PE, Dykes PC, Fiskio JM, Silvers ER, Orav EJ, Eguale T, Bates DW. Evaluation of medication-related clinical decision support overrides in the intensive care unit. J Crit Care 2017; 39: 156-61.
• 15 Nanji KC, Slight SP, Seger DL, Cho I, Fiskio JM, Redden LM, Volk LA, Bates DW. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21: 487-91.
• 16 McEvoy DS, Sittig DF, Hickman T, Aaron S, Ai A, Amato M, Bauer DW, Fraser GM, Harper J, Kennemer A, Krall MA, Lehmann CU, Malhotra S, Murphy DR, O’Kelley B, Samal L, Schreiber R, Singh H, Thomas EJ, Vartian CV, Westmorland J, McCoy AB, Wright A. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc 2017; 24: 331-8.
• 17 Kleinsasser A, Kuenszberg E, Loeckinger A, Keller C, Hoermann C, Kindner KH, Puehringer F. Sevoflurane, but not propofol, significantly prolongs the Q-T interval. Anesth Analg 2000; 90: 25-7.
• 18 Scalese MJ, Herring HR, Rathbun RC, Skrepnek GH, Ripley TL. Propofol-associated QTc prolongation. Ther Adv Drug Saf 2016; 7: 68-78.
• 19 Armahizer MJ, Seybert AL, Smithburger PL, Kane-Gill SL. Drug-drug interactions contributing to QT prolongation in cardiac intensive care units. J Crit Care 2013; 28: 243-9.
• 20 Ng TM, Olsen KM, McCartan MA, Puumala SE, Speidel KM, Miller MA, Sears TD. Drug-induced QTc-interval prolongation in the intensive care unit: incidence and predictors. J Pharm Pract 2010; 23: 19-24.
• 21 Tisdale JE, Jaynes HA, Kingery JR, Overholser BR, Mourad NA, Trujillo TN, Kovacs RJ. Effectiveness of a clinical decision support system for reducing the risk of QT interval prolongation in hospitalized patients. Circ Cardiovasc Qual Outcomes 2014; 7: 381-90.
• 22 Schreiber R, Gregoire JA, Shaha JE, Shaha SH. Think time: a novel approach to analysis of clinicians’’ behavior after reduction of drug-drug interaction alerts. Int J Med Inform 2017; 97: 59-67.
• 23 Payne TH, Hines LE, Chan RC, Hartman S, Kapusnik-Uner J, Russ AL, Chaffee BW, Hartman C, Tamis V, Galbreth B, Glassman PA, Phansalkar S, van der Sijs H, Gephart SM, Mann G, Strasberg HR, Grizzle AJ, Brown M, Kuperman GJ, Steiner C, Sullins A, Ryan H, Wittle MA, Malone DC. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22: 1243-50.
• 24 Adityanjee, Aderibigbe YA, Matthews T. Epidemiology of neuroleptic malignant syndrome. Clin Neuro -pharmacol 1999; 22: 151-8.
• 25 Graham DJ, Staffa JA, Shatin D, Andrade SE, Schech SD, La Grenade L. Incidence of hospitalized rhabdomyolysis in patients treated with lipid-lowering drugs. JAMA 2004; 292: 2585-90.
• 26 Busca C, Farcas A, Leucuta D, Mogosan C, Bojita M, Dumitrascu DL. Drug-drug interactions of statins potentially leading to muscle-related side effects in hospitalized patients. Rom J Intern Med 2015; 53: 329-35.
• 27 Chatzizisis YS, Koskinas KC, Misirli G, Vaklavas C, Hatzitolios A, Giannoglou GD. Risk factors and drug interactions predisposing to statin-induced myopathy: implications for risk assessment, prevention and treatment. Drug Saf 2010; 33: 171-87.
• 28 Neofytos D, Lombardi LR, Shields RK, Ostrander D, Warren L, Nguyen MH, Thompson CB, Marr KA. Administration of voriconazole in patients with renal dysfunction. Clin Infect Dis 2012; 54: 913-21.
• 29 Lilly CM, Welch VL, Mayer T, Ranauro P, Mesiner J, Luke DR. Evaluation of intravenous voriconazole in patients with compromised renal function. BMC Infect Dis 2013; 13: 14.
• 30 Baysari MT, Reckmann MH, Li L, Day RO, Westbrook JI. Failure to utilize functions of an electronic prescribing system and the subsequent generation of ‘technically preventable’ computerized alerts. J Am Med Inform Assoc 2012; 19: 1003-10.
• 31 Russ AL, Weiner M, Saleem JJ, Wears RL. When ‘technically preventable’ alerts occur, the design –not the prescriber –has failed. J Am Med Inform Assoc 2012; 19: 1119.
• 32 American Geriatrics Society 2015 Beers Criteria Update Expert Panel.. American Geriatrics Society 2015 updated Beers Criteria for potentially inappropriate medication use in older adults. J Am Geriatr Soc 2015; 63: 2227-46.
• 33 Office of the National Coordinator for Health Information Technology. Hospital EHR Vendors. Available at: https://dashboard.healthit.gov/quickstats/pages/FIG-Vendors-of-EHRs-to-Participating-Hospitals.php Updated September 2016. Accessed May 1, 2017.
• 34 Cho I, Slight SP, Nanji KC, Seger DL, Maniam N, Fiskio JM, Dykes PC, Bates DW. The effect of provider characteristics on the responses to medication-related decision support alerts. Int J Med Inform 2015; 84: 630-9.