Appl Clin Inform 2022; 13(02): 419-430
DOI: 10.1055/s-0042-1745830
Case Report

Impact of a Vendor-Developed Opioid Clinical Decision Support Intervention on Adherence to Prescribing Guidelines, Opioid Prescribing, and Rates of Opioid-Related Encounters

1   Department of Family and Community Medicine, University of Missouri School of Medicine, Columbia, Missouri, United States
Bernie Eskridge
2   Department of Child Health, University of Missouri School of Medicine, Columbia, Missouri, United States
Brandi Ross
3   Tiger Institute, Cerner Corporation, Columbia, Missouri, United States
Matthew Wright
4   University of Missouri Health Care, Columbia, Missouri, United States
Thomas Selva
2   Department of Child Health, University of Missouri School of Medicine, Columbia, Missouri, United States
› Author Affiliations
Funding This study was funded directly by the University of Missouri Health Care.


Background Provider prescribing practices contribute to an excess of opioid-related deaths in the United States. Clinical guidelines exist to assist providers with improving prescribing practices and promoting patient safety. Clinical decision support systems (CDSS) may promote adherence to these guidelines and improve prescribing practices. The aim of this project was to improve opioid guideline adherence, prescribing practices, and rates of opioid-related encounters through the implementation of an opioid CDSS.

Methods A vendor-developed, provider-targeted CDSS package was implemented in a multi-location academic health center. An interrupted time-series analysis was performed, evaluating 30 weeks pre- and post-implementation time periods. Outcomes were derived from vendor-supplied key performance indicators and directly from the electronic health record (EHR) database. Opioid-prescribing outcomes included count of opioid prescriptions, morphine milligram equivalents per prescription, counts of opioids with concurrent benzodiazepines, and counts of short-acting opioids in opioid-naïve patients. Encounter outcomes included rates of encounters for opioid abuse and dependence and rates of encounters for opioid poisoning and overdose. Guideline adherence outcomes included rates of provision of naloxone and documentation of opioid treatment agreements.

Results The opioid CDSS generated an average of 1,637 alerts per week. Rates of provision of naloxone and opioid treatment agreements improved after CDSS implementation. Vendor-supplied prescribing outcomes were consistent with prescribing outcomes derived directly from the EHR, but all prescribing and encounter outcomes were unchanged.

Conclusion A vendor-developed, provider-targeted opioid CDSS did not improve opioid-prescribing practices or rates of opioid-related encounters. The CDSS improved some measures of provider adherence to opioid-prescribing guidelines. Further work is needed to determine the optimal configuration of opioid CDSS so that opioid-prescribing patterns are appropriately modified and encounter outcomes are improved.

Protection of Human and Animal Subjects

The University of Missouri institutional review board (IRB) determined that the project was quality improvement activity, not human subject research, and did not require additional IRB review. The requirement for consent was waived.

Supplementary Material

Publication History

Received: 20 September 2021

Accepted: 18 February 2022

Article published online:
20 April 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 National Institute on Drug Abuse.. Overdose death rates. Accessed April 6, 2020 at:
  • 2 National Institute on Drug Abuse Missouri opioid summary. Accessed April 8, 2020 at:
  • 3 Wilson N, Kariisa M, Seth P, Smith IV H, Davis NL. Drug and opioid-involved overdose deaths—United States, 2017-2018. MMWR Morb Mortal Wkly Rep 2020; 69 (11) 290-297
  • 4 Bohnert ASB, Valenstein M, Bair MJ. et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA 2011; 305 (13) 1315-1321
  • 5 Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. MMWR Recomm Rep 2016; 65 (01) 1-49
  • 6 Nuckols TK, Anderson L, Popescu I. et al. Opioid prescribing: a systematic review and critical appraisal of guidelines for chronic pain. Ann Intern Med 2014; 160 (01) 38-47
  • 7 Centers for Disease Control and Prevention.. Quality Improvement and Care Coordination: Implementing the CDC Guideline for Prescribing Opioids for Chronic Pain. Atlanta, GA: Division of Unintentinal Injury Prevention, National Center for Injury Prevention and Control; 2018
  • 8 Starrels JL, Becker WC, Weiner MG, Li X, Heo M, Turner BJ. Low use of opioid risk reduction strategies in primary care even for high risk patients with chronic pain. J Gen Intern Med 2011; 26 (09) 958-964
  • 9 Tournebize J, Gibaja V, Muszczak A, Kahn JP. Are physicians safely prescribing opioids for chronic noncancer pain? A systematic review of current evidence. Pain Pract 2016; 16 (03) 370-383
  • 10 Losby JL, Hyatt JD, Kanter MH, Baldwin G, Matsuoka D. Safer and more appropriate opioid prescribing: a large healthcare system's comprehensive approach. J Eval Clin Pract 2017; 23 (06) 1173-1179
  • 11 Huang KTL, Blazey-Martin D, Chandler D, Wurcel A, Gillis J, Tishler J. A multicomponent intervention to improve adherence to opioid prescribing and monitoring guidelines in primary care. J Opioid Manag 2019; 15 (06) 445-453
  • 12 Dorflinger L, Moore B, Goulet J. et al. A partnered approach to opioid management, guideline concordant care and the stepped care model of pain management. J Gen Intern Med 2014; 29 (Suppl. 04) 870-876
  • 13 Nielsen J. 10 Usability Heuristics for User Interface Design. Nielsen Norman Group.. Accessed May 17, 2020 at:
  • 14 Boeger MR. 7-Day Opiate Supply for Initial Prescriptions for Acute Pain. Missouri Bureau of Narcotics and Dangerous Drugs. Aug 29, 2018. Available at:
  • 15 Opioid Oral Morphine Milligram Equivalent (MME) Conversion Factors. Centers for Medicare & Medicaid Services. 2020. HHS-0938-2020-F-5924. Available at:
  • 16 Value Set Authority Center.. NLM Value Set Repository. Accessed April 15, 2020 at:
  • 17 Linden A. Conducting interrupted time-series analysis for single- and multiple-group comparisons. Stata J 2015; 15 (02) 480-500
  • 18 Lopez Bernal J, Cummins S, Gasparrini A. The use of controls in interrupted time series studies of public health interventions. Int J Epidemiol 2018; 47 (06) 2082-2093
  • 19 Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2016; 25 (12) 986-992
  • 20 Smart R, Pardo B, Davis CS. Systematic review of the emerging literature on the effectiveness of naloxone access laws in the United States. Addiction 2021; 116 (01) 6-17
  • 21 Guy Jr GP, Haegerich TM, Evans ME, Losby JL, Young R, Jones CM. Vital signs: pharmacy-based naloxone dispensing—United States, 2012-2018. MMWR Morb Mortal Wkly Rep 2019; 68 (31) 679-686
  • 22 Starrels JL, Becker WC, Alford DP, Kapoor A, Williams AR, Turner BJ. Systematic review: treatment agreements and urine drug testing to reduce opioid misuse in patients with chronic pain. Ann Intern Med 2010; 152 (11) 712-720
  • 23 Chapman L, dela Cruz AM, Hutto J. Opioid treatment agreements: helpful or hurtful?. Am J Psychiatry Resid J 2016; 11 (09) 14-16
  • 24 McGlothlin AE, Lewis RJ. Minimal clinically important difference: defining what really matters to patients. JAMA 2014; 312 (13) 1342-1343
  • 25 Hawkins AT, Samuels LR. Use of confidence intervals in interpreting nonstatistically significant results. JAMA 2021; 326 (20) 2068-2069
  • 26 Osheroff JA. ed. Improving Medication Use and Outcome with Clinical Decision Support: a Step-by-step Guide. Healthcare Information and Management Systems Society; 2009
  • 27 Ratwani RM, Savage E, Will A. et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25 (09) 1197-1201
  • 28 Ratwani R, Fairbanks T, Savage E. et al. Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform 2016; 7 (04) 1069-1087
  • 29 Hettinger AZ, Melnick ER, Ratwani RM. Advancing electronic health record vendor usability maturity: progress and next steps. J Am Med Inform Assoc 2021; 28 (05) 1029-1031
  • 30 Monica K. Cerner Launching EHR-Integrated Opioid Toolkit for Safe Prescribing. Xtelligent Healthcare Media, LLC; 2021 Accessed August 1, 2021 at:
  • 31 Ross C. Epic's AI algorithms, shielded from scrutiny by a corporate firewall, are delivering inaccurate information on seriously ill patients. STAT.. 2021 Accessed August 1, 2021 at:
  • 32 Wong A, Otles E, Donnelly JP. et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med 2021; 181 (08) 1065-1070
  • 33 Coiera E, Tong HL. Replication studies in the clinical decision support literature-frequency, fidelity, and impact. J Am Med Inform Assoc 2021; 28 (09) 1815-1825
  • 34 Smith LB, Golberstein E, Anderson K. et al. The association of EHR drug safety alerts and co-prescribing of opioids and benzodiazepines. J Gen Intern Med 2019; 34 (08) 1403-1405
  • 35 Price-Haywood EG, Burton J, Burstain T. et al. Clinical effectiveness of decision support for prescribing opioids for chronic noncancer pain: a prospective cohort study. Value Health 2020; 23 (02) 157-163
  • 36 Anderson DR, Zlateva I, Coman EN, Khatri K, Tian T, Kerns RD. Improving pain care through implementation of the stepped care model at a multisite community health center. J Pain Res 2016; 9: 1021-1029
  • 37 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
  • 38 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
  • 39 Committee on Patient Safety and Health Information Technology; Institute of Medicine.. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press; 2011
  • 40 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
  • 41 Genco EK, Forster JE, Flaten H. et al. Clinically inconsequential alerts: the characteristics of opioid drug alerts and their utility in preventing adverse drug events in the emergency department. Ann Emerg Med 2016; 67 (02) 240-248