Appl Clin Inform 2021; 12(05): 984-995
DOI: 10.1055/s-0041-1736339
Research Article

Development of a Perioperative Medication-Related Clinical Decision Support Tool to Prevent Medication Errors: An Analysis of User Feedback

Karen C. Nanji
1   Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States
2   Department of Anaesthesiology, Harvard Medical School, Boston, Massachusetts, United States
3   Mass General Brigham, Inc., Boston, Massachusetts, United States
Pamela M. Garabedian
3   Mass General Brigham, Inc., Boston, Massachusetts, United States
Sofia D. Shaikh
1   Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States
Marin E. Langlieb
1   Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States
Aziz Boxwala
4   Elimu Informatics, Inc., La Jolla, California, United States
William J. Gordon
3   Mass General Brigham, Inc., Boston, Massachusetts, United States
5   Division of General Internal Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
6   Department of Medicine, Harvard Medical School, Boston, Massachusetts, United States
David W. Bates
3   Mass General Brigham, Inc., Boston, Massachusetts, United States
5   Division of General Internal Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
6   Department of Medicine, Harvard Medical School, Boston, Massachusetts, United States
› Author Affiliations


Objectives Medication use in the perioperative setting presents many patient safety challenges that may be improved with electronic clinical decision support (CDS). The objective of this paper is to describe the development and analysis of user feedback for a robust, real-time medication-related CDS application designed to provide patient-specific dosing information and alerts to warn of medication errors in the operating room (OR).

Methods We designed a novel perioperative medication-related CDS application in four phases: (1) identification of need, (2) alert algorithm development, (3) system design, and (4) user interface design. We conducted group and individual design feedback sessions with front-line clinician leaders and subject matter experts to gather feedback about user requirements for alert content and system usability. Participants were clinicians who provide anesthesia (attending anesthesiologists, nurse anesthetists, and house staff), OR pharmacists, and nurses.

Results We performed two group and eight individual design feedback sessions, with a total of 35 participants. We identified 20 feedback themes, corresponding to 19 system changes. Key requirements for user acceptance were: Use hard stops only when necessary; provide as much information as feasible about the rationale behind alerts and patient/clinical context; and allow users to edit fields such as units, time, and baseline values (e.g., baseline blood pressure).

Conclusion We incorporated user-centered design principles to build a perioperative medication-related CDS application that uses real-time patient data to provide patient-specific dosing information and alerts. Emphasis on early user involvement to elicit user requirements, workflow considerations, and preferences during application development can result in time and money efficiencies and a safer and more usable system.

Protection of Human and Animal Subjects

This study was approved by our Institutional Review Board and the requirement for written consent was waived.

Publication History

Received: 15 June 2021

Accepted: 01 September 2021

Article published online:
24 November 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Nanji KC, Patel A, Shaikh S, Seger DL, Bates DW. Evaluation of perioperative medication errors and adverse drug events. Anesthesiology 2016; 124 (01) 25-34
  • 2 Merry AF, Webster CS, Hannam J. et al. Multimodal system designed to reduce errors in recording and administration of drugs in anaesthesia: prospective randomised clinical evaluation. BMJ 2011; 343: d5543
  • 3 Hodkinson A, Tyler N, Ashcroft DM. et al. Preventable medication harm across health care settings: a systematic review and meta-analysis. BMC Med 2020; 18 (01) 313
  • 4 Barker KN, Flynn EA, Pepper GA, Bates DW, Mikeal RL. Medication errors observed in 36 health care facilities. Arch Intern Med 2002; 162 (16) 1897-1903
  • 5 Kaushal R, Bates DW, Landrigan C. et al. Medication errors and adverse drug events in pediatric inpatients. JAMA 2001; 285 (16) 2114-2120
  • 6 Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA 1998; 279 (15) 1200-1205
  • 7 Magrabi F, Li SY, Day RO, Coiera E. Errors and electronic prescribing: a controlled laboratory study to examine task complexity and interruption effects. J Am Med Inform Assoc 2010; 17 (05) 575-583
  • 8 Devine EB, Hansen RN, Wilson-Norton JL. et al. The impact of computerized provider order entry on medication errors in a multispecialty group practice. J Am Med Inform Assoc 2010; 17 (01) 78-84
  • 9 Gandhi TK, Weingart SN, Seger AC. et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med 2005; 20 (09) 837-841
  • 10 Nanji KC, Rothschild JM, Salzberg C. et al. Errors associated with outpatient computerized prescribing systems. J Am Med Inform Assoc 2011; 18 (06) 767-773
  • 11 Webster CS, Larsson L, Frampton CM. et al. Clinical assessment of a new anaesthetic drug administration system: a prospective, controlled, longitudinal incident monitoring study. Anaesthesia 2010; 65 (05) 490-499
  • 12 Gandhi TK, Weingart SN, Borus J. et al. Adverse drug events in ambulatory care. N Engl J Med 2003; 348 (16) 1556-1564
  • 13 Pharmacists ASoHS. Standardize 4 Safety. Accessed March 12, 2021 at:
  • 14 Eichhorn J. APSF hosts medication safety conference: consensus group defines challenges and opportunities for improved practice. APSF Newsletter 2010; 25: 1-8
  • 15 Leotsakos A, Zheng H, Croteau R. et al. Standardization in patient safety: the WHO High 5s project. Int J Qual Health Care 2014; 26 (02) 109-116
  • 16 The International Organization for Standardization (ISO). Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems. Accessed January 16, 2020 at:
  • 17 User-Centered Design Basics. Accessed October 25, 2020 at:
  • 18 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 22 (06) 1179-1182
  • 19 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
  • 20 Bates DW, Leape LL, Cullen DJ. et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15) 1311-1316
  • 21 Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients–a systematic review. Int J Med Inform 2008; 77 (06) 365-376
  • 22 Wright A, Feblowitz J, Phansalkar S. et al. Preventability of adverse drug events involving multiple drugs using publicly available clinical decision support tools. Am J Health Syst Pharm 2012; 69 (03) 221-227
  • 23 Mishuris RG, Linder JA, Bates DW, Bitton A. Using electronic health record clinical decision support is associated with improved quality of care. Am J Manag Care 2014; 20 (10) e445-e452
  • 24 Griffey RT, Lo HG, Burdick E, Keohane C, Bates DW. Guided medication dosing for elderly emergency patients using real-time, computerized decision support. J Am Med Inform Assoc 2012; 19 (01) 86-93
  • 25 van Rosse F, Maat B, Rademaker CM, van Vught AJ, Egberts AC, Bollen CW. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009; 123 (04) 1184-1190
  • 26 Marasinghe KM. Computerised clinical decision support systems to improve medication safety in long-term care homes: a systematic review. BMJ Open 2015; 5 (05) e006539
  • 27 Simpao AF, Tan JM, Lingappan AM, Gálvez JA, Morgan SE, Krall MA. A systematic review of near real-time and point-of-care clinical decision support in anesthesia information management systems. J Clin Monit Comput 2017; 31 (05) 885-894
  • 28 Freundlich RE, Ehrenfeld JM. Perioperative information systems: opportunities to improve delivery of care and clinical outcomes in cardiac and vascular surgery. J Cardiothorac Vasc Anesth 2018; 32 (03) 1458-1463
  • 29 Freundlich RE, Ehrenfeld JM. Anesthesia information management: clinical decision support. Curr Opin Anaesthesiol 2017; 30 (06) 705-709
  • 30 Nair BG, Gabel E, Hofer I, Schwid HA, Cannesson M. Intraoperative clinical decision support for anesthesia: a narrative review of available systems. Anesth Analg 2017; 124 (02) 603-617
  • 31 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 13 (05) 547-556
  • 32 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
  • 33 Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45 (06) 1202-1216
  • 34 Miller K, Mosby D, Capan M. et al. Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support. J Am Med Inform Assoc 2018; 25 (05) 585-592
  • 35 Marcilly R, Ammenwerth E, Roehrer E, Niès J, Beuscart-Zéphir MC. Evidence-based usability design principles for medication alerting systems. BMC Med Inform Decis Mak 2018; 18 (01) 69
  • 36 Fitch K, Bernstein SJ, Aguilar MD. et al. The RAND/UCLA appropriateness method user's manual. RAND Corporation. Accessed September 17, 2021 at:
  • 37 Mangione-Smith R, DeCristofaro AH, Setodji CM. et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med 2007; 357 (15) 1515-1523
  • 38 Hinkelbein J, Lamperti M, Akeson J. et al. European Society of Anaesthesiology and European Board of Anaesthesiology guidelines for procedural sedation and analgesia in adults. Eur J Anaesthesiol 2018; 35 (01) 6-24
  • 39 SooHoo NF, Lieberman JR, Farng E, Park S, Jain S, Ko CY. Development of quality of care indicators for patients undergoing total hip or total knee replacement. BMJ Qual Saf 2011; 20 (02) 153-157
  • 40 Pagel C, Brown KL, McLeod I. et al. Selection by a panel of clinicians and family representatives of important early morbidities associated with paediatric cardiac surgery suitable for routine monitoring using the nominal group technique and a robust voting process. BMJ Open 2017; 7 (05) e014743
  • 41 Utley M, Gallivan S, Mills M, Mason M, Hargraves C. A consensus process for identifying a prioritised list of study questions. Health Care Manage Sci 2007; 10 (01) 105-110
  • 42 Nanji KC, Rothschild JM, Boehne JJ, Keohane CA, Ash JS, Poon EG. Unrealized potential and residual consequences of electronic prescribing on pharmacy workflow in the outpatient pharmacy. J Am Med Inform Assoc 2014; 21 (03) 481-486
  • 43 Clarke V, Braun V. Successful Qualitative Research: A practical guide for beginners. Thousand Oaks, CA: Sage Publications Inc.; 2013
  • 44 O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014; 89 (09) 1245-1251
  • 45 International HLS. Fast Healthcare Interoperability Resources. Accessed August 10, 2020 at:
  • 46 RedHat. Overview. Accessed January 21, 2020 at:
  • 47 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
  • 48 Payne TH, Hines LE, Chan RC. et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22 (06) 1243-1250
  • 49 Schild S, Sedlmayr B, Schumacher AK, Sedlmayr M, Prokosch HU, St Pierre M. German Cognitive Aid Working Group. A digital cognitive aid for anesthesia to support intraoperative crisis management: results of the user-centered design process. JMIR Mhealth Uhealth 2019; 7 (04) e13226
  • 50 Nielsen J. Ten usability heuristics. Accessed September 17, 2021 at:
  • 51 Wickens CD, Lee J, Liu YD, Gordon-Becker S. Designing for People. An Introduction to Human Factors Engineering. 2nd ed.. New York, NY: Pearson; 2003
  • 52 Merrill D. Book review: Usability Inspection Methods edited by Jakob Nielsen and Robert L. Mack (John Wiley & Sons, Inc., 1994). ACM SIGART Bulletin 1995; 6 (03) 19-20
  • 53 Wanderer JP, Ehrenfeld JM. Clinical decision support for perioperative information management systems. Semin Cardiothorac Vasc Anesth 2013; 17 (04) 288-293
  • 54 Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth Analg 2015; 121 (03) 678-692
  • 55 Zachariah M, Phansalkar S, Seidling HM. et al. Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems–I-MeDeSA. J Am Med Inform Assoc 2011; 18 (Suppl. 01) i62-i72
  • 56 Mazur LM, Mosaly PR, Moore C, Marks L. Association of the usability of electronic health records with cognitive workload and performance levels among physicians. JAMA Netw Open 2019; 2 (04) e191709
  • 57 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
  • 58 Mayhew D. The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design. Burlington MA: Morgan Kaufmann Publishers Inc.; 1999
  • 59 National Institute of Standards and Technology.. Schumacher RM, Lowry SZ. NIST guide to the processes approach for improving the usability of electronic health records. Accessed September 17, 2021 at:
  • 60 Middleton B, Bloomrosen M, Dente MA. et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8
  • 61 Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005; 38 (01) 75-87
  • 62 Katsulis Z, Ergai A, Leung WY. et al. Iterative user centered design for development of a patient-centered fall prevention toolkit. Appl Ergon 2016; 56: 117-126
  • 63 Phansalkar S, Edworthy J, Hellier E. et al. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010; 17 (05) 493-501