Reduced Verification of Medication Alerts Increases Prescribing ErrorsFunding This research was supported by a doctoral Scholarship for David Lyell provided by the HCF (Hospitals Contribution Fund of Australia Limited) Research Foundation.
04 September 2018
29 November 2018
30 January 2019 (online)
Objective Clinicians using clinical decision support (CDS) to prescribe medications have an obligation to ensure that prescriptions are safe. One option is to verify the safety of prescriptions if there is uncertainty, for example, by using drug references. Supervisory control experiments in aviation and process control have associated errors, with reduced verification arising from overreliance on decision support. However, it is unknown whether this relationship extends to clinical decision-making. Therefore, we examine whether there is a relationship between verification behaviors and prescribing errors, with and without CDS medication alerts, and whether task complexity mediates this.
Methods A total of 120 students in the final 2 years of a medical degree prescribed medicines for patient scenarios using a simulated electronic prescribing system. CDS (correct, incorrect, and no CDS) and task complexity (low and high) were varied. Outcomes were omission (missed prescribing errors) and commission errors (accepted false-positive alerts). Verification measures were access of drug references and view time percentage of task time.
Results Failure to access references for medicines with prescribing errors increased omission errors with no CDS (high-complexity: χ 2(1) = 12.716; p < 0.001) and incorrect CDS (Fisher's exact; low-complexity: p = 0.002; high-complexity: p = 0.001). Failure to access references for false-positive alerts increased commission errors (low-complexity: χ 2(1) = 16.673, p < 0.001; high-complexity: χ 2(1) = 18.690, p < 0.001). Fewer participants accessed relevant references with incorrect CDS compared with no CDS (McNemar; low-complexity: p < 0.001; high-complexity: p < 0.001). Lower view time percentages increased omission (F(3, 361.914) = 4.498; p = 0.035) and commission errors (F(1, 346.223) = 2.712; p = 0.045). View time percentages were lower in CDS-assisted conditions compared with unassisted conditions (F(2, 335.743) = 10.443; p < 0.001).
Discussion The presence of CDS reduced verification of prescription safety. When CDS was incorrect, reduced verification was associated with increased prescribing errors.
Conclusion CDS can be incorrect, and verification provides one mechanism to detect errors. System designers need to facilitate verification without increasing workload or eliminating the benefits of correct CDS.
Keywordsautomation bias - clinical decision support systems - medication alerts - cognitive load - medication errors - human–computer interaction
David Lyell conceived this research and designed and conducted the study with guidance from and under the supervision of Enrico Coiera and Farah Magrabi. David Lyell drafted the manuscript with input from all authors. All authors provided revisions for intellectual content and have approved the final manuscript.
Protection of Human and Animal Subjects
The research was conducted in accordance with protocols approved by the Macquarie University Human Research Ethics Committee (5201401029) and the University of New South Wales Human Research Ethics Advisory Panel (2014–7-32).
- 1 Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007; 41 (09) 1411-1426
- 2 Tully MP, Ashcroft DM, Dornan T, Lewis PJ, Taylor D, Wass V. The causes of and factors associated with prescribing errors in hospital inpatients: a systematic review. Drug Saf 2009; 32 (10) 819-836
- 3 Wolfstadt JI, Gurwitz JH, Field TS. , et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008; 23 (04) 451-458
- 4 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; 15 (05) 585-600
- 5 van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts AC, Bollen CW. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009; 123 (04) 1184-1190
- 6 Sweidan M, Williamson M, Reeve JF. , et al. Evaluation of features to support safety and quality in general practice clinical software. BMC Med Inform Decis Mak 2011; 11 (01) 1-8
- 7 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
- 8 Wright A, Hickman TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
- 9 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
- 10 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
- 11 Nanji KC, Slight SP, Seger DL. , et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
- 12 Oxford English Dictionary. verification, n. Oxford University Press; 2018
- 13 Manzey D, Reichenbach J, Onnasch L. Human performance consequences of automated decision aids: the impact of degree of automation and system experience. J Cogn Eng Decis Mak 2012; 6 (01) 57-87
- 14 Bahner J, Huper AD, Manzey D. Misuse of automated decision aids: complacency, automation bias and the impact of training experience. Int J Hum Comput Stud 2008; 66 (09) 688-699
- 15 Bahner J, Elepfandt MF, Manzey D. Misuse of diagnostic aids in process control: the effects of automation misses on complacency and automation bias. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 22–26, 2008, New York, NY
- 16 Mosier KL, Skitka LJ. Human decision makers and automated decision aids: made for each other. In: Parasuraman R, Mouloua M. , eds. Automation and Human Performance: Theory and Applications. Hillsdale, NJ: Lawrence Erlbaum Associates; 1996: 201-220
- 17 Lyell D, Magrabi F, Raban MZ. , et al. Automation bias in electronic prescribing. BMC Med Inform Decis Mak 2017; 17 (01) 28
- 18 Mosier KL, Skitka LJ, Heers S, Burdick M. Automation bias: decision making and performance in high-tech cockpits. Int J Aviat Psychol 1997; 8 (01) 47-63
- 19 Bagheri N, Jamieson GA. The impact of context-related reliability on automation failure detection and scanning behaviour. Paper presented at the IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), October 10–13, 2004
- 20 Bagheri N, Jamieson GA. Considering subjective trust and monitoring behavior in assessing automation-induced “complacency.”. In: Vincenzi DA, Mouloua M, Hancock PA. , eds. Human Performance, Situation Awareness and Automation: Current Research and Trends. Vol. 2. Mahwah, NJ: Lawrence Erlbaum Associates; 2004: 54-59
- 21 Reichenbach J, Onnasch L, Manzey D. Misuse of automation: the impact of system experience on complacency and automation bias in interaction with automated aids. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 27 to October 1, 2010, San Francisco, CA
- 22 Reichenbach J, Onnasch L, Manzey D. Human performance consequences of automated decision aids in states of sleep loss. Hum Factors 2011; 53 (06) 717-728
- 23 Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 423-431
- 24 Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc 2012; 19 (01) 121-127
- 25 Bailey NR, Scerbo MW. Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust. Theor Issues Ergon Sci 2007; 8 (04) 321-348
- 26 Povyakalo AA, Alberdi E, Strigini L, Ayton P. How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Making 2013; 33 (01) 98-107
- 27 Sweller J. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educ Psychol Rev 2010; 22 (02) 123-138
- 28 Lyell D, Magrabi F, Coiera E. The effect of cognitive load and task complexity on automation bias in electronic prescribing. Hum Factors 2018; 60 (07) 1008-1021
- 29 De Vries TPGM, Henning RH, Hogerzeil HV, Fresle DA. Guide to Good Prescribing. Geneva: World Health Organization; 1994
- 30 Australian Medicines Handbook Pty Ltd. Australian Medicines Handbook 2015. Available at: http://amhonline.amh.net.au/
- 31 Day RO, Snowden L. Where to find information about drugs. Aust Prescr 2016; 39 (03) 88-95
- 32 Sweller J. Cognitive load theory, learning difficulty, and instructional design. Learn Instr 1994; 4 (04) 295-312
- 33 Sweller J, Chandler P. Why some material is difficult to learn. Cogn Instr 1994; 12 (03) 185-233
- 34 Hoffman L, Rovine MJ. Multilevel models for the experimental psychologist: foundations and illustrative examples. Behav Res Methods 2007; 39 (01) 101-117
- 35 Tabachnick BG. Using Multivariate Statistics. In: Tabachnick BG, Fidel LS. 6th ed. Boston, MA: Pearson; 2013
- 36 Peugh JL. A practical guide to multilevel modeling. J Sch Psychol 2010; 48 (01) 85-112
- 37 Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995; 310 (6973): 170
- 38 Twisk J. Applied Multilevel Analysis: A Practical Guide. Cambridge, UK: Cambridge University Press; 2006
- 39 Hayes AF. A primer on multilevel modeling. Hum Commun Res 2006; 32 (04) 385-410
- 40 Mack A, Rock I. Inattentional Blindness. Cambridge, MA: MIT Press; 1998
- 41 Fiske ST, Taylor SE. Social cognition. New York, NY: Random House; 1984
- 42 Mosier KL, Skitka LJ, Dunbar M, McDonnell L. Aircrews and automation bias: the advantages of teamwork?. Int J Aviat Psychol 2001; 11 (01) 1-14