Download PDF
Yearb Med Inform 2006; 15(01): 20-25
DOI: 10.1055/s-0038-1638469
Survey
Georg Thieme Verlag KG Stuttgart

Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems

E. Coiera
1   Centre for Health Informatics, University of New South Wales, Australia
,
J. I. Westbrook
1   Centre for Health Informatics, University of New South Wales, Australia
,
J. C. Wyatt
2   Health Informatics Centre, University of Dundee, Dundee, Scotland
› Author Affiliations
Further Information

Publication History

Publication Date:
07 March 2018 (online)

   Permissions and Reprints

Summary

Objectives

The use of clinical decision support systems (CDSS) can improve the overall safety and quality of health care delivery, but may also introduce machine-related errors. Recent concerns about the potential for CDSS to harm patients have generated much debate, but there is little research available to identify the nature of such errors, or quantify their frequency or clinical impact.

Methods

A review of recent literature into electronic prescribing systems, as well as related literature in decision support.

Results

There seems to be some evidence for variation in the outcomes of using CDSS, most likely reflecting variations in clinical setting, culture, training and organizational process, independent of technical variables. There is also preliminary evidence that poorly implemented CDSS can lead to increased mortality in some settings. Studies in the US, UK and Australia have found commercial prescribing systems often fail to uniformly detect significant drug interactions, probably because of errors in their knowledge base. Electronic medication management systems may generate new types of error because of user-interface design, but al so because of events in the workplace such as distraction affecting the actions of system users. Another potential source of CDSS influenced errors are automation biases, including errors of omission where individuals miss important data because the system does not prompt them to notice them, and errors of commission where individuals do what the decision aid tells to do, even when this contradicts their training and other available data. Errors of dismissal occur when relevant alerts are ignored. On-line decision support systems may also result in errors where clinicians come to an incorrect assessment of the evidence, possibly shaped in part by cognitive decision biases.

Conclusions

The effectiveness of decision support systems, like all other health IT, cannot be assessed purely by evaluating the usability and performance of the software, but is the outcome of a complex set of cognitive and socio-technical interactions. A deeper understanding of these issues can result in the design of systems which are not just intrinsically ‘safe’ but which also result in safe outcomes in the hands of busy or poorly resourced clinicians.

Keywords

Decision support systems - safety - error - automation bias - electronic prescribing
 

  • References

  • 1 Hunt D, Haynes B, Hanna S, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: A systematic review. J Am Med Assoc 1998; 280 (15) 1339-46.
    CrossrefPubMedGoogle Scholar
  • 2 Garg A, Adhikari N, McDonald H. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. A systematic review. J Am Med Assoc 2005; 293 (10) 1223-38.
    PubMedGoogle Scholar
  • 3 Rind D, Safran C, Phillips R, Wang Q, Calkins D, Delbanco T. et al. Effect of computer-based alerts on the treatment and outcomes of hospitalised patients. Arch Intern Med 1994; 154: 1511-7.
    CrossrefPubMedGoogle Scholar
  • 4 Evans R, Pestotnik S, Classen D, Clemmer T, Weaver L, Orme J. et al. A computer-assisted management program for antibiotics and other anitinfective agents. N Engl J Med 1998; 338 (04) 232-8.
    CrossrefPubMedGoogle Scholar
  • 5 Protti D. The Application of Computer Technology in GP Surgeries is Beginning To Have Positive Effects on Chronic Disease Management. World View Reports. London: National Health Service; 2005. http://www.connectingforhealth.nhs.uk/worldview/protti6
    Google Scholar
  • 6 Benson T. Why general practitioners use computers and hospital doctors do not: Part 1 incentives. Br Med J 2002; 325: 1086-9.
    CrossrefPubMedGoogle Scholar
  • 7 Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D. et al. Effect of computerised evidence based guidelines on management of asthma and angina in primary care: cluster randomised controlled trial. Br Med J 2002; 325: 941-6.
    CrossrefPubMedGoogle Scholar
  • 8 Peters T, Montgomery AA, Fahey T. How accurately do primary health care professionals use cardiovascular risk tables in the management of hypertension?. Br J Gen Pract 1999; 49: 987-8.
    PubMedGoogle Scholar
  • 9 Ash J, Gorman PN, Seshadri V, Hersh W. Computerized physician order entry in U.S. Hospitals: results of a 2002 survey. J Am Med Inform Assoc 2004; 11 (02) 95-9.
    PubMedGoogle Scholar
  • 10 Westbrook J, Gosling AS, Coiera E. Do clinicians use online evidence to support patient care?. A study of 55,000 clinicians. J Am Med Inform Assoc 2004; 11 (02) 113-20.
    PubMedGoogle Scholar
  • 11 Gosling A, Westbrook J, Coiera E. Variation in the use of online clinical evidence: a qualitative analysis. Int J Med Inform 2003; 69: 1-16.
    CrossrefPubMedGoogle Scholar
  • 12 Ash JS, Gorman PN, Lavelle M, Payne TH, Massaro TA, Frantz GL. et al. A cross-site qualitative study of physician order entry.[see comment]. J Am Med Inform Assoc 2003; 10 (02) 188-200.
    CrossrefPubMedGoogle Scholar
  • 13 Coiera E. Four rules for the reinvention of health care. BMJ 2004; 328 (7449): 1197-9.
    CrossrefPubMedGoogle Scholar
  • 14 Wear R, Berg M. Computer Technology and Clinical Work: Still Waiting for Godot. JAMA 2005; 293: 1261-3.
    CrossrefPubMedGoogle Scholar
  • 15 Nemeth C, Cook RI. Hiding in plain sight: What Koppel et al. tell us about healthcare IT. J Biomed Inform 2005; 38: 262-3.
    PubMedGoogle Scholar
  • 16 Koppel R, Localio A, Cohen A, Strom BL. Neither panacea nor black box: Responding to three Journal of Biomedical Informatics papers on computerized physician order entry system. J Biomed Inform 2005; 38: 267-9.
    CrossrefPubMedGoogle Scholar
  • 17 Bates DW. Computerized physician order entry and medication errors: finding a balance. J Biomed Inform 2005; 38: 262-3.
    CrossrefPubMedGoogle Scholar
  • 18 Friedman C, Elstein A, Wolf F, Murphy G, Franz T, Heckerling P. et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: A multisite study of 2 systems. JAMA 1999; 282 (19) 1851-6.
    CrossrefPubMedGoogle Scholar
  • 19 Friedman C, Wyatt J. Evaluation methods in biomedical informatics. Second edition.. New York: Springer; 2005
    Google Scholar
  • 20 Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N. et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 06 (04) 313-21.
    CrossrefPubMedGoogle Scholar
  • 21 Graber M. The safety of computer-based medication systems. Arch Intern Med 2004; 164 (03) 339-40.
    PubMedGoogle Scholar
  • 22 Coombes ID, Stowasser DA, Mitchell CA, Varghese P. Effect of computerised prescribing on use of antibiotics. Med J Aust 2004; 180 (03) 140-1.
    PubMedGoogle Scholar
  • 23 Bates DW, Cohen M, Leape L, Overhage JM, Shabot M, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001; 08 (04) 299-308.
    CrossrefPubMedGoogle Scholar
  • 24 Ash J, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information systems related errors. JAm Med Inform Assoc 2004; 11 (02) 104-12.
    PubMedGoogle Scholar
  • 25 US Pharmacopiea. MEDMARX: 5th Anniversary Data Report: A Chartbook of 2003 Findings and Trends 1999-2003. 2005
    PubMedGoogle Scholar
  • 26 Koppel R, Metlay J, Cohen A, Abaluck B, Localio A, Kimmel S. et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-203.
    CrossrefPubMedGoogle Scholar
  • 27 Reason J. Human error. Cambridge: Cambridge University Press; 1990
    Google Scholar
  • 28 Fernando B, Savelyich BS, Avery AJ, Sheikh A, Bainbridge M, Horsfield P. et al. Prescribing safety features of general practice computer systems: evaluation using simulated test cases. BMJ 2004; 328 (7449): 1171-2.
    CrossrefPubMedGoogle Scholar
  • 29 Liaw S, Kerr S. Computer aided prescribing: decision support needs to be evidence based. Br Med J 2004; 328: 1566.
    PubMedGoogle Scholar
  • 30 Coiera E. Guide to Health Informatics. 2nd Edition ed.. London: Hodder Arnold; 2003
    Google Scholar
  • 31 Skitka L. Does automation bias decision-making?. Int J Hum Comput Stud 1999; 51: 991-1006.
    CrossrefPubMedGoogle Scholar
  • 32 Wickens C, Hollands J. Engineering Psychology and Human Performance. New Jersey: Prentice Hall; 2000
    Google Scholar
  • 33 Skitka L, Mosier K, Burdick M, Rosenblatt B. Automation bias and errors: are crews better than individuals. Int J Aviat Psychol 2000; 10 (01) 85-97.
    CrossrefPubMedGoogle Scholar
  • 34 Smith A, Mort M, Goodwin D, Pope C. Making monitoring ‘work’: human-machine interaction and patient safety in anaesthesia. Anaesthesia 2003; 58 (11) 1070.
    CrossrefPubMedGoogle Scholar
  • 35 Koski E, MkivirtaA SukuvaaraT, Kari A. Frequency and reliability of alarms in the monitoring of cardiac postoperative patients. J Clin Monit Comput 1990; 07 (07) 129-33.
    CrossrefPubMedGoogle Scholar
  • 36 Lawless S. Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med 1994; 22 (06) 981-5.
    CrossrefPubMedGoogle Scholar
  • 37 Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. JAm Med Inform Assoc 2004; 11 (04) 270-7.
    PubMedGoogle Scholar
  • 38 Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21) 2625-31.
    CrossrefPubMedGoogle Scholar
  • 39 Hsieh T, Kuperman GJ, Jaggi T, Hojnowski-Diaz P, Fiskio J, Williams D. et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004; 11 (06) 482-91.
    CrossrefPubMedGoogle Scholar
  • 40 Friedman C, Elstein A, Wolf F, Murphy G, Franz T, Heckerling P. et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: A multisite study of 2 systems. JAMA 1999; 282 (19) 1851-6.
    CrossrefPubMedGoogle Scholar
  • 41 Westbrook J, Coiera E, Gosling AS. Do online information retrieval systems help experienced clinicians answer clinical questions?. J Am Med Inform Assoc 2005; 12 (03) 315-21.
    CrossrefPubMedGoogle Scholar
  • 42 Lau A, Coiera E. A Bayesian model that predicts the impact of Web searching on decision-making. Journal of American Society of Information Science and Technology. In press 2006
    PubMedGoogle Scholar
  • 43 Open Clinical. Quality and safety of clinical decision support systems a draft protocol for discussion, Open clinical green paper. 2002 http:// www.openclinical.org/docs/int/docs/qands011.pdf
    PubMedGoogle Scholar
  • 44 Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC. et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; Dec; 116 (06) 1506-12.
    CrossrefPubMedGoogle Scholar