Cognitive Analysis of Decision Support for Antibiotic Ordering in a Neonatal Intensive Care Unit

Summary

Background: Clinical decision support systems (CDSS) are a method used to support prescribing accuracy when deployed within a computerized provider order entry system (CPOE). Divergence from using CDSS is exemplified by high alert override rates. Excessive cognitive load imposed by the CDSS may help to explain such high rates. Objectives: The aim of this study was to describe the cognitive impact of a CPOE-integrated CDSS by categorizing system use problems according to the type of mental processing required to resolve them.

Methods: A qualitative, descriptive design was used employing two methods; a cognitive walk-through and a think-aloud protocol. Data analysis was guided by Norman’s Theory of Action and a theory of cognitive distances which is an extension to Norman’s theory.

Results: The most frequently occurring source of excess cognitive effort was poor information timing. Information presented by the CDSS was often presented after clinicians required the information for decision making. Additional sources of effort included use of language that was not clear to the user, vague icons, and lack of cues to guide users through tasks.

Conclusions: Lack of coordination between clinician’s task-related thought processes and those presented by a CDSS results in excessive cognitive work required to use the system. This can lead to alert overrides and user errors. Close attention to user’s cognitive processes as they carry out clinical tasks prior to CDSS development may provide key information for system design that supports clinical tasks and reduces cognitive effort.

Keywords

NICU - clinical decision support - alerts - cognitive impact - antibiotics

 

Conflict of Interest

The authors declare they have no conflicts of interest in this research.

• References

• 1 Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, Classen DC, Bates DW. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14 (01) 29-40.
• 2 Evans RS, Pestotnik SL, Classen DC, Burke JP. Evaluation of a computer-assisted antibiotic-dose monitor. Ann Pharmacother 1999; 33 (Suppl. 10) 1026-1031.
• 3 Eslami S, Abu-Hanna A, de Keizer NF, de Jonge E. Errors associated with applying decision support by suggesting default doses for aminoglycosides. Drug Saf 2006; 29 (09) 803-809.
• 4 Van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147.
• 5 Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. Journal of the American Medical Association 2005; 293 (10) 1223-1238.
• 6 Lin C-P, Payne TH, Nichol WP, Hoey PJ, Anderson CL, Gennari JH. Evaluating clinical decision support systems: Monitoring CPOE order check override rates in the Department of Veterans Affairs’ Computerized Patient Record System. J Am Med Inform Assoc. 2008 M2453.
• 7 Sharma V, Simpson RC, LoPresti EF, Mostowy C, Olson J, Puhlman J, Hayashi S, Cooper RA, Konarski E, Kerley B. Participatory design in the development of the wheelchair convoy system. J Neuroeng Rehabil 2008; 5: 1.
• 8 Hsieh TC, Kuperman GJ, Jaggi T, Hojnowski-Diaz P, Fiskio J, Williams DH, Bates DW, Gandhi TK. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004; 11 (Suppl. 06) 482-491.
• 9 Unertl KM, Weinger MB, Johnson KB, Lorenzi NM. Describing and modeling workflow and information flow in chronic disease care. J Am Med Inform Assoc 2009; 16 (06) 826-836.
• 10 Cresswell K, Worth A, Sheikh A. Actor-network theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inform Decis Mak 2010; 10 (01) 67-77.
• 11 Johnson KB, Fitzhenry F. Case report: activity diagrams for integrating electronic prescribing tools into clinical workflow. J Am Med Inform Assoc 2006; 13 (04) 391-395.
• 12 Hutchins E, Hollan J, Norman D. Direct manipulation interfaces. Human-Computer Interaction 1985; 1 (04) 311-338.
• 13 Fitousi D, Wenger MJ. Processing capacity under perceptual and cognitive load: A closer look at load theory. J Exp Psychol Human 2011; 37 (03) 781-798.
• 14 Hazlehurst B, Gorman PN, McMullen CK. Distributed cognition: An alternative model of cognition for medical informatics. Int J Med Inform 2008; 77 (04) 226-234.
• 15 Westbrook JI, Braithwaite J, Georgiou A, Ampt A, Creswick N, Coiera E, Iedema R. Multimethod evaluation of information and communication technologies in health in the context of wicked problems and socio-technical theory. J Am Med inform Assoc 2007; 14 (06) 746-755.
• 16 Rizzo A, Marchigiani E, Andreadis A. editors. The AVANTI project: Prototyping and evaluation with a cognitive walkthrough based on the Norman’s model of action. Symposium on Designing Interactive Systems;; 1997 Siena, Italy.:
• 17 Baxter GD, Monk AF, Tan K, Dear PR, Newell SJ. Using cognitive task analysis to facilitate the integration of decision support systems into the neonatal intensive care unit. Artif Intell Med 2005; 35 (03) 243-257.
• 18 Polson P, Lewis C, Rieman J, Wharton C. Cognitive Walkthroughs: A method for theory-based evaluation of user interfaces. Institute of Cognitive Sciences Technical Report 91–01: 53.
• 19 Saitwal H, Feng X, Walji M, Patel V, Zhang J. Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis. Int J Med Inform 2010; 79 (07) 501-506.
• 20 Pfautz J, Roth E. Using cognitive engineering for system design and evaluation: A visualization aid for stability and support operations. Int J Ind Ergonom 2006; 36 (05) 389-407.
• 21 Currie L, Sheehan B, Graham PL, 3rd, Stetson P, Cato K, Wilcox A. Sociotechnical analysis of a neonatal ICU. Stud Health Technol Inform 2009; 146: 258-262.
• 22 Nielsen J. Ten Usability Heuristics. 2005 [cited 2011 Oct. 6]; Available from: http://www.useit.com/papers heuristics/heuristiclist.html
• 23 Senathirajah Y, Kaufman D, Bakken S. Cognitive analysis of a highly configurable web 2.0 EHR interface. AMIA Annu Symp Proc 2010: 732-736.
• 24 Killelea BK, Kaushal R, Cooper M, Kuperman GJ. To what extent do pediatricians accept computer-based dosing suggestions?. Pediatrics 2007; 119 (01) e69-e75.
• 25 Xie Z, Zhang J. Development of a taxonomy of representational affordances for electronic health record system. AMIA Annu Symp Proc. 2006: 1149.
• 26 Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530.