CC BY-NC-ND 4.0 · Yearb Med Inform 2019; 28(01): 071-077
DOI: 10.1055/s-0039-1677907
Section 2: Human Factors and Organizational Issues
Survey
Georg Thieme Verlag KG Stuttgart

Human Factors and Usability for Health Information Technology: Old and New Challenges

Pascale Carayon
1  Department of Industrial and Systems Engineering, Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, Madison, USA
,
Peter Hoonakker
1  Department of Industrial and Systems Engineering, Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, Madison, USA
› Author Affiliations
Further Information

Correspondence to

Pascale Carayon, Ph.D.
Procter & Gamble Bascom Professor in Total Quality Department of Industrial & Systems Engineering
Director of the Center for Quality and Productivity Improvement, Director of the Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison 3126 Engineering Centers Building 1550 Engineering Drive Madison
WI 53706, USA

Publication History

Publication Date:
16 August 2019 (online)

 

Summary

Objectives: Despite national mandates, incentives, and other programs, the design of health information technology (IT) remains problematic and usability problems continue to be reported. This paper reviews recent literature on human factors and usability of health IT, with a specific focus on research aimed at applying human factors methods and principles to improve the actual design of health IT, its use, and associated patient and clinician outcomes.

Methods: We reviewed recent literature on human factors and usability problems of health IT and research on human-centered design of health IT for clinicians and patients.

Results: Studies continue to show usability problems of health IT experienced by multiple groups of health care professionals (e.g., physicians and nurses) as well as patients. Recent research shows that usability is influenced by both designers (e.g., IT vendors) and implementers in health care organizations, and that the application of human-centered design practices needs to be further improved and extended. We welcome emerging research on the design of health IT for teams as team-based care is increasingly implemented throughout health care.

Conclusions: Progress in the application of human factors methods and principles to the design of health IT is occurring, with important information provided on their actual impact on care processes and patient outcomes. Future research should examine the work of health IT designers and implementers, which would help to develop strategies for further embedding human factors engineering in IT design processes.


#

1 Introduction

In several countries, we are moving beyond issues related to implementation and change management of health information technology (IT); today, the basic IT infrastructure is in place and health care organizations are investing in designing and implementing the next generation of health IT “by-products”, such as clinical dashboard, status display, clinical decision support, and patient-facing technologies. Usability and other human factors challenges with health IT continue to be reported, as illustrated in a New Yorker article by Atul Gawande entitled “Why doctors hate their computers” [1]. The experience of usability problems with health IT extends to other health care professionals, including nurses [2]. Staggers et al. [2] gathered input on health IT problems experienced by nurses from a group of nursing leaders, IT experts, engineers and human factors experts. The consensus among the group of 27 experts was that “current vendors do not yet understand how to support nurses’ work, their critical thinking, and decision making” (page 192). For instance, current EHR designs allow for single patient views whereas nurses care for complex groups of patients in a non-linear, non-standardized manner. Patients also experience usability challenges with health information technologies, such as patient portals [3].

Over time, implementation of health IT may lead to benefits for patients as demonstrated by the study by Lin et al. [4], which shows positive outcomes (e.g. hospital mortality) associated with health IT after sufficient time and effort has been spent on implementation, maintenance, and improvement of the system. However, we cannot rely on “time and effort” and just hope that positive outcomes of health IT implementation will happen after a while; in the meantime, negative outcomes may occur, which can endanger patients (e.g., hazards and medical errors), frustrate healthcare professionals, and impede their performance (e.g., burnout). We need to be vigilant and proactive in order to achieve benefits with health IT and avoid negative patient safety outcomes [5]. Human factors and human-centered design play a critical role in ensuring that health IT is well designed and fits with clinical and patient workflows.

We conducted a review of the literature on human factors and usability of health IT published between 2016 and 2019 in various databases: PubMed/Medline, Google Scholar, and PsycINFO. We also searched recent tables of content of major health informatics journals, such as JAMIA, IJMI, ACI, and Health Informatics. Building on chapters published in previous IMIA Yearbooks [6]–[8], we continue to see reports of usability problems with health IT. Emerging literature shows that development and implementation of human-centered design methods to use existing IT infrastructure can create value for patients and clinicians, for instance, in implementing usable technologies, e.g., clinical decision support (CDS) for supporting clinical decision making and care coordination. This chapter also highlights new issues in human-centered design of health IT, such as design of health IT for teams.


#

2 Usability Problems of Health IT and Their Negative Impact on Patients and Clinicians

An extensive account of patient safety issues with health IT was recently published in the US trade literature with a dramatic title: “Death By 1,000 Clicks: Where Electronic Health Records Went Wrong” [9]. The authors provided examples of poor health IT usability experienced by physicians and their impact on patient outcomes. Health IT usability problems experienced by physicians, nurses, and other health care professionals can produce multiple negative outcomes for both patients (e.g. threats to patient safety) and clinicians (e.g. frustration, work-arounds, burnout) [2], [10]–[12].

2.1 Impact of Health IT on Patient Safety

We continue to see evidence of the linkage between poor design of health IT and patient safety consequences. An analysis of medication safety events reported by pediatric clinicians in three children hospitals across the US showed that more than one-third (36%) were related to EHR usability issues, in particular the lack of system feedback and poor visual display of information [13]. The alarming statistic of this study is that 19% of the EHR usability-related events could have resulted in patient harm. A prospective study of medication safety among 624 patients in two ICUs of a medical center identified a total of1,622 preventable adverse drug events after EHR implementation [14]. Thirty-four percent of medication events were related to EHR technology; they included orders with missing information and accepted duplicate medication orders. These findings echo conclusions by Kushniruk et al. [6], [7] in previous IMIA Yearbooks that the lack of attention to human factors with health IT and the failure to engage end-users in the design and implementation processes produce negative consequences for patient safety. Users’ participation and analysis of their work are key elements of the human-centered design process. Increased recognition of the benefits of deep analyses of clinical workflow in this human-centered design process fits with the focus of resilience engineering on the actual work done as opposed to “work-as-imagined” [15], [16].


#

2.2 Impact of Health IT on Clinician Quality of Working Life

In addition to the impact of poor health IT usability on patient safety, there is increasing attention on the negative impact of health IT use on clinicians. Unmanageable demands for clinical documentation and poor health IT usability and workflow integration are some of the EHR-related factors that have been suggested as contributing to clinician burnout. In a survey of more than 5,000 physicians across all specialties in the US, Shanafelt et al. [11] found that the use of EHR and CPOE technologies by physicians was associated with lower physician satisfaction with time spent on clerical tasks and a higher likelihood of professional burnout. A study of EHR event log data showed that primary care physicians spent almost six hours per day interacting with the EHR both during and after clinic hours, therefore, indicating potential challenges with workload and interferences between the professional and private life spheres [17]. A survey of 4,197 physicians confirms the challenges and potential stressful impact of health IT [18]. Sixty-four percent of the physicians agreed or strongly agreed that the EHR added to their daily frustration, 38% reported that time spent on the EHR at home was moderately high or excessive, and 46% indicated that time spent on documentation was “poor or marginal”. All of these EHR-related stressors were significant predictors of physician burnout. Clinicians have the feeling that they put a lot of time and effort to support technology, whereas technology should support them in their clinical activities [1].


#

2.3 Old and New Challenges for Human Factors and Usability of Health IT

Studies on the impact of poor health IT design (and implementation) for both patients and clinicians are important; but we need to better understand the linkage between health IT usability and patient safety. We also need to move beyond analyzing health IT usability and its impact on patient safety, and create more evidence for actually improving the design of health IT. If we want human factors to be taken seriously into account, we should not be shouting from the sideline, but get actively involved in the design and implementation of health IT, and evaluate the impact of our human factors methods and principles on the technology in practice. This research evidence should include information on how to design better (more usable) technologies, as well as their positive impact on key outcomes, e.g., patient safety. Section 4 provides examples of human-centered design applications that produce benefits in usability as well as improvements in care process and patient outcomes. We end the chapter with a discussion of the emerging challenge of designing health IT for teams.


#
#

3 Improved Understanding of Health IT Usability and How It Is Linked to Patient Safety

Two recent viewpoints in JAMA raised the alarm about usability of health IT that “remains suboptimal” and provided policy recommendations to deepen and broaden the application of human factors methods and principles in health IT [19], [20]. Deeper understanding of how usability of health IT can actually be improved is at the core of these recommendations. This improvement will require that we more clearly identify where and when usability gets created or breaks down, how usability is linked to patient safety, and how usability methods are used (and not used) by health IT designers and implementers.

3.1 Improving Usability of Health IT at Design and Implementation Stages

Recent research is helping us to better understand the problem of health IT usability and its relation to patient safety. Usability of health IT is created (or influenced) at both stages of design (e.g., IT vendors) and implementation (i.e. software configuration by health care organizations). Early research showed the lack of systematic integration and use of human factors methods by health IT vendors [21], [22]. Recently, Ratwani et al. [23] documented variations in health IT usability at the level of health care organizations. Using screen capture software, they recorded EHR tasks performed by Emergency Department (ED) physicians in four different hospitals that used two different EHR technologies (i.e., Epic and Cerner). At each hospital, between 12 and 15 ED physicians participated in the study; each physician performed a total of six tasks: two diagnostic imaging orders, two laboratory orders, and two electronic prescriptions. The researchers extracted the following usability measures from the videos: total task duration, number of clicks, and error rate. Results showed large variation in all three usability measures across the two EHR technologies as well as across the hospitals. For instance, the error rate varied between 17% and 50% for the ordering of prednisone taper, between 0% and 30% for ordering Tylenol, and was consistently 0% for entering the lab order of lactate. These data confirm the role of implementation processes in influencing usability. Health care organizations make decisions about customization, configuration, and other implementation processes (e.g., training, user input) that can have significant influence on the final usability of the EHR technology in use. Therefore, improving usability and other human and organizational aspects of health IT will require action and effort by both designers and implemented of health IT.


#

3.2 Developing Better Understanding of Linkage Between Usability and Patient Safety

In addition to understanding that usability is constructed by various actors involved in the design and implementation of health IT, we are developing a deeper knowledge of the mechanisms by which usability influences patient safety. Research by the French human factors group at the University of Lille is producing important knowledge about the impact of usability on patient safety; this is critical research as it helps to untangle the “causal chain” between the technology and outcomes, including patient safety [24] [25] [26]. Watbled and colleagues [26] provided empirical evidence for the linkage between usability flaws (i.e., violations of usability principles), usage problems, and negative outcomes in the context of health IT for discharge summaries. The researchers used three usability evaluation methods: heuristic evaluation based on Scapin and Bastien’s method [27], user testing in an experimental setting, and in-situ observations of the technologies’ use after implementation. They identified multiple usability flaws in the two technologies, and documented the chain of evidence between usability flaws, usage problems, and negative outcomes. For instance, the heuristic evaluation identified the need for additional steps to insert a dictated discharge summary in the electronic medical record. This usability flaw led to a usage problem, as additional tasks were needed. Because of the increased workload, there was a potential negative outcome of inserting the wrong discharge summary in a given patient’s record. This type of in-depth analysis helps to identify the specific usability flaws that can have negative consequences on patient safety, therefore, helping to focus on high-risk design issues that need to be addressed by designers and/or implementers.


#

3.3 Need to Understand Work System of IT Designers and Implementers

As we are developing a deeper knowledge about health IT usability and its role in patient safety, we are clearly justifying the need for more and better application of human factors methods and principles in the design and implementation of health IT. However, a major open issue remains: our lack of knowledge about the work of health IT designers and implementers. This issue represents an interesting dilemma for the human factors community as a core principle of our discipline is to first understand the work situation and then to use this knowledge to develop tools and technologies to support the actual work; but we have not yet applied our human factors approaches to analyze the work of health IT designers. We know very little about the work of health IT designers and implementers; without knowledge on their work context, it remains difficult to elaborate recommendations on how to integrate human factors in the health IT development life cycle.

Outside of health care, research has been conducted to examine the actual work of designers and engineers and their experience of constraints, demands, and stress, which may limit their ability to apply human factors methods and principles systematically while designing, for instance, off-shore oil platforms [28], [29]. Ironically, Charles Perrow, one of the fathers of safety, and the author of “Normal Accidents” [30], conducted the first study on the organizational context of human factors engineering implementation [31], and identified several organizational factors (e.g., lack of support from top management) that can limit the impact of human factors engineers on the design of technologies and equipment. We need to do justice to our own human factors discipline and apply our human factors methods to develop knowledge about the work system of health IT designers and implementers; this knowledge is a critical open research area that will contribute to the effective translation of our human factors knowledge into practice [6]. This research would need to address multiple barriers, such as researchers’ access to health IT vendors and trust between designers, implementers, and researchers [20].


#
#

4 Human-centered Design of Health IT

In previous editions of the IMIA Yearbook, authors emphasized the importance of embedding human factors methods in the early stage of health IT [7], [32] and they described approaches for implementing human-centered design for health IT [32], [33]. We continue to see research that shows the positive impact of human-centered design on usability, as well as important care process and patient outcomes. This research is necessary to demonstrate the “value” of human factors-based approaches in health IT design.

4.1 Impact of Human Factors Engineering in Health IT Design on Usability

We continue to see studies that implement various human factors methods and principles to design health IT applications and demonstrate benefits on various usability dimensions. For instance, Savoy et al. [34], [35] developed a deep understanding of cognitive requirements related to primary care physicians or specialists requesting a consultation; this was accomplished using a combination of semi-structured interviews and direct observations [35]. An interdisciplinary team (including human factors engineers) developed design guidelines for each cognitive requirement, which were then used to develop a prototype. A within-subject experiment of the prototype in comparison to the usual EHR consultation template showed improvement in multiple usability dimensions: perceived satisfaction, fewer mouse clicks, and faster completion but only for one of the subtasks in the consultation request process [34]. A team of ED clinicians and human factors engineers applied cognitive systems engineering methods to develop an ED information system prototype [36]. A usability evaluation of the prototype with 18 ED clinicians (attending physicians, resident physicians, physician assistants, and nurses) provided strong evidence that the ED information system prototype supported a range of cognitive activities, e.g., patient identification with specific characteristics (e.g., longest waiting time in the ED) and assessment of resources for current patient demands. The study participants rated the prototype’s usability and usefulness very high.

Studies are also published on the role of human factors engineering in designing usable health IT for patients, such as a diabetes dashboard embedded in a patient portal [37], an app for geriatric heart failure self-care [38], and an app for medication management [39]. The domain of consumer health IT has embraced human factors and usability methods [40]. Participation of patients and caregivers in health IT design processes can not only help to produce more usable technologies, but also lead to other benefits and patient-centered outcomes such as empowerment and shared decision making [41].

Studies demonstrate that critical applications of human factors engineering in the early stage of health IT design lead to benefits for both clinicians and patients. However, the question remains about dissemination of human factors knowledge on a larger scale. Brunner et al. [42] provided a unique perspective on that question when they examined implementation of user-centered design practices in 170 United States Veteran Affairs primary care clinics. They assessed whether primary care clinics were using the following practices when developing computerized clinical reminders and/ or disease-specific templates, i.e., CDS: (1) pilot testing of CDS before full-scale implementation, (2) assessment of provider satisfaction post-implementation, (3) formal evaluation of CDS usability, and (4) analysis of the impact of CDS on performance. The implementation of these four practices was uneven across the 170 primary care clinics: 74% implemented pilot testing, 62% carried out the assessment of post-implementation provider satisfaction, 36% performed formal usability assessment, and 79% did the analysis of the CDS impact. The practice “analysis of CDS impact on performance improvement” was associated with a greater utility of the CDS as perceived by primary care clinic directors. This research helps us to understand the organizational context (e.g., primary care clinic) in which usability and other human factors methods can (should) be implemented.


#

4.2 Value of Human Factors Engineering in Health IT Design in Improving Care Process and Patient Outcomes

A few studies are emerging to show the actual value of incorporating human factors methods and principles in the health IT design process, demonstrating impact on key care process and patient outcomes. This is a critical step forward in demonstrating the value of human factors engineering as we show that human factors approaches “make a difference” in improving care processes and patient outcomes, such as medication safety [43] and appropriate medication selection during a code [44]. An interdisciplinary team at Johns Hopkins University in the US (including a human factors engineer) applied two rounds of usability testing to significantly reduce the incidence of duplicate medical records from 4% to 2.3% [45]. The improvement was sustained even two years after the new medical record creation process was implemented. Round 1 of usability testing included the mapping of the patient search and selection process used by registrars, the direct observation of registrars to understand their search behaviors, and the identification of barriers experienced by registrars in searching patient names. Results of this round led to multiple modifications to the patient name search process, which were tested in the second round of usability testing. This study confirms the need for an iterative process with multiple methods and approaches in order to ensure good usability of health IT [7]. A critical aspect of this study is demonstrating the impact of using human factors methods on key care processes and outcome measures, such as reduction of the number of duplicate medical records. We need more research that demonstrates the value of human factors engineering for the design of health IT in improving care process and patient outcomes.


#

4.3 Design of Health IT for Teams

As team-based care is increasingly implemented in the continuum of patient care [46], we are seeing an unfortunate mismatch between health IT designed for individual tasks and what we need, i.e., health IT designed for care processes and teams [47]. To design technologies for teams, we need a better understanding of the critical information that team members need to share and communicate for shared team awareness [48], [49]. A couple of research projects focus on the application of human factors engineering to design health IT for teams. Two research teams in Ottawa, Canada [48], [50]–[52] and at the University of Illinois [53] used various human factors methods that provided input to build shared team displays for supporting cognitive work and communication during resuscitations. In Canada, Parush et al., designed a smart display to support cognitive work and team communication during ED resuscitation [48], [50]–[52]. Using multiple methods (e.g., interview of ED clinicians, observation of live resuscitations), they identified key information needs, and then implemented an iterative human-centered design process to develop a prototype for the shared situation display [48], [52]. They tested the display in a simulator environment with three teams made of attending physicians, residents, nurses, and respiratory therapists. Resuscitation teamwork was rated using the Clinical Team Scale (CTS). CTS scores were higher when teams used the shared display than when they did not use it [54]. When teams did not use the shared display, they spent more time exchanging information on the patient status; when they used the display, they spent more time discussing the treatment for the patient. This provided some evidence that the shared status display helped the team in focusing on the patient’s treatment rather than on just exchanging information. A qualitative follow-up analysis of data gathered in the experiment debriefs and focus groups studies confirmed how the shared display helped to build team situation awareness during resuscitations [50].

Wu et al. [53] used the contextual design methodology [55] to develop initial design requirements for a shared display to support cognitive work (i.e., information assembling, recall, and real-time tracking) in cardiac arrest resuscitations. They conducted three simulation sessions with ICU physicians and nurses who were also interviewed post-simulation. This iterative process helped to update the list of design requirements and provided feedback on the design of the shared display. The final stage of this human-centered design process consisted of three scenario-based simulations with 12 ICU physicians and nurses using a mannequin in a simulator. Different physicians and nurses participated in the three simulations in order to increase the diversity of perspectives and inputs in the design process. When using the shared display, physicians and nurses reported lowers levels of mental workload. During the debriefing interviews, they indicated the reasons why and how the shared display could improve their performance. For instance, they mentioned that the system could improve their ability to adhere to best practice guidelines for resuscitation, as well as their common (and shared) understanding of the treatment progression.

There is an urgent need to develop research and knowledge about how to design health IT that can support the work of teams, in particular care teams that are distributed over time and space along the patient journey [56]–[58]. This research requires a shift in focus from the individual to the team, as well as innovative methods to evaluate the impact of these team-based technologies on clinical care processes and team functioning. Several studies [43], [52], [53] have shown how scenario-based simulation can be used to evaluate the impact of human factors-based design of health IT on important clinical outcomes (e.g., prescription errors). We should extend the evaluation of health IT design beyond individual “subjective” measures such as end-user satisfaction, and include team outcomes such as shared understanding and team communication, as well as “objective” measures, e.g., task duration, number of clicks and scrolls needed to navigate through the technology, and proxy clinical outcomes such as the number of medication errors, the number of correct diagnoses, and appropriate clinical decision making. We need further research efforts to develop innovative human factors and usability methodologies such as those relying on multiple methods, including sensors and EHR-based methods [59].


#
#

5 Conclusion

Recent literature on human factors and organizational aspects of health IT continues to bring up problems of usability, poor workflow integration, and potential negative impact on patient safety and other patient and clinician outcomes (e.g., frustration, burnout). It is encouraging to see emerging and growing literature on the actual role of human factors engineering in designing more usable and safer health information technologies. This research needs to grow and address critical aspects of the changing landscape of health care, including designing health IT for teams. Designing usable health IT for individual tasks of clinicians or patients is challenging, but designing health IT for teams with various needs, perspectives, and motivations is even more challenging. Further understanding how the collaborative health IT design process can integrate multiple perspectives is paramount [60].

Recent media attention to the problem of health IT usability and its negative impact on patient safety and clinicians [1], [9] may provide the impetus for increased resources dedicated to health IT usability and the broad application of human factors methods and principles in the design and implementation of health IT. In order to respond to the call for increased integration of human factors in health IT design, we need to better understand the actual work performed by health IT designers. This can be done by applying human factors method and principles to the work of health IT design and implementation: applying human factors approaches to human factors in practice (i.e., health IT design and implementation) is needed.


#
#

Acknowledgments

This research was supported by the Clinical and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational Sciences (NCATS), Grant UL1TR000427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Funding for this research was also provided by the Agency for Healthcare Research and Quality (AHRQ) [Grant No. R01HS022086].


Correspondence to

Pascale Carayon, Ph.D.
Procter & Gamble Bascom Professor in Total Quality Department of Industrial & Systems Engineering
Director of the Center for Quality and Productivity Improvement, Director of the Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison 3126 Engineering Centers Building 1550 Engineering Drive Madison
WI 53706, USA