Are We There Yet? Human Factors Knowledge and Health Information Technology – the Challenges of Implementation and Impact

Summary

Objective: To review the developments in human factors (HF) research on the challenges of health information technology (HIT) implementation and impact given the continuing incidence of usability problems and unintended consequences from HIT development and use.

Methods: A search of PubMed/Medline and Web of Science® identified HF research published in 2015 and 2016. Electronic health records (EHRs) and patient-centred HIT emerged as significant foci of recent HF research. The authors selected prominent papers highlighting ongoing HF and usability challenges in these areas. This selective rather than systematic review of recent HF research highlights these key challenges and reflects on their implications on the future impact of HF research on HIT.

Results: Research provides evidence of continued poor design, implementation, and usability of HIT, as well as technology-induced errors and unintended consequences. The paper highlights support for: (i) strengthening the evidence base on the benefits of HF approaches; (ii) improving knowledge translation in the implementation of HF approaches during HIT design, implementation, and evaluation; (iii) increasing transparency, governance, and enforcement of HF best practices at all stages of the HIT system development life cycle.

Discussion and Conclusion: HF and usability approaches are yet to become embedded as integral components of HIT development, implementation, and impact assessment. As HIT becomes ever-more pervasive including with patients as end-users, there is a need to expand our conceptualisation of the problems to be addressed and the suite of tactics and strategies to be used to calibrate our pro-active involvement in its improvement.

Keywords

Human factors engineering - usability - patient safety - health information technology

1 Introduction

In the healthcare and bio-medical domains, health professionals, health researchers, and increasingly patients are interacting with ever-greater numbers of digital information systems, applications and services. From artificial intelligence and robotics through to the internet of things (IoT), smartphone apps, and wearable computing, massive amounts of health related data and information are being collected, analysed, and exchanged, with the aim of delivering benefits from enhanced information access, accuracy, and usability [[1]].

For governments, the healthcare industry, health professionals, and patients, digitisation presents many transformative opportunities but also poses risks. At the broadest level, improving the implementation of data security protocols and data privacy standards continues to be recognised as critical for ensuring the protection of individual rights and the functioning of mission critical health systems as we move ever closer to personalised medicine [[2]]. For health professionals working in complex socio-technical care environments, ensuring the delivery of safe and high quality care remains paramount when the introduction of health information systems continues to produce unintended consequences [[3]]. While for patients, differences in technical, textual, and health literacy highlight the risks of accentuating pre-existing inequalities as digital behaviour change interventions become more common [[4]].

These risks and challenges are not new. From the early days of the ‘computer age’, alongside considerable techno-centric optimism, many researchers recognised that successful technology adoption and use relied on a range of factors. These included understanding human factors, the human-computer interface, the organisational contexts of use, and the complexity of the tasks to be undertaken [[5], [6], [7]]. In healthcare, for more than two decades, we have been actively engaged in developing and deploying methods to explore human factors (HF) and the usability and safety of health information systems, applications, and services [[8], [9], [10]]. Across the design, implementation, and evaluation of these systems, we have long understood that most challenges result from a failure to design for the complexity of socio-technical systems in healthcare and/or a failure to understand and engage end-users in technology adoption and use in context [[11], [12], [13]]. Grappling with the complexity of socio-technical systems is not an easy task. But HF approaches have been enhanced by recognising that the level of healthcare complexity can be understood as depending on the number and degree of interrelatedness amongst healthcare system components [[14]]. Similarly, socio-technical approaches do promote a focus on the interactions and emergent properties of social systems and technological artefacts in context as part of much contemporary HF research [[15], [16]]. Indeed for nearly twenty years, socio-technical models and frameworks suitable to enhance HF and usability research have been deployed in healthcare environments [[17], [18]] with more recent approaches specifically adapted to enhance the understanding of complex adaptive healthcare systems [[19]].

Unfortunately, despite this depth of HF knowledge and the wide range of methodological tools and techniques available, research continues to highlight contemporary evidence of poor design, poor usability, technology-induced errors, and unintended consequences from health information systems, applications, and services. In a recent systematic review of the types and causes of prescribing errors arising from the use of computerized provider order entry (CPOE) systems, Brown et al. [[20]] identified eight themes related to these errors including: computer screen display; drop-down menus and auto-population; wording; default settings; non-intuitive or inflexible ordering; repeat prescriptions and automated processes; users’ work processes; and clinical decision support systems. They also highlighted evidence showing that a lack of CPOE system flexibility directly contributed to users developing error-prone workarounds, e.g. inclusion of, sometimes, contradictory free-text commentaries. In another systematic review, Gephart et al. [[21]] examined available evidence on nurses’ experiences with the unintended consequences of using electronic health record (EHR) systems. The results highlighted the impact of EHRs on changing workflows in ways that may threaten patient safety, cause difficulties in accessing information necessary for patient care decisions, and sometimes impact on the efficiency of work.

The frequency and the volume of contemporary reporting on usability problems remain a cause for concern. As the diversity of HIT grows, the persistence of usability issues has led to considerable discussion and debate. As Ratwani et al. [[22]] and McCray et al. [[23]] highlighted, there is a need for more effective translation of our knowledge into practice and for finding ways to enhance our impact on the usability and safety of HIT systems being implemented. These themes have continued to be a focus in relation to ongoing problems with major EHR systems. As Koppel et al. [[24]] pointed out, there is a danger when vendors of large EHR systems continue to limit access to the data they hold on the software glitches, errors, and usability problems exhibited by their systems. This resonates with earlier calls by Sinsky et al. [[25]] for improved transparency on comparative user experience data and greater clarity on how it should be structured and reported. Unfortunately, while EHR usability challenges are widely reported in the press [[26], [27], [28]], the human factors knowledge available to mitigate these problems has yet to be applied consistently. Aligned to this, as Shanafelt et al. [[29]] have recently reported, is another unintended consequence from continued poor EHR usability - that of increasing levels of clinician “burn-out” [[29]].

Beyond EHRs, another area with numerous reported usability issues relates to the widespread diffusion of patient-centred e-Health applications and services. Wildenbos et al. [[30]] and Baysari et al. [[31]] have investigated the extent to which HF knowledge is contributing (or not) to ensuring safe and usable designs and/or reducing unintended consequences for patients as users. Purkayastha et al. [[32]] advocated for finding new ways to tailor approaches in this rapidly expanding area, and Sawesi et al. [[33]] provided a systematic review on the impact of technology on patient engagement and health behaviour change. This review highlights the need for more research, as patients are encouraged to adopt and use ever greater numbers of health-related digital applications and services.

This paper reviews recent developments in human factors (HF) research on the challenges of health information technology (HIT) implementation and impact given the continued incidence of usability problems and unintended consequences from HIT development and use. A search of PubMed/ Medline and Web of Science® identified HF research published in 2015 and 2016. Electronic health records (EHRs) and patient-centred HIT emerged as significant foci of recent HF research. Following review, the authors selected prominent papers highlighting ongoing HF and usability challenges in these areas. This selective rather than systematic review of recent HF research highlights these key challenges and reflects on their implications for the future impact of HF research on HIT. The research presented provides evidence of continued poor design, implementation, and usability of HIT, as well as technology-induced errors and unintended consequences. The paper promotes support for:(i) strengthening the scientific evidence base on the benefits and impacts of HF approaches;(ii) improving knowledge translation and dissemination on how to practically implement HF approaches during design, implementation, and evaluation of HIT; and (iii) increasing transparency, governance, and enforcement of HF best practices at all stages of the HIT system development life cycle.

2 Technology, Human Factors, and Complex HealthCare Environments

Understanding interactions within socio-technical systems and applying theory and methods to these systems to optimise their design and usability remain at the heart of HF research. To do this requires consideration of the organisational, physical, and cognitive aspects of these systems, as well as the tools and techniques that ensure they are developed to be ‘compatible with the needs, abilities, and limitations of people’ [[34]].

In healthcare environments, the increased use of HF approaches has grown in response to evidence of usability problems, technology-induced errors, negative outcomes in terms of patient care and safety, and other unintended consequences from HIT [[35]]. Unsurprisingly, as HIT has become more pervasive and complex, there has been a growing awareness of the need to evolve the approaches, tools, and techniques deployed by HF researchers [[11], [13]]. As Patel et al. [[36]] have noted, numerous HF models have been developed to aid the understanding and analysis of healthcare settings including: aviation-based models; macro-ergonomic models for improving patient safety and quality; the systems engineering initiative for patient safety (SEIPS); health professional performance models; and four stage HF work system optimization model. However, in increasingly complex environments with multiple interactions amongst people, technology artefacts, and contexts of use, the relevance of socio-technical approaches for integrating and framing the use of different models, tools, and techniques has been recognised. The socio-technical approach also supports the incorporation of new tools and techniques that may better aid the implementation of HF knowledge practice, e.g. clinical simulations, temporal methods for analysis, and integration with quality improvement (QI) ‘lean’ methods [[36]].

As Kushniruk et al. [[37]] have recently discussed, a reflection on the evolution of HF and usability engineering approaches to HIT is useful for stimulating thinking about how far we have come and where the key future challenges lie. This is especially the case if we are to be able to assess the success of our approaches in the design, implementation, and evaluation of HIT for delivering improved patient care and system safety. Kushniruk et al. argued that there is an increasing need to apply HF approaches earlier in the HIT system development life cycle to reduce the incidence of technology-induced errors and optimise system safety. Usability problems contributing to these errors are well known and include screen navigation issues, inconsistent user interfaces, limited end-user feedback, and poor system integration into the clinical workflow. Kushniruk et al. went on to advocate for a ‘layered safety net approach to ensuring usability and safety'. This approach employs a phased sequence of tests and mixed HF methods (e.g. cognitive walkthroughs, heuristic evaluations, video-observations, clinical simulations) prior to HIT deployment. The aim is to increase the probability of identifying and rectifying negative unintended consequences before HIT systems are released into healthcare environments [[37]].

In the last twenty years, most HF approaches to HIT have been focused in three main areas: (i) the analysis of healthcare work situations involving HIT; (ii) usability studies of HIT; (iii) studies providing evidence or evaluation of the results of using HF approaches on HIT. Combining these studies confirmed that much HIT had poor usability, contributed to technology-induced errors and/or had unintended consequences in healthcare environments. These studies also provided evidence that using HF approaches could positively impact on adoption/use of HIT and may contribute to patient and system safety [[38]]. However, it was readily acknowledged that there was a need to move rapidly towards more standardisation of HF methods for analysing HIT and in reporting results; to enhance translation/calibration of HF findings into tangible specifications for IT designers; and to develop consistent methods for evaluating and validating the impact of HF on HIT. Beyond this, it was also well-understood that the implementation of most HIT was focused on configuring system features and functions to local circumstances rather than on paying attention to usability or end-users contexts of use. Nearly a decade on, it is perhaps sobering to reflect on how many of these issues continue to resonate with contemporary HF concerns about HIT [[36], [37], [38], [39]].

As Beuscart-Zephir et al. [[39]] have discussed, governments and international bodies in a number of countries have now introduced guidelines and regulations aimed at reducing medical errors, enhancing the usability of HIT, and improving patient and system safety. However, despite these developments, new research indicates that there remain a number of substantive, methodological, and conceptual challenges inhibiting the widespread and systematic use of HF knowledge in the design, implementation, and evaluation of HIT.

At the substantive level, the widespread adoption of EHR systems has stimulated a large number of recent studies. Ratwani et al. [[22], [40], [41]] have raised serious concerns about continued poor usability in EHR design and implementation, as well as limitations relating to the regulation around their certification; Babbott et al. [[42]] have also reported negative impacts from poor EHR usability on increased clinician stress extending to primary care [[42]]. Friedman et al. [[43]] have provided a useful typology of EHR workarounds in primary care [[43]]; and Graber et al. [[44]] have confirmed the continued occurrence of ‘adverse events associated with health IT vulnerabilities causing extensive harm and mortality across the continuum of health care settings’ [[44]]. At the same time, a systematic review by Ellesworth et al. [[45]] has questioned the quality of many published usability evaluations and identified ‘a paucity of quality published studies describing scientifically valid and reproducible usability evaluations at various stages of EHR system development’. Ellesworth et al. identified this lack of formal and standardized reporting of EHR usability evaluation results as a major knowledge gap in the field. Similarly, in the rapidly emerging area of patient-centred HIT, evidence on the use and impact of HF methods leaves many questions unanswered especially in relation to ‘whether and to what extent’ these methods mediate the safe use and positive impact of these technologies [[30], [31]]. As Yardley et al. [[46]] and Kayser et al. [[47]] also pointed out, the extent to which contemporary approaches are adequately engaging patients and capturing their user needs in ways that meaningfully accommodate different levels of technical, textual, and e-health literacy in technology design and implementation remains unclear.

At the methodological level, recent research by Ammenwerth [[13]] has highlighted that with the increasing complexity of healthcare and HIT there is a genuine need for more comprehensive evaluation studies and approaches to address the challenges that pertain to ‘the quality of studies; publication bias; reporting quality; availability of publications; systematic reviews and meta-analyses; training of health IT evaluation experts; translation of evidence into health practice; and post-market surveillance’. Following Ammenwerth et al. [[48]], it is also evident that alongside this need to enhance the evidence base, there is a widespread awareness that much work is still to be done.

At the conceptual level, Kushniruk et al. [[37]] have also highlighted the gap that continues to exist between evidence, knowledge, and insight within HF research. This gap inhibits the optimisation of usability and safety of HIT and contributes to the ongoing prevalence of usability issues and problems. Kushniruk et al. attributed part of this gap to a lack of appropriate knowledge translation and limited dissemination and education of HF approaches within government, industry, and society. Singh et al. [[49]] also acknowledged the gap but attributed it to the lack of a solid and convincing conceptual foundation for use in practice. However, as Koppel [[50]] has argued, beyond these perspectives, most contemporary models of incident causation at the interface between users and HIT are inadequate, overly static and potentially hazardous if used as explanatory models. More fundamentally, as we move from studies involving post-mortem descriptions of HF related problems with HIT to engage more pro-actively in implementations of HIT and dynamic assessments of impact and outcomes, there may be a need for reconfiguring our conceptualisation of the problems to be addressed. As Coiera [[51]] has recently argued, maybe there is also a need for a ‘new informatics geography’ where we better understand and can quantify the value of information and the role of HIT in realising that value per se.

Before discussing some of the more positive developments evident in recently published HF research, it is useful to briefly review the recently published evidence of continued poor design, implementation, and usability of HIT, as well as the reported evidence of ongoing technology-induced errors and unintended consequences experienced by different types of end users.

3 Technology, Human Factors and Usability: Old and New Problems

The diffusion and market dominance of a small number of very large vendors of complex EHR systems has continued apace and now extends well-beyond North America. Solution implementations of these large EHR systems tend to follow a familiar pattern regardless of the individual product. A standard framework is modified and configured to achieve local interoperability and terminological compliance. While vendors promote this approach as being cost-competitive, it remains relatively unsuited to meaningfully tailoring these systems to accommodate local user needs, user contexts, or workflow tasks.

In this context, it is perhaps unsurprising that, as these systems become more pervasive, there has been an increase in reports of HF and usability problems with EHR systems both in North America and internationally. Already by 2009, the US Healthcare Information and Management Systems Society (HIMSS) EHR Usability Task Force [[52]] had identified, ‘usability as one of the main factors – possibly the most important factor – hindering widespread adoption of EMRs’. More recently, as these EHRs have become more ubiquitous, usability rather than low adoption has been identified as an ongoing problem. Riskin et al. [[53]] have highlighted continuing issues including electronic records being difficult to read and cumbersome to use with difficulties for users in being able to rapidly identify essential information. They also identified how these usability problems have been associated with new forms of technology-induced errors resulting from poor information presentation and design and limited appropriate tailoring to end users, their work tasks, or their contexts of use. Kellermann et al. [[54]] have raised the question as to what it will take to achieve ‘the as-yet-unfulfilled promises of health information technology’. They argued that the disappointing performance of HIT is primarily due to lack of ease of use, failure of healthcare providers and institutions to re-engineer healthcare processes, and poor choice of systems in terms of HF and interoperability. Perhaps one of the most problematic issues remains this lack of user fit between systems procured by healthcare organizations on the basis of organizational needs and actual user requirements and required usability in context for clinical workflows on the ground.

Recognition of these problems has in recent years led to action by governments in North-America and in Europe to require the use of validated HF and usability methods. Although, as Beuscart et al. [[39]] have pointed out, many vendors and manufacturers continue to experience genuine and practical difficulties in applying HF and usability methods beyond any mere reluctance to respond to these new types of rules and regulations. Certainly these movements towards regulation and certification of usability in HIT are to be welcomed and may lead to positive improvements in the usability of systems, reduce technology-induced errors, and mitigate unintended consequences. Vendors in both Europe and the USA are already applying user-centred design techniques to comply with the European Medical Device Directive [[55]] and USA certification requirements of the Office of the National Coordinator (ONC) for HIT [[56]]. In the USA, ONC certification requirements involve vendors being required to provide written statements on the development process they used, along with the results of the usability tests conducted. However, Ratwani et al. [[22], [41]] have recently studied the current application of these certifications and found they are generally weak with poor governance and oversight. One review conducted found that 63% of the 41 vendors studied used only 15 study participants for usability testing, and only 9% used at least 15 participants with any clinical background. These studies highlight that HIT products can be certified for use with very limited and weak usability analyses. This work shows that although it is a positive direction to include usability certification, such certification must be conducted with much more rigor and in-depth. These studies also echo the European experience with a clear need to systematically improve knowledge of usability processes amongst HIT vendors given that many continue to display misconceptions about user-centred design and usability processes [[22], [39], [40], [41]].

The American Medical Informatics Association (AMIA), the Agency for Healthcare Research and Quality (AHRQ), and the National Institute of Standards and Technology (NIST) have continued to explore ways to address these ongoing usability challenges. They have also recently recognised that increased requirements for computer-based documentation by health professionals to meet meaningful use requirements have contributed to further human-computer interaction issues. This has led to reports of increased time and effort expended by clinicians to document interactions with patients (i.e. increased number of mouse clicks and steps needed to complete documentation of patient cases) [[56]]. In response, there have been numerous white papers workshops, webinars, and training programs addressing the HF and usability challenges of EHRs. To specifically target these issues, AMIA developed a task force that has generated 14 guiding principles for improving usability in HIT. Following Middleton et al. [[57]], these principles include: consistency of design and standards, visibility of system state, match between systems and world, minimalist design, minimization of memory load, informatics feedback, flexible and customizable systems, useful error messages, technology-induced error prevention, reversible actions, clear closure, user language utilization, user control and appropriate help and documentation. Beyond these principles, AMIA has also promoted a HF and usability research agenda to prioritize standardized use cases for EHR functionalities, develop a set of measures for adverse events related to health IT use, and research and promote best practices for safe implementation of EHRs. AMIA have also identified the need for policy and regulatory changes and made recommendations including: standardization and interoperability across EHR systems should take into account usability concerns; establishment of an adverse events reporting system for health IT and voluntary health IT event reporting; and the need for the development and dissemination of an educational campaign targeting safe and effective use of EHRs [[57]]. They also made recommendations focused on the health IT industry and clinicians. For vendors, AMIA is promoting the development of a common user interface style guide for selecting EHR functionalities and formal usability assessments on patient-safety EHR functionalities, while for clinicians, there is encouragement for the adoption of best practices during EHR system implementations, project management, and for the reporting of IT-related adverse events.

Of course, the question of the impact of these and other types of initiatives on the safety of HIT has also been an area of strong interest within recent HF research. Magrabi et al. [[58], [59]] have used patient safety concepts to identify and classify the range and type of unintended consequences arising from HIT. While Marcilly et al. [[60]] and Schiff et al. [[61]] have analysed and tabulated usability flaws and medication errors arising from medication alerting systems and physician order entry systems respectively. Aligned with these types of studies, Castro et al. [[62]] have recently identified 120 health IT-related sentinel events primarily associated with the socio-technical dimensions of the human-computer interface, workflows, communications, and clinical content.

Another area of HIT area that has been a recent focus for HF research relates to patients as users of HIT. As was noted above, concerns about whether, and to what extent, HF and usability research is contributing to ensuring safe and usable health technologies, applications, and services require further analysis [[30], [31]]. But as Brenner et al. [[63]] noted in their systematic review exploring links between HIT and patient outcomes, this is perhaps part of a broader problem as ‘many areas of health IT application remain understudied and the majority of studies have non-significant or mixed findings’. Similarly, Guise et al. [[64]] have identified patient safety risks associated with telecare and recommended greater use of HF and usability approaches.

Interestingly, some attempts to resolve these issues in specific patient-centred HIT interventions have recently been made. Matthew-Maich et al. [[65]] have produced a scoping review investigating mobile health technologies for managing chronic conditions in older adults: they reported that most applications target specific conditions, are designed primarily for use by patients and care providers, and that evidence-based guidance on how to ensure acceptability and usability of mHealth innovations continues to be limited. Zapata et al. [[66]] also explored mobile health technologies through a systematic review to investigate usability evaluation processes. They highlighted the importance of usability for adoption and use of mHealth by patients, many of them having limited experience and/or problems using mobile applications. Based on their review, they recommended that usability evaluation processes be enhanced by combining more than a single method when evaluating mHealth. Queiros et al. [[67]] illustrated how rapidly HIT is developing in this area with a systematic review of usability and accessibility in ambient assisted living (AAL) technologies. Their review aims to understand how and to what extent user interaction occurs in AAL development and evaluation processes. The results revealed a need to improve integration and interoperability of existing technologies and to promote user-centred design and increased end-user involvement to better address usability and accessibility issues.

This section has highlighted some of the recent HF research confirming evidence of continued poor design, and implementation and usability issues with HIT. The recent focus of HF research into EHRs and patient-centred HIT provides evidence of ongoing technology-induced errors and unintended consequences experienced by a variety of different end users including patients. Continuing challenges with EHRs include a limited system ‘fit’ with users, contexts, and tasks that has contributed to negative impacts on clinician workflow, clinician stress, and poor patient outcomes. For patient-centred HIT, it is evident that there is still considerable work to do before we better understand usability amongst different types of patients and the safety impacts and benefits of rapidly emerging mobile and ambient technologies, applications, and services.

4 Discussion: Moving Forward

Despite the ongoing problems discussed above, this review has also identified a number of positive developments and recommendations that make HF and usability research on HIT progress Along with well-known HF successes in enhancing medication safety and re-engineering discharge (RED), Kushniruk et al. [[37]] provided ample evidence that effective HF and usability methods, tools, and techniques do exist. They also indicated that when these methods are applied correctly and early in the HIT system development life cycle, they can significantly improve usability, reduce technology-induced errors, and mitigate unintended consequences. However, as previously discussed in this paper, a major problem continues to be how to ensure these approaches are actually embedded as integral components of all HIT development, implementation, and impact assessment.

In the USA, Sheikh et al. [[68]] consider that some of the problems with HIT are the policy and regulatory settings that support a fee-for-service paradigm that does not prioritise usability and interoperability. To address these challenges, they have recommended ‘mandating vendors open-up their application program interfaces (APIs)’. They also advocated ‘incentivising the development of low-cost consumer informatics tools and improvements in the balance between data privacy and reuse’. The National Patient Safety Foundation (NPSF) [[69]] and Zhang et al. [[70]] have also both made significant contributions to approaches that will practically address many of the HF and usability issues with EHRs. For example, the NPSF has recommended: greater transparency on health IT safety and best practices; identification and risk mitigation related to adverse effects and unintended consequences of HIT; instantiation of routine testing of system safety; and improved engagement of patients and their families with HIT. Similarly, other national institutes including the NIST, AHRQ, and ONC have made recommendations and guidelines to improve usability and safety of EHRs [[71], [72], [73]]. The problem is not that we do not know what to do, but rather to make it happen!

Similarly in Europe efforts are underway to practically address the challenges of usability with HIT. Kaipio et al. [[74]] have engaged in repeated national cross-sectional surveys with clinicians in Finland to benchmark improvements over time with the usability of currently used EHR systems in the period 2010–2014. While the results highlight continuing problems, this approach appears to be one of the first national surveys of this type (i.e. focused on the usability of EHRs). It also clearly provides information for healthcare providers, decision makers, and politicians about the current state of EHR usability, differences between vendor products, as well as options for improving EHR usability. Other practical efforts to improve the implementation and impact of HIT through HF research have been to use simulation to assess multi-functional systems in complex work situations as conducted by Jensen et al. [[75]] in Denmark. There has also been the development of a knowledge base comprised of evidence-based approaches to decrease technology-induced errors by Marcilly et al. [[76]] in France. Lyon et al. [[77]] have developed the HIT academic and commercial evaluation (HIT-ACE) methodology underpinned by user-centred design and implementation science approaches to aid stakeholder decision-making related to HIT adoption and use. In Australia, the Commission for Safety and Quality in HealthCare [[78]] has been active in developing practical resources to aid the safe implementation and use of electronic medication management systems including guidelines with the on-screen display of clinical and consumer medication information.

As highlighted by this brief selected review, steps are already underway within the HF and usability research community to expand the impact of our research on the design, implementation, and evaluation of HIT. It is clear that there is still much to do and that moving forward will require action at multiple levels including: (i) strengthening the scientific evidence base on the benefits and impacts of HF approaches; (ii) improving knowledge translation and dissemination on how to practically implement HF approaches during design, implementation, and evaluation of HIT; and (iii) increasing transparency, governance, and enforcement of HF best practices at all stages of the HIT system development life cycle.

5 Conclusion: It’s the Journey not the Destination that Matters

This paper has reviewed recent significant developments in human factors (HF) research relating to the ongoing challenges of implementation and impact of health information technology (HIT) in the context of the continuing incidence of usability problems and unintended consequences from its development and use. Based on the research reported in this paper, there is a need for the discipline to continue to develop more calibrated responses across the design, implementation, and evaluation of HIT to support moving closer to embedding HF and usability knowledge into the development of a genuine safety culture for HIT. Moving forward will involve HF researchers becoming more pro-actively involved during HIT design and implementation, and in the dynamic assessment of our impact on safer and more usable health technologies, applications, and services. In addition, as a greater number of vendor-based systems (such as EHRs) are implemented, the area of bringing usability engineering into their customization and deployment also remains to be explored (and this may account to some extent for the continued reports of usability problems). Recent evidence indicates that certification of product development (i.e. certification of an EHR product development cycle) may not ensure that it will be either usable or safe when the certified system is purchased by healthcare organizations and deployed in a hospital setting that is likely to be far from the healthcare site(s) where it was developed. Indeed with the dominance of a small number of large EHR vendors in North America and now internationally, the evidence suggests that this is increasingly the case, with systems being deployed in different healthcare systems, contexts, and countries from where these systems were originally developed [[79]].

Beyond the increasing complexity of HIT, this paper has also highlighted the increasing diversity of health technology, applications, and services. HF research must directly grapple with the increasing diversity of HIT developed and deployed as part of healthcare interventions, including those that engage patients and/or their family carers as the primary users of these technologies, applications, and services. This suggests that the practice of HF and usability research is not a destination per se, but rather a journey. We must therefore explicitly recognise the dynamic and ever-changing relationships that pertain in each socio-technical system involving technology, health professionals, and patients. While this implies that we must be ready to tailor our approaches to the circumstances and contexts we find in the field, we must also simultaneously find meaningful ways to generate a stronger evidence base for our work.

As HIT becomes ever-more pervasive in healthcare including with patients as end-users, there is also a need to expand our conceptualisation of the problems that are to be addressed and the suite of tactics and strategies that we use across the design, implementation, and evaluation of HIT. We must be willing to evolve and build partnerships with other disciplinary colleagues, and we must also be careful not to over prioritise either techno-centric or info-centric perspectives in our analyses. Following Martin et al. [[80]], it is important that we reflect on what we are trying to achieve with HIT, as they argue, for example, that medical records currently conceptualised as documentation are ‘at risk of over-taking the delivery of care in terms of time, clinician focus, and perceived importance’. They go on to argue that ‘complete and verbatim documented accounts of any clinical encounter is not desirable and potentially harmful’ to the aim of aiding cognition, communication, and care delivery that build relationships and support decision-making – we must aspire to make sure HIT is not just usable and safe but that it is of value to the primary goal of better care!

• References

• 1 Park HA. Are We Ready for the Fourth Industrial Revolution?. Yearb Med Inform 2016; 1-3.
• 2 Sankar PL, Parker LS. The Precision Medicine Initiative’s All of Us Research Program: an agenda for research on its ethical, legal, and social issues. Genet Medicine. 2016 Dec 8..
• 3 Coiera E, Ash J, Berg M. The Unintended Consequences of Health Information Technology Revisited. Yearb Med Inform 2016; 163-9.
• 4 Yardley L, Choudhury T, Patirck K, Michie S. Current Issues and Future Directions for Research Into Digital Behavior Change Interventions. Am J Prev Med 2016; 51 (05) 814-5.
• 5 Chapanis A, Safrin MA. Of misses and medicines. J Chronic Dis 1960; 12 (04) 403-8.
• 6 Weizenbaum J. Computer Power and Human Reason: From Judgment to Calculation. W.H. Freeman and Co; 1976
• 7 Perrow C. Normal Accidents: Living With High Risk Technologies. Basic Books; 1984
• 8 Burnum JF. The misinformation era: the fall of the medical record. Ann Intern Med 1989; Mar 15: 110 (06) 482-4.
• 9 Klatzky RL, Kober N, Mavor A. editors. Safe, Comfortable, Attractive, and Easy to Use: Improving the Usability of Home Medical Devices. National Research Council (US) Committee on Human Factors; 1996
• 10 Berg M. Medical work and the computer-based patient record: a sociological perspective. Methods Inf Med 1998; 37: 294-301.
• 11 Carayon P, Kianfar S, Li YQ, Xie AP, Alyousef B, Wooldridge A. A systematic review of mixed methods research on human factors and ergonomics in health care. Appl Ergon 2015; 51: 291-321.
• 12 Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc 2014; Nov-Dec; 21 (06) 1053-9.
• 13 Ammenwerth E. Evidence-based Health Informatics: How Do We Know What We Know?. Methods Inf Med 2015; 54 (04) 298-307.
• 14 Kannampallil TG, Schauer GF, Cohen T, Patel VL. Considering complexity in healthcare systems. J Biomed Inform 2011; Dec; 44 (06) 943-7.
• 15 Sawyer S, Jarrahi M. Sociotechnical approaches to the study of information systems. In Tucker A. editors: Computing Handbook: Information Systems and Information Technology. 3rd ed.. V2 CRC Boca Raton, FL: Chapman and Hall; 2014
• 16 Kaplan B. Evaluation of People, Social, and Organizational Issues - Sociotechnical Ethnographic Evaluation Evidence-Based Health Informatics. Ammenwerth E, Rigby M. editors IOS Press; 2016
• 17 Berg M, Langenberg C, Berg IVD, Kwakkernaat J. Considerations for sociotechnical design: experiences with an electronic patient record in a clinical context. Int J Med Inform 1998; 52: 243-51.
• 18 Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inform 1999; 55: 87-101.
• 19 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; Oct 19 (Suppl 3): i68-74.
• 20 Brown CL, Mulcaster HL, Triffitt KL, Sittig DF, Ash JS, Reygate K. et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017; Mar 1; 24 (02) 432-40.
• 21 Gephart S, Carrington JM, Finley BA. Systematic Review of Nurses’ Experiences With Unintended Consequences When Using the Electronic Health Record. Nurs Adm Q 2015; Oct-Dec; 39 (04) 345-56.
• 22 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda N. Electronic Health Record Usability: Analysis of the User Centered Design Processes of Eleven Electronic Health Record Vendors. J Am Med Inform Assoc 2015; November 11; 22 (06) 1179-82.
• 23 McCray AT, Glaser J, Koppel R, Langlotz CP, Silverstein J. Health IT vendors and the academic community: The 2014 ACMI debate. J Biomed Inform 2016; Apr; 60: 365-75.
• 24 Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and healthcare systems. J Am Med Inform Assoc 2015; Mar; 22 (02) 465-71.
• 25 Sinsky CA, Hess J, Karsh B, Keller JP, Koppel R. Comparative user experiences of health IT products: how user experiences would be reported and used. Institute of Medicine; 2012
• 26 Miliard M. Frustrations linger around electronic health records and user-centered design. Mar 10, 2016 http://www.healthcareitnews.com/news/frustrations-linger-around-electronic-health-records-and-user-centered-design.
• 27 McCann E. How satisfied are you with your EHR?. 2015 Satisfaction Survey results. Sept 18, 2015 http://www.healthcareitnews.com/ news/2015-ehr-satisfaction-survey.
• 28 Off ice of the National Coordinator. Safer guides will help optimize safety. Jan 14, 2014 https://www.healthit.gov/buzzblog/electron-ic-health-and-medical-records/safer-guides-optimize-safety/.
• 29 Shanafelt TD, Hasan O, Dyrbye LN, Sinsky C, Satele D, Sloan J. et al. Changes in Burnout and Satisfaction With Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. Mayo Clin Proc December 2015; 90 (12) 1600-13.
• 30 Wildenbos GA, Peute LW, Jaspers MWM. Impact of Patient-centered eHealth Applications on Patient Outcomes: A Review on the Mediating Influence of Human Factor Issues. Yearb Med Inform 2016; 113-9.
• 31 Baysari MT, Westbrook J. Mobile Applications for Patient-centered Care Coordination: A Review of Human Factors Methods Applied to their Design, Development, and Evaluation. Yearb Med Inform 2015; 10: 47-54.
• 32 Purkayastha S, Price A, Biswas R, Jai AUGanesh, Otero P. From Dyadic Ties to Information Infrastructures: Care-Coordination between Patients, Providers, Students and Researchers Contribution of the Health Informatics Education Working Group. Yearb Med Inform 2015; 10: 68-74.
• 33 Sawesi S, Rashrash M, Phalakornkule K, Carpenter JS, Jones JF. The Impact of Information Technology on Patient Engagement and Health Behavior Change: A Systematic Review of the Literature. JMIR Med Inform 2016; Jan 21; 04 (01) e1.
• 34 International Ergonomics Association (IEA). Defnition & Domains of Ergonomics. http://www.iea.cc/whats/index.html.
• 35 Carayon P. Human Factors and ergonomics in healthcare and patient safety. In; Carayon P. 2011
• 36 Patel VL, Kannampallil TG. Human factors and health information technology: current challenges and future directions. Yearb Med Inform 2014; 58-66.
• 37 Kushniruk A, Nohr C, Borycki E. Human Factors for more usable and safer information technology: where are we now and Where do we go from here?. Yearb Med Inform 2016; Nov 10: 120-5.
• 38 Beuscart-Zephir MC, Elkin P, Pelayo S, Beuscart R. The Human Factors Engineering Approach to biomedical Informatics projects: State of the Art, Results, Benefits and Challenges. Yearb Med Inform 2007; 109-27.
• 39 Beuscart-Zephir MC, Borycki E, Jaspers MWM, Pelayo S. Evolution of Human Factors research and Studies of Health Information Technologies: The role of patient Safety. Yearb Med Inform 2013; 67-77.
• 40 Ratwani RM, Fairbanks T, Savage E, Adams K, Wittie M, Boone E. et al. Mind the Gap: A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform 2016; Nov 16; 07 (04) 1069-87.
• 41 Ratwani RM, Benda NC, Hettinger A, Fairbanks R. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA 2015; Sep 8; 314 (10) 1070-1.
• 42 Babbott S, Manwell LB, Brown R, Montague E, Williams E, Schwartz M. et al. Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc 2014; Feb;21 (e1): e100-106.
• 43 Friedman A, Crosson JC, Howard J, Clark EC, Pellerano M, Karsh BT. et al. A typology of electronic health record workarounds in small-to-medium size primary care practices. J Am Med Inform Assoc 2014; 21: e78-e83.
• 44 Graber ML, Siegal D, Riah H, Johnston D, Kenyon K. Electronic health record-related events in medical malpractice claims. J Patient Saf. 2015 Nov 6
• 45 Ellsworth MA, Dziadzko M, O’Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc 2017; Jan; 24 (01) 218-26.
• 46 Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K. et al. Understanding and Promoting Effective Engagement with Digital Behavior Change Interventions. Am J Prev Med 2016; Nov 51 (05) 833-42.
• 47 Kayser L, Kushniruk A, Osborne RH, Norgaard O, Turner P. Enhancing the Effectiveness of Consumer-Focused Health Information Technology Systems Through eHealth Literacy: A Framework for Understanding Users’ Needs. JMIR Hum Factors 2015; May 20, 02 (01) e9.
• 48 Ammenwerth E, Rigby M. editors. Evidence-Based Health Informatics - Promoting Safety Efficiency Through Scientific Methods Ethical Policy. 2016. Vol 222 Studies in Health Technology & Informatics..
• 49 Singh H, Sittig DF. Measuring and improving patient safety through health information technology: The Health IT Safety Framework. BMJ Qual Saf 2016; Apr; 25 (04) 226-32.
• 50 Koppel R. The health information technology safety framework: building great structures on vast voids. BMJ Qual Saf 2016; Apr; 25 (04) 218-20.
• 51 Coiera E. A New Informatics Geography, Yearb Med Inform. 2016; 251-5.
• 52 HIMSS EHR Usability Task Force. “Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating.”. June 2009
• 53 Riskin L, Koppel R, Riskin D. Re-examining health IT policy: what will it take to derive value from our investment?. J Am Med Inform Assoc 2015; Mar; 22 (02) 459-64.
• 54 Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013; Jan 1; 32 (01) 63-8.
• 55 Medical Device Directive MDD93/42/EEC. http:// www.mdss.com/download/download_station.htm accessed December 21, 2016..
• 56 ONC Meaningful Use and Usability Testing. http:// www.healthcareusability.com/article/onc-meaningful-use-and-usabilitytesting accessed September 20, 2016..
• 57 Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM. et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; Jun 1; 20 (e1): e2-8.
• 58 Magrabi F, Baker M, Sinha I, Ong MS, Harrison S, Kidd MR. et al. Clinical safety of England’s national programme for IT: A retrospective analysis of all reported safety events 2005 to 2011. Int J Med Inform 2015; Mar; 84 (03) 198-206.
• 59 Magrabi F, Liaw ST, Arachi D, Runciman W, Coiera E, Kidd MR. Identifying patient safety problems associated with information technology in general practice: an analysis of incident reports. BMJ Qual Saf 2016; 25 (11) 870-80.
• 60 Marcilly R, Ammenwerth E, Roehrer E, Pelayo S, Vasseur F, Beuscart-Zéphir MC. Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System. Yearb Med Inform 2015; 55-67.
• 61 Schiff G, Amato M, Eguale T, Boehne JJ, Wright A, Koppel R. et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015; Apr; 24 (04) 264-71.
• 62 Castro GM, Buczkowski L, Hafner JM. The Contribution of Sociotechnical Factors to Health Information Technology-Related Sentinel Events. Jt Comm J Qual Patient Saf 2016; Feb; 42 (02) 70-6.
• 63 Brenner SK, Kaushal R, Grinspan Z, Joyce C, Kim I, Allard RJ. et al. Effects of health information technology on patient outcomes: a systematic review. J Am Med Inform Assoc 2016; Sep; 23 (05) 1016-36.
• 64 Guise V, Anderson J, Wiig S. Patient safety risks associated with telecare: a systematic review and narrative synthesis of the literature. BMC Health Serv Res 2014; Nov 25; 14: 588.
• 65 Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S. et al. Designing, Implementing, and Evaluating Mobile Health Technologies for Managing Chronic Conditions in Older Adults: A Scoping Review. JMIR mHealth & uHealth 2016; 04 (02) 164-81.
• 66 Zapata BC, Fernandez-Aleman JL, Idri A, Toval A. Empirical Studies on Usability of mHealth Apps: A Systematic Literature Review. J Med Syst 2015; 39 (02) 1.
• 67 Queiros A, Silva A, Alvarelhao J, Rocha NP, Teixeira A. Usability, accessibility and ambient-assisted living: a systematic literature review. Universal Access in the Information Society 2015; 14 (01) SI57-66.
• 68 Sheikh A, Sood HS, Bates DW. Leveraging health information technology to achieve the “triple aim” of healthcare reform. J Am Med Inform Assoc 2015; Jul; 22 (04) 849-56.
• 69 National Patient Safety Foundation. Free from Harm: Accelerating Patient Safety Improvement Fifteen Years after To Err Is Human. Boston, MA: http://www.npsf.org/ 2015
• 70 Zhang J, Muhammad W. editors. Better EHR: Usability, Workflow and Cognitive Support in Electronic Health Records. National Center for Cognitive Informatics & Decision Making in Healthcare. 2014 https://sbmi.uth.edu/nccd/ better-ehr/.
• 71 National Institute of Standards and Technology Health IT Usability. http://www.nist.gov/healthcare/usability (Accessed January 5, 2017)
• 72 Agency for Healthcare Research and Quality, Developing Evidence-Based, User-Centered Design and Implementation Guidelines to Improve Health Information Technology Usability (Maryland). (Accessed January 5, 2017) https:// healthit.ahrq.gov/ahrq-funded-projects/develop-ing-evidence-based-user-centered-design-and-implementation-guidelines.
• 73 Office of the National Coordinator. Safer guides will help optimize safety. 2014 https://www. healthit.gov/safer/safer-guides (Accessed January 5, 2017)
• 74 Kaipio J, Lääveri T, Hyppönen H, Vainiomäki S, Reponen J, Kushniruk A. et al. Usability problems do not heal by themselves: National survey on physicians’ experiences with EHRs in Finland. Int J Med Inform 2017; Jan 97: 266-81.
• 75 Jensen S, Rasmussen SL, Lyng KM. Use of clinical simulation for assessment in EHR-procurement: design of method. Stud Health Technol Inform 2013; 192: 576-80.
• 76 Marcilly R, Peute L, Beuscart-Zephir MC. From Usability Engineering to Evidence-based Usability in Health IT. Stud Health Technol Inform 2016; 222: 126-38.
• 77 Lyon AR, Lewis CC, Melvin A, Boyd M, Nicodimos S, Liu FF. et al. Health Information Technologies—Academic and Commercial Evaluation (HIT-ACE) methodology: description and application to clinical feedback systems. Implement Sci 2016; Sep 22; 11 (01) 128.
• 78 Australian Commission on Safety and Quality in Health Care (ACSQHC). National Guidelines for On-Screen Display of Clinical Medicines Information, Sydney, New South Wales, Commonwealth of Australia. 2016 http://www.safetyandquality.gov.au/ our-work/medication-safety/electronic-medication-management-systems/.
• 79 Kushniruk AW, Borycki EM. Low-cost rapid usability testing: Its application in both product development and system implementation. Stud Health Technol Inform 2017; 234: 195-200.
• 80 Martin SA, Sinsky CA. The map is not the territory: medical records and 21st century practice. Lancet 2016; Oct 22; 388 (10055): 2053-6.