Trends and Progress in Human Factors and Organizational Issues in 2016: Learning from Experience

Summary

Objective: To summarize significant research contributions on human factors and organizational issues in medical informatics published in 2016.

Methods: An extensive search using PubMed/Medline and Web of Science® was conducted to identify the scientific contributions published in 2016 that address human factors and organizational issues in medical informatics. The selection process comprised three steps: (i) 15 candidate best papers were first selected by the two section editors, (ii) external reviewers from internationally renowned research teams reviewed each candidate best paper, and (iii) the final selection of five best papers was conducted by the editorial board of the Yearbook.

Results: The five selected best papers present studies with rigorous methods, properly designed and described and are, therefore, efficiently reusable for other researches.

Conclusion: Human factors and ergonomics- based interventions must be tailored to the context, but meaningful ways must be simultaneously found to generate a stronger evidence base for research and to provide efficient, easy to implement, and useful methods.

Keywords

Human factors - ergonomics - health information technology - usability - organizational issues

Introduction

Even if Human Factors and Ergonomics (HFE) research in healthcare and patient safety is beginning to demonstrate some impact on important care processes and patient outcomes [[1]], challenges for HFE research and intervention in health care are still topical. Some of them have been highlighted in the last two years IMIA Yearbooks. The increasing complexity of the healthcare domain and Health Information Technology (HIT) systems calls for more comprehensive approaches [[2]]. But to achieve this, further efforts in HFE research are still needed to think and consider specific or additional methods [[3]]. As stressed by Turner, et al., [[4]] in the survey paper of the Human Factors and Organizational Issues (HFOI) section of this IMIA Yearbook, HFE research “is not a destination per se, but rather a journey”. HFE-based interventions must be tailored to the context, but meaningful ways must be simultaneously found to generate a stronger evidence base for research.

In 2017, the selection of papers in HFOI intends to illuminate the current progress of HFE research in terms of rigorous and properly described methods, therefore efficiently reusable for other researches.

Paper Selection Method

Two electronic databases were searched, PubMed/Medline and Web of Science^®. Searches were performed in November and December 2016 to identify peer-reviewed journal articles published in 2016, in the English language, related to HFE research in medical informatics. In addition to the search through electronic databases, manual searches of key themes were performed in major biomedical journals (e.g. Journal of the American Medical Informatics Association, Methods of information in medicine, Journal of Medical Internet Research, etc.).

Keywords used included both free-text and coded keywords. Free-text keywords were listed as regards to the questions addressed by the section. Corresponding relevant MeSH terms were identified. Pubmed was queried to test keywords in an iterative process. Consequently, two queries were built: one based on MeSH terms used as major topic in the Pubmed/ Medline database, the second one based on free-text keywords searched in title or abstracts through Pubmed/Medline and Web of Science^® databases.

One of the two section editors performed the searches, which yielded a total of 818 references. Then, the two section editors undertook independently the initial screening of titles and abstracts to identify papers relevant to the field of interest. Both section editors classified the papers into three categories: accepted, rejected, or pending. They then reviewed in detail the accepted and pending full-text articles to finally reach a consensual list of 15 candidate best papers. Papers were considered according to their originality, innovativeness, scientific and/ or practical impact, and scientific quality.

In accordance with the IMIA Yearbook selection process [[5]], the 15 candidate best papers were evaluated by the two section editors and by additional external reviewers (at least four reviewers per paper). Five papers were finally selected as best papers ([Table 1]). A content summary of the selected best papers can be found in the appendix of this synopsis.

Table 1

Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2017 in the section ‘Human Factors and Organizational Issues’. The articles are listed in alphabetical order of the first author’s surname.

Section Human Factors and Organizational Issues

Castro GM, Buczkowski L, Hafner JM. The Contribution of Sociotechnical Factors to Health Information Technology-Related Sentinel Events. Jt Comm J Qual Patient Saf 2016 Feb;42(2):70-6. Horsky J, Ramelson HZ. Development of a cognitive framework of patient record summary review in the formative phase of user-centered design. J Biomed Inform 2016 Dec;64:147-57. Kobayashi L, Gosbee JW, Merck DL. Development and Application of a Clinical Microsystem Simulation Methodology for Human Factors-Based Research of Alarm Fatigue. HERD 2016 Jan 1:1937586716673829. Percival J, McGregor C. An Evaluation of Understandability of Patient Journey Models in Mental Health. JMIR Hum Factors 2016 Jul 28;3(2):e20. Schnittker R, Schmettow M, Verhoeven F, Schraagen JM. Combining situated Cognitive Engineering with a novel testing method in a case study comparing two infusion pump interfaces. Appl Ergon 2016 Jul;55:16-26.

Conclusions and Outlook

All the five selected best papers are convincing by their rigor and their understandable and useful way of describing their methods and the results of their research.

Schnittker, et al., [[6]] wrote a good methodological paper with good explanations of a novel usability testing technique in conjunction with a situated Cognitive Engineering method for designing interfaces. They addressed shortcomings of previous methods - i.e. lack of process-tracing techniques, lack of repeated measurements to reveal learning effects, and lack of combination of subjective and objective measures - in combining qualitative and quantitative analyses with objective and subjective measures in a usability validation study. This kind of method provides a good picture of the user-technology interaction and maximize reproducibility.

Kobayashi, et al., [[7]] provided a good example of a rigorous evaluation with a well-established method. They developed a clinical simulation methodology for human-based assessment of alarm fatigue. In addition to generating constructive feedback for continued innovation, the method generates realistic provider responses to true and false monitor alarms and could streamline research efforts investigating different approaches to patient telemetry monitoring and provider notification.

The paper of Horsky and Ramelson [[8]] presents the initial stages of a User-Centered Design (UCD) process dedicated to the design of an Electronic Health Record (EHR) module intended to help clinicians to efficiently complete a summary review of an electronic patient record before an ambulatory visit. The study is focused on the description of the collection of data and its interpretation to gain understanding of the reasoning and review process. The paper gives an unusually clear and easy to follow description of a UCD process and describes how it has impacted on the results. There is a need for more of these types of studies that show what must be done during the first steps of a UCD which are crucial for an efficient UCD process but tricky.

Castro, et al., [[9]] explored how HIT may contribute to reduce adverse events that result in death or severe harm to the patient using a socio-technical approach based on well-known HFE models. They performed a root cause analysis of more than 3000 de-identified events reported to the Joint Commission. This research was integrated in the selection of the IMIA Yearbook as it provides insights for this type of method which becomes essential nowadays in the field. Improved identification of HIT-related contributing factors in the context of socio-technical dimensions may help the different stakeholders proactively identify vulnerabilities and hazards, ultimately reducing the risk and harm to patients.

Percival and McGregor [[10]] spotlighted the importance of the patient journey modeling. The authors performed a good job in arguing the challenges of integrating information systems with existing clinical practice. They rigorously compared the understandability of different patient journey models. In particular, the authors demonstrated the value of a relatively new patient journey modeling technique called the Patient Journey Modeling Architecture (PaJMa) when compared to traditional manufacturing-based process modeling tools.

The other candidate best papers are in the same line with innovative and/or effective HFE tools to support the identification, and the prevention and/or mitigation of HFE concerns during the design, evaluation, and implementation of HIT.

Lee, et al., [[11]] drew on qualitative data collected at two case study sites to focus on how the procurement process can impact the deployment and the adoption of HIT. The authors explained that a joint procurement model may paradoxically make it difficult to fulfill the goals. Blaar, et al., [[12]] and Noyes, et al., [[13]] analyzed workflows respectively in the Operating Room and in regional care centers that manage cancer patients. Both studies draw the lessons for evidence-based sustainable solutions and discuss areas for research of teamwork. Three of the candidate best papers [[14]–[16]] carried out rigorous evaluations to examine the usability and/or the efficacy of different HIT solutions. These studies highlight the need for multiple and mixed methods for more relevant and impactful HFE research. One of them [[14]] presents a well-described 3-arms randomized controlled trial to compare two different versions of a web-based physical activity intervention with a control group.

Campbell. et al.,[[17]] provided useful worldwide advices to those involved in the evaluations of new technologies with limited evidence. A team of the United Kingdom focused on HIT implementation processes [[18]–[19]]. They respectively presented a typology for workarounds and a taxonomy of causes of implementation delays to help clarify important aspects of implementation and facilitate the process. Finally, Mitchell, et al., [[20]] discussed five key challenges to explain why incident reporting has not reached its potential yet. They explored how the processes and the systems of incident reporting can be optimized to increase the likelihood of safer patient care.

Summary of Best Papers Selected for the 2017 Edition of the IMIA Yearbook, Section HFOI

Castro G, Buczkowski L, Hafner J The contribution of socio-technical factors to health information technology-related sentinel events Jt Comm J Qual Patient Saf 2016; 42(2):70-6

An understanding of how health information technology (HIT) can contribute to sentinel events is necessary to learn how to safely implement and use HIT. Castro, Buczkowski, and Hafner conducted an analysis to explore how HIT may contribute to adverse events that result in death or severe harm to the patient. They showed that HIT-related events are primarily associated with the socio-technical dimensions of human-computer interface, workflow and communication, and clinical content. The authors advocate that improved identification of HIT-related contributing factors in the context of the socio-technical dimensions may help software developers, device manufacturers, and end users in healthcare organizations proactively identify vulnerabilities and hazards, ultimately reducing the risk of harm to patients.

Horsky J, Ramelson H Development of a cognitive framework of patient record summary review in the formative phase of user-centered design J Biomed Inform 2016;64:147-57

A User-Centered Design (UCD) process increases the usability of products in order to insure human performance and patient safety. However, this design process is challenging and tricky. In this report, Horsky and Ramelson describe the initial stage of a UCD process in which foundational design concepts are formulated. They designed and developed a functional prototype of an ambulatory electronic health record interface that allows clinicians to briefly review patient data prior to the office visit. Cognitively-based studies were performed and results are used to develop a cognitive framework that subsequently guides the design of a prototype.

Kobayashi L, Gosbee J, Merck D Development and application of a clinical microsystem simulation methodology for human factors-based research of alarm fatigue HERD 2017 Jul;10(4):91–104

Patient monitoring systems with telemetry features are widespread. However, problems with the design, implementation, and real-world use of these systems result in alarm fatigue. Therefore, clinical alarms may be ignored or not noticed causing potential harmful situations to patient safety. Kobayashi, Gosbee, and Merck developed a clinical micro-system simulation methodology for alarm fatigue research with a human factors engineering assessment framework. This novel methodology allows not only the assessment of systems but also supports experimental research purposes.

Percival J, McGregor C An evaluation of understandability of patient journey models in mental health JMIR Hum Factors 2016 Jul 28;3(2):e20

Little awareness exists about the challenges of integrating information systems with clinical practice. Recently some work has focused on process modeling through the lens of the patient, using patient journey modeling techniques. These models can help understand the potential consequences of the changes in processes and information flows due to HIT implementation. Percival and McGregor demonstrated the value of a relatively new patient journey modeling technique called the Patient Journey Modeling Architecture when compared with traditional manufacturing-based process modeling tools.

Schnittker R, Schmettow M, Verhoeven F, Schraagen JMC Combining situated cognitive engineering with a novel testing method in a case study comparing two infusion pump interfaces Appl Ergon 2016;55:16–26

Infusion pumps contribute to patient care but several adverse drug events have been associated to their use. Many of those use-related hazards were related to user-interface design deficiencies. Design solutions using human factors engineering have proven to be effective to enhance positive performance outcomes. In this regard, Schnittker, et al., validated the usability of a new infusion pump interface designed with a situated Cognitive Engineering approach by comparing it to a reference interface using a novel testing method. The observed reduction of errors, normative path deviations, task completion times, and keystrokes demonstrated that this method addresses various shortcomings of previous testing methods.

Acknowledgement

We would like to thank Martina Hutter and Adrien Ugon for their support and the reviewers for their participation in the selection process of the IMIA Yearbook.

• References

• 1 Carayon P, Kianfar S, Li Y, Xie A, Alyousef B, Wooldridge A. A systematic review of mixed methods research on human factors and ergonomics in health care. Appl Ergon 2015; Nov; 51: 291-321.
• 2 Pelayo S, Santos R. Human Factors and Organizational Issues in 2015: The Increasing Complexity of the Healthcare Domain Calls for More Comprehensive Approaches. Yearb Med Inform 2016; Nov 10; (01) 126-9.
• 3 Pelayo S, Ong M. Human Factors and Ergonomics in the Design of Health Information Technology: Trends and Progress in 2014. Yearb Med Inform 2015; Aug 13; 10 (01) 75-8.
• 4 Turner P, Kushniruk A, Nohr C. Are We There Yet? Human Factors Knowledge and Health Information Technology – the challenges of implementation and impact. Yearb Med Inform 2017; 84-91.
• 5 Lamy J-B, Séroussi B, Griffon N, Kerdelhué G, Jaulent MC, Bouaud J. Toward a formalization of the process to select IMIA Yearbook best papers. Methods Inf Med 2015; 54: 135-44.
• 6 Schnittker R, Schmettow M, Verhoeven F, Schraagen JM. Combining situated Cognitive Engineering with a novel testing method in a case study comparing two infusion pump interfaces. Appl Ergon 2016; Jul; 55: 16-26.
• 7 Kobayashi L, Gosbee JW, Merck DL. Development and Application of a Clinical Microsystem Simulation Methodology for Human Factors-Based Research of Alarm Fatigue. HERD 2017; Jul; 10 (04) 91-104.
• 8 Horsky J, Ramelson HZ. Development of a cognitive framework of patient record summary review in the formative phase of user-centered design. J Biomed Inform 2016; Dec; 64: 147-57.
• 9 Castro GM, Buczkowski L, Hafner JM. The Contribution of Sociotechnical Factors to Health Information Technology-Related Sentinel Events. Jt Comm J Qual Patient Saf 2016; Feb; 42 (02) 70-6.
• 10 Percival J, McGregor C. An Evaluation of Understandability of Patient Journey Models in Mental Health. JMIR Hum Factors 2016; Jul 28; 03 (02) e20.
• 11 Lee L, Williams R, Sheikh A. How does joint procurement affect the design, customisation and usability of a hospital ePrescribing system?. Health Informatics J 2016; Dec; 22 (04) 828-38.
• 12 Blaar M, Janß A, Dell’Anna J, Höllig A, Radermacher K, Clusmann H. Bottlenecks and needs in human-human and human-machine interaction - a view from and into the neurosurgical OR. Biomed Tech (Berl) 2016; Apr 1; 61 (02) 135-46.
• 13 Noyes K, Monson JR, Rizvi I, Savastano A, Green JS, Sevdalis N. Regional Multiteam Systems in Cancer Care Delivery. J Oncol Pract 2016; Nov; 12 (11) 1059-66.
• 14 Gomez SQuiñonez, Walthouwer MJ, Schulz DN, de Vries H. mHealth or eHealth? Efficacy, Use, and Appreciation of a Web-Based Computer-Tailored Physical Activity Intervention for Dutch Adults: A Randomized Controlled Trial. J Med Internet Res 2016; Nov 9; 18 (11) e278.
• 15 Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of Commercially Available Wearable Activity Trackers Among Adults Aged Over 50 and With Chronic Illness: A Mixed-Methods Evaluation. JMIR Mhealth Uhealth 2016; Jan 27; 04 (01) e7.
• 16 Shah MH, Peikari HR. Electronic Prescribing Usability: Reduction of Mental Workload and Prescribing Errors Among Community Physicians. Telemed J E Health 2016; Jan; 22 (01) 36-44.
• 17 Campbell B, Knox P. Promise and Plausibility: Health Technology Adoption Decisions with Limited Evidence. Int J Technol Assess Health Care 2016; Jan; 32 (03) 122-5.
• 18 Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A. Workarounds to hospital electronic prescribing systems: a qualitative study in English hospitals. BMJ Qual Saf. 2016 Apr 29.
• 19 Mozaffar H, Cresswell KM, Lee L, Williams R, Sheikh A. NIHR ePrescribing Programme Team. Taxonomy of delays in the implementation of hospital computerized physician order entry and clinical decision support systems for prescribing: a longitudinal qualitative study. BMC Med Inform Decis Mak 2016; Feb 24; 16: 25.
• 20 Mitchell I, Schuster A, Smith K, Pronovost P, Wu A. Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after ‘To Err is Human’. BMJ Qual Saf 2016; Feb; 25 (02) 92-9.