Human Factors and Sociotechnical Issues

Summary

Objective: To summarize significant research contributions on human factors (HF) and organizational issues in medical informatics published in 2018.

Methods: An extensive search using PubMed/Medline and Web of Science^® was conducted to identify the scientific contributions published in 2018 that address human factors and organizational issues in medical informatics. The selection process comprised three steps: (i) 15 candidate best papers were first selected by the two section editors, (ii) external reviewers from internationally renowned research teams reviewed each candidate best paper, and (iii) the final selection of four best papers was conducted by the editorial board of the Yearbook.

Results: The four selected best papers are excellent contributions to the HF literature: they show the added value of HF studies by providing nice illustrated and rigorous interventions.

Conclusion: HF interventions are known to have great potential to contribute to efficient HIT design, but the interventions still face challenges in successfully demonstrating their value to the main stakeholders of the healthcare domain. There is a need to strengthen the demand for high-quality HF studies by increasing awareness among powerful stakeholders of the value of high-quality HF studies.

Keywords

Human factors - ergonomics - health information technology - usability - organizational issues

Introduction

There is increasing evidence of the benefits of human factors (HF)-based health IT design for patient safety [1]. The selection here highlights a growing number of high-quality studies, based on solid well-described methodologies. This year, we saw the same trend as last year for the Human Factors and Organizational Issues section with an increase of the number of papers meeting the International Medical Informatics Association (IMIA) Yearbook scientific quality requirements [2]. However, we continue to see poorly designed health IT and negative impact on patients [3] and clinicians [4]. We know HF interventions have great potential to contribute to efficient HIT design, but they still face challenges in successfully demonstrating their value to the main stakeholders in the healthcare domain. Therefore, there is a need to strengthen the demand for high-quality HF studies by increasing awareness among powerful stakeholders of the value of high-quality HF studies by communicating with stakeholders, building partnerships, and educating stakeholders [5]. The gaps between stakeholders, particularly vendors, researchers, clinicians, healthcare organization administrations, and purchasers, need also to be reduced. Currently the results of research and the need for systematic HF studies and improvements are not necessarily recognized among all parties, and this has resulted in potential improvements (even those which may compromise safety) not being incorporated into products and/or system deployments. Better pathways for HF studies need to be incorporated into the process of development and purchasing to help diminish these gaps and create safer more usable health technologies. The four selected best papers for the section strongly contribute to the dissemination of the added value of HF studies by providing nicely illustrated and rigorous interventions.

Paper Selection Method

Two electronic databases were searched, PubMed/Medline and Web of Science^®. Searches were performed in October and December 2018 to identify peer-reviewed journal articles published in 2018 in the English language and related to HF research in medical informatics. In addition to the search of electronic databases, manual searches of key themes were performed in major biomedical journals (e.g., Journal of the American Medical Informatics Association, Methods of Information in Medicine, Journal of Medical Internet Research, etc.).

Keywords used included both free-text and coded keywords. Free-text keywords were listed in regards to the questions addressed by the section. Corresponding relevant MeSH terms were identified. Pubmed was queried to test keywords in an iterative process. Consequently, two queries were built: one based on MeSH terms used as major topics in the Pubmed/Medline database, and the second one based on freetext keywords searched in title or abstracts through Pubmed/Medline and Web of Science^® databases.

One of the two section editors performed the searches. Database searches yielded a total of 1,545 references. The two section editors independently carried out the initial screening of titles and abstracts to identify papers relevant to the field of interest. Both section editors classified the papers into three categories: accepted, rejected, or pending. They then reviewed in detail the accepted and pending full-text articles to finally reach a consensus list of 15 candidate best papers. Papers were considered according to their originality, innovativeness, scientific and/ or practical impact, and scientific quality.

In accordance with the IMIA Yearbook selection process, the 15 candidate best papers were evaluated by the two section editors and by additional external reviewers (at least four reviewers per paper). Four papers were finally selected as best papers ([Table 1]). A content summary of the selected best papers can be found in the appendix of this synopsis.

Table 1 Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2019 in the section ‘Human Factors and Organizational Issues’. The articles are listed in alphabetical order of the first author’s surname

Section Human Factors and Organizational Issues
▪ Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A, Katsulis Z, Benneyan J, Gershanik E, Bates DW, Collins SA. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform 2018 Apr;9(2):302-12.
▪ Miller A, Koola JD, Matheny ME, Ducom JH, Slagle JM, Groessl EJ, Minter FF, Garvin JH, Weinger MB, Ho SB. Application of contextual design methods to inform targeted clinical decision support interventions in sub-specialty care environments. Int J Med Inform 2018 Sep;11 7:55-65.
▪ Tamblyn R, Winslade N, Lee TC, Motulsky A, Meguerditchian A, Bustillo M, Elsayed S, Buckeridge DL, Couture I, Qian CJ, Moraga T, Huang A. Improving patient safety and efficiency of medication reconciliation through the development and adoption of a computer-assisted tool with automated electronic integration of population-based community drug data: the RightRx project. J Am Med Inform Assoc 2018 May 1 ;25(5):482-95.
▪ Tscholl DW, Handschin L, Neubauer P, Weiss M, Seifert B, Spahn DR, Noethiger CB. Using an animated patient avatar to improve perception of vital sign information by anesthesia professionals. Br J Anaesth 2018 Sep;1 21(3):662-71.

Conclusions and Outlook

Noteworthy papers in 2018 contribute to the dissemination of the HF approach by providing nice examples of robust interventions. Tamblyn et al. [6] present a study aiming at developing and evaluating a web-based software application to semi-automate the medication reconciliation process by prepopulating the community and hospital medication lists using a regional clinical data repository and the local hospital pharmacy system. They provide a nice presentation of a cluster-randomized trial with positive effects along with qualitative feedback based on an interesting sociotechnical framework, on which different levels of lessons learned are structured (at the macro, meso, and micro-levels).

Couture et al. [7] focus on a mobile application for patients and care partners during hospitalization for the reporting of safety-related events/ problems. The paper describes a user-centered design approach based on an extensive and iterative process with different kinds of evaluations and diverse user groups.

Miller et al [8] describe the analyses of clinical work in outpatient and inpatient settings as a basis for developing guidelines for optimizing computer decision support design. The paper is an excellent contribution to the HF literature since the authors perform a well-done description of how communication and interactions occur across specialties.

Tscholl et al. [9] raise the question of the added value of a patient avatar to improve the perception of vital signs information in the operating room. The study provides some first empirical evidence that an animated avatar offers the opportunity to transmit vital signs significantly more quickly with a reduced cognitive effort compared to traditional supports. The paper addresses an important topic relevant to medical informatics looking for original and innovative ways to provide information to professionals. We actually observed a significant trend of virtual reality (VR) emerging developments and we expect such papers to become more prominent in years to come.

Although not selected as best papers, the remaining candidate best papers are just as interesting. Some of them bring a real added value for the HF community by disseminating rigorous methods to be adopted in a sufficiently detailed way to help guiding other interventions. Cochon et al. [10] present a study in which five information sources about causes of errors in radiology systems were examined. Each was categorized into imaging chain steps impacted by reported events, and the sociotechnical factors involved as per the Systems Engineering for Patient Safety (SEIPS) framework. The study presents a nice example of the use of existing sources and a thoughtful tracking of processes to illuminate process errors and understand their consequences. Riis et al. [11] describe a detailed protocol for a randomized controlled trial comparing a standard reference website with guideline-driven information. Apart from the website evaluation, the protocol includes secondary outcomes related to pain, functionality disability, quality of life, absences, and contacts to care, providing a comprehensive view of how the different sites may affect patient outcomes overall. Russ et al. [12] carried out a study to evaluate the usability of a collaborative medication reconciliation tool and assess healthcare professionals and patients’ detection of medication discrepancies. They integrate artificial safety probes into standard usability methods and demonstrate that embedded safety probes can enhance standard usability methods by measuring additional, clinically focused, usability outcomes. Savoy et al. [13] implemented a cognitive systems engineering (CSE) design approach to develop a template that supports the cognitive needs of referring clinicians and improves referral communications. This study reinforces the significance of applying a CSE design approach to inform the design of health information technology. In addition, knowledge elicitation methods enabled identification of physicians’ cognitive requirements and challenges when completing electronic medical consultation orders. Marcilly et al. [14] developed a comprehensive and structured list of evidence-based usability design principles for medication alerting systems to aid designers and prevent the release of flawed systems. They provide a high-impact tool for designers and others.

Other papers provide additional useful examples of well-conducted HF interventions or valuable tools to support the development of HIT systems. Tavares et al. [15] focuses on the factors that drive individuals to adopt EHR portals combining three different theories, namely, extended unified theory of acceptance and use of technology, health belief model, and the diffusion of innovation. All three theories provide relevant contributions for the understanding of EHR portals. Giunti [16] is interested in factors for the design of mHealth solutions for chronic patients using negotiations between medical knowledge, behavioral change models, and gamification. Dijkstra et al. [17] describe a rigorous 6-phase process of development of a digital home care tool, eHOME, for handling drug-related problems in home care. Their study serves as an example of how to convert paper-based tools into usable and useful multidisciplinary digital tools. Schobel et al. [18] describe a comprehensive study of the QuestionSys framework, which is designed to allow researchers and clinicians to develop smart mobile apps without programming skills. As might be expected, novices initially showed greater errors than experts but learned significantly, and after seven days, they were able to achieve results similar to those of experts. The paper is important for showing the feasibility of end-user programming tools for research and patient care, and addressing issues in the current way mobile apps are created. Mishra et al. [19] conducted a 12-month crossover study and address an important issue about how the use of scribes to deal with electronic health records is on the rise, and how substantial decreases are happening in documentation burden. This study presents implications for possible methods to prevent physician burnout, but the authors call attention to several areas of further research needed, including work on patient privacy and willingness to discuss sensitive topics. Baysari et al. [20] describe a comprehensive study of a CPOE deployment at multiple time points, showing interesting changes in views over time, and when the emergence of workarounds, realization of safety benefits and problems, and new types of errors took place. The study informs us about issues that need to be considered over time in implementations and suggests adaptive implementation and support. This study is a good example of how implementations should be studied comprehensively over time, in order to anticipate and resolve problems as phases of acceptance and experience progress.

Acknowledgement

We would like to acknowledge the support of Adrien Ugon, Martina Hutter, Kate Fultz Hollis, Lina Soualmia, Brigitte Seroussi, and the whole Yearbook editorial team as well as the reviewers for their contribution to the selection process of the Human Factor and Organizational Issues section for the IMIA Yearbook 2019.

• References

• 1 Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR. , et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014; 21: e287-296
• 2 Pelayo S, Kaipio J. Findings from the 2018 Yearbook Section on Human Factors and Organizational Issues. Yearb Med Inform 2018; 27: 79-82
• 3 Ratwani RM, Savage E, Will A, Fong A, Karavite D, Muthu N. , et al. Identifying electronic health record usability and safety challenges in pediatric settings. Health Aff (Millwood) 2018; 37 (11) 1752-9
• 4 Gardner RL, Cooper E, Haskell J, Poplau S, Kroth PJ, Linzer M. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26: 106-14
• 5 Dul J, Bruder R, Buckle P, Carayon P, Falzon P, Marras WS. , et al. A strategy for human factors/ ergonomics: developing the discipline and profession. Ergonomics 2012; 55 (04) 377-95
• 6 Tamblyn R, Winslade N, Lee TC, Motulsky AS, Meguerditchian A, Bustillo M. , et al. Improving patient safety and efficiency of medication reconciliation through the development and adoption of a computer-assisted tool with automated electronic integration of population-based community drug data: the RightRx project. J Am Med Inform Assoc 2018; 25: 482-95
• 7 Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A. , et al. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform 2018; 9 (02) 302-312
• 8 Miller A, Koola JD, Matheny ME, Ducom JH, Slagle JM, Groessl EJ. , et al. Application of contextual design methods to inform targeted clinical decision support interventions in sub-specialty care environments. Int J Med Inform 2018; 117: 55-65
• 9 Tscholl DW, Handschin L, Neubauer P, Weiss M, Seifert B, Spahn DR. , et al. Using an animated patient avatar to improve perception of vital sign information by anaesthesia professionals. Br J Anaesth 2018; 121 (03) 662 :71
• 10 Cochon L, Lacson R, Wang A, Kapoor N, Ip IK, Desai S. , et al. Assessing information sources to elucidate diagnostic process errors in radiologic imaging – a human factors framework. J Am Med Inform Assoc 2018; 25 (11) 1507-15
• 11 Riis A, Hartvigsen J, Rathleff MS, Afzali T, Jensen MB. Comparing satisfaction with a participatory driven web-application and a standard website for patients with low back pain: a study protocol for a randomised controlled trial (part of the ADVIN Back Trial). Trials 2018; 19 (01) 399
• 12 Russ AL, Jahn MA, Patel H, Porter BW, Nguyen KA, Zillich AJ. , et al. Usability evaluation of a medication reconciliation tool: Embedding safety probes to assess users’ detection of medication discrepancies. J Biomed Inform 2018; 82: 178-86
• 13 Savoy A, Militello LG, Patel H, Flanagan ME, Russ AL, Daggy JK. , et al. A cognitive systems engineering design approach to improve the usability of electronic order forms for medical consultation. J Biomed Inform 2018; 85: 138-48
• 14 Marcilly R, Ammenwerth E, Roehrer E, Nies J, Beuscart-Zephir MC. Evidence-based usability design principles for medication alerting systems. BMC Med Inform Decis Mak 2018; 18 (01) 69
• 15 Tavares J, Oliveira T. New integrated model approach to understand the factors that drive electronic health record portal adoption: cross-sectional national survey. J Med Internet Res 2018; 20 (11) e11032
• 16 Giunti G. 3MD for Chronic Conditions, a Model for Motivational mHealth Design: Embedded Case Study. JMIR Serious Games 2018; 6 (03) e11631
• 17 Dijkstra NE, Sino CGM, Heerdink ER, Schuurmans MJ. Development of eHOME, a Mobile Instrument for Reporting, Monitoring, and Consulting Drug-Related Problems in Home Care: Human-Centered Design Study. JMIR Hum Factors 2018; 5 (01) e10
• 18 Schobel J, Pryss R, Probst T, Schlee W, Schickler M, Reichert M. Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study. JMIR Mhealth Uhealth 2018; 6 (06) e148
• 19 Mishra P, Kiang JC, Grant RW. Association of Medical Scribes in Primary Care With Physician Workflow and Patient Experience. JAMA Intern Med 2018; 178 (11) 1467-72
• 20 Baysari MT, Hardie RA, Lake R, Richardson L, McCullagh C, Gardo A. , et al. Longitudinal study of user experiences of a CPOE system in a pediatric hospital. Int J Med Inform 2018; 109: 5-14