Development and Validation of SafeHIT: An Instrument to Assess the Self-Reported Safe Use of Health Information Technology

Abstract

Background Implementing health information technology (HIT) may cause unintended consequences and safety risks when incorrectly designed and used. Yet, the tools to assess self-reported safe use of HIT are not well established.

Objective This study aims to develop and validate SafeHIT, an instrument to assess self-reported safe use of HIT among health care practitioners.

Methods Systematic literature review and a semistructured interview with 31 experts were adopted to generate SafeHIT instrument items. In total, 450 physicians from various departments at three Malaysian public hospitals participated in the questionnaire survey to validate SafeHIT. Exploratory factor analysis and confirmatory factor analysis (CFA) were undertaken to explore the items that best represent a specific construct and to confirm the reliability and validity of the SafeHIT, respectively.

Results The final SafeHIT consisted of 14 constructs and 58 items in total. The result of the CFA confirmed that all constructs demonstrated adequate convergent and discriminant validity.

Conclusion A reliable and valid theoretically underpinned measure of determinants of safe HIT use behavior has been developed. Understanding external factors that influence safe HIT use is useful for developing targeted interventions that favor the quality and safety of health care.

Keywords

health information technology - safety - adoption - questionnaire survey - quantitative instrument

Protection of Human and Animal Subjects

The study was performed in compliance with the Ethical Principles for Medical Research Involving Human Subjects and received ethics approval from Malaysia's Ministry of Health Medical Review and Ethics Committee.

Publication History

Received: 06 December 2023

Accepted: 05 June 2023

Article published online:
30 August 2023

• References

• 1 Feldman SS, Buchalter S, Hayes LW. Health information technology in healthcare quality and patient safety: literature review. JMIR Med Inform 2018; 6 (02) e10264
• 2 Kruse CS, Beane A. Health information technology continues to show positive effect on medical outcomes: systematic review. J Med Internet Res 2018; 20 (02) e41
• 3 Wang T, Wang Y, McLeod A. Do health information technology investments impact hospital financial performance and productivity?. Int J Account Inf Syst 2017; 2018 (28) 1-13
• 4 Magrabi F, Baker M, Sinha I. et al. Clinical safety of England's national programme for IT: a retrospective analysis of all reported safety events 2005 to 2011. Int J Med Inform 2015; 84 (03) 198-206
• 5 Palojoki S, Mäkelä M, Lehtonen L, Saranto K. An analysis of electronic health record-related patient safety incidents. Health Informatics J 2017; 23 (02) 134-145
• 6 Martin G, Ghafur S, Cingolani I. et al. The effects and preventability of 2627 patient safety incidents related to health information technology failures: a retrospective analysis of 10 years of incident reporting in England and Wales. Lancet Digit Health 2019; 1 (03) e127-e135
• 7 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 246-250
• 8 Carayon P, Hoonakker P. Human factors and usability for health information technology: old and new challenges. Yearb Med Inform 2019; 28 (01) 71-77
• 9 Singh H, Sittig DF. Measuring and improving patient safety through health information technology: The Health IT Safety Framework. BMJ Qual Saf 2016; 25 (04) 1-7
• 10 Middleton B, Bloomrosen M, Dente MA. et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 ( e1) e2-e8
• 11 Committee on Patient Safety and Health Information Technology; Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington (DC): National Academies Press (US); 2011
• 12 Carayon P, Wetterneck TB, Rivera-Rodriguez AJ. et al. Human factors systems approach to healthcare quality and patient safety. Appl Ergon 2014; 45 (01) 14-25
• 13 Petter S, DeLone W, McLean E. Measuring information systems success: models, dimensions, measures, and interrelationships. Eur J Inf Syst 2008; 17 (03) 236-263
• 14 Sorra JS, Dyer N. Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv Res 2010; 10 (01) 199
• 15 Quality A for HR and Hospital Survey on Patient Safety Culture. Accessed November 01, 2022 at: https://www.ahrq.gov/sops/surveys/hospital/index.html
• 16 Secginli S, Erdogan S, Monsen KA. Attitudes of health professionals towards electronic health records in primary health care settings: a questionnaire survey. Inform Health Soc Care 2014; 39 (01) 15-32
• 17 Sittig DF, Ash JS, Singh H. The SAFER guides: empowering organizations to improve the safety and effectiveness of electronic health records. Am J Manag Care 2014; 20 (05) 418-423
• 18 Hinkin TR. A review of scale development practices in the study of organization. J Manage 1995; 21 (05) 967-988
• 19 Salahuddin L, Ismail Z. Classification of antecedents towards safety use of health information technology: a systematic review. Int J Med Inform 2015; 84 (11) 877-891
• 20 Salahuddin L, Ismail Z, Hashim UR, Raja Ikram RR, Ismail NH, Naim Mohayat MH. Sociotechnical factors influencing unsafe use of hospital information systems: a qualitative study in Malaysian government hospitals. Health Informatics J 2019; 25 (04) 1358-1372
• 21 Salahuddin L, Ismail Z, Hashim UR. et al. Healthcare practitioner behaviours that influence unsafe use of hospital information systems. Health Informatics J 2020; 26 (01) 420-434
• 22 Bhattacherjee A. Social Science Research: Principles, Methods, and Practices. 2nd ed. Scotts Valley, CA: Createspace Independent Pub; 2012
• 23 George D, Mallery P. SPSS for Windows Step by Step: A Simple Guide and Reference. 4th ed. Allyn & Bacon: The University of Michigan; 2003
• 24 Zikmund WG. Business Research Methods. 7th ed. Thomson South-Western; Indiana University; 2003
• 25 Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999; 4 (03) 272-299
• 26 Hair JF, Anderson R, Tatham R, Black W. Multivariate Data Analysis. 4th ed. Englewood Cliffs: Prentice-Hall Inc.; 1995
• 27 Williams B, Onsman A, Brown T. Exploratory factor analysis: a five-step guide for novices. J Emerg Prim Health Care 2010; 8 (03) 1-13
• 28 Hair J, Black W, Babin B, Anderson R, Tatham R. Multivariate Data Analysis. 7th ed. Prentice Hall; 2010
• 29 Hair JF, Hult GTM, Ringle CM, Sarstedt M. A primer on partial least squares structural equation modeling (PLS-SEM). Sage 2014
• 30 Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002; 287 (02) 226-235
• 31 Palojoki S, Saranto K, Reponen E, Skants N, Vakkuri A, Vuokko R. Classification of electronic health record–related patient safety incidents: development and validation study. JMIR Med Inform 2021; 9 (08) e30470
• 32 Davy A, Borycki EM. Copy and paste in the electronic medical record: a scoping review. Knowl Manag E-Learning 2021; 13 (04) 522-535
• 33 Kinlay M, Zheng WY, Burke R, Juraskova I, Moles R, Baysari M. Medication errors related to computerized provider order entry systems in hospitals and how they change over time: a narrative review. Res Social Adm Pharm 2021; 17 (09) 1546-1552
• 34 Joseph AL, Stringer E, Borycki EM, Kushniruk AW. Evaluative frameworks and models for health information systems (HIS) and health information technologies (HIT). Stud Health Technol Inform 2022; 289: 280-285
• 35 Zheng K, Ratwani RM, Adler-Milstein J. Studying workflow and workarounds in electronic health record-supported work to improve health system performance. Ann Intern Med 2020; 172 (11) S116-S122
• 36 Redwood S, Rajakumar A, Hodson J, Coleman JJ. Does the implementation of an electronic prescribing system create unintended medication errors? A study of the sociotechnical context through the analysis of reported medication incidents. BMC Med Inform Decis Mak 2011; 11: 29
• 37 Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65 (09) S63-S67
• 38 Sittig DF, Singh H. Eight rights of safe electronic health record use. JAMA 2009; 302 (10) 1111-1113
• 39 Janols R, Lind T, Göransson B, Sandblad B. Evaluation of user adoption during three module deployments of region-wide electronic patient record systems. Int J Med Inform 2014; 83 (06) 438-449
• 40 DeLone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manage Inf Syst 2003; 19 (04) 9-30
• 41 Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability. Int J Med Inform 2011; 80 (10) 708-725
• 42 Yu P, Zhang Y, Gong Y, Zhang J. Unintended adverse consequences of introducing electronic health records in residential aged care homes. Int J Med Inform 2013; 82 (09) 772-788
• 43 Walji MF, Kalenderian E, Tran D. et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform 2013; 82 (02) 128-138
• 44 Seddon PB, Kiew MY. A partial test and development of Delone and Mclean's Model of IS success. AJIS Aust J Inf Syst 1996; 4 (01) 90-109
• 45 Pitt L, Watson R, Kavan C. Service quality: a measure of information systems effectiveness. Manage Inf Syst Q 1995; 19 (02) 173-187
• 46 Singer S, Meterko M, Baker L, Gaba D, Falwell A, Rosen A. Workforce perceptions of hospital safety culture: development and validation of the patient safety climate in healthcare organizations survey. Health Serv Res 2007; 42 (05) 1999-2021
• 47 Thompson D, Duling L. Computerized physician order entry, a factor in medication errors: descriptive analysis of events in the intensive care unit safety reporting system. J Clin Outcomes Manag 2005; 12 (08) 407-412
• 48 Bradford M, Florin J. Examining the role of innovation diffusion factors on the implementation success of enterprise resource planning systems. Int J Account Inf Syst 2003; 4 (03) 205-225
• 49 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc 1996; 3 (01) 42-55
• 50 Evans B, Glendon AI, Creed PA. Development and initial validation of an Aviation Safety Climate Scale. J Safety Res 2007; 38 (06) 675-682
• 51 Ostrom L, Wilhelmsen C, Kaplan B. Assessing safety culture. Nucl Saf 1993; 34 (02) 163-172
• 52 Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand 2009; 53 (02) 143-151
• 53 Sexton JB, Helmreich RL, Neilands TB. et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006; 6: 44
• 54 Pirnejad H, Niazkhani Z, van der Sijs H, Berg M, Bal R. Evaluation of the impact of a CPOE system on nurse-physician communication—a mixed method study. Methods Inf Med 2009; 48 (04) 350-360
• 55 Konradt U, Hertel G, Schmook R. Quality of management by objectives, task-related stressors, and non-task-related stressors as predictors of stress and job satisfaction among teleworkers. Eur J Work Organ Psychol 2003; 12 (01) 61-79
• 56 Kinzl JF, Knotzer H, Traweger C, Lederer W, Heidegger T, Benzer A. Influence of working conditions on job satisfaction in anaesthetists. Br J Anaesth 2005; 94 (02) 211-215
• 57 Beuscart-Zéphir MC, Borycki E, Carayon P, Jaspers MWM, Pelayo S. Evolution of human factors research and studies of health information technologies: the role of patient safety. Yearb Med Inform 2013; 8 (01) 67-77
• 58 Saleem JJ, Russ AL, Neddo A, Blades PT, Doebbeling BN, Foresman BH. Paper persistence, workarounds, and communication breakdowns in computerized consultation management. Int J Med Inform 2011; 80 (07) 466-479
• 59 Niazkhani Z, Pirnejad H, van der Sijs H, Aarts J. Evaluating the medication process in the context of CPOE use: the significance of working around the system. Int J Med Inform 2011; 80 (07) 490-506
• 60 Odukoya OK, Chui MA. e-Prescribing: characterisation of patient safety hazards in community pharmacies using a sociotechnical systems approach. BMJ Qual Saf 2013; 22 (10) 816-825
• 61 Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20 (06) 1159-1167
• 62 Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; 76 (Suppl. 01) S21-S27
• 63 Embi PJ, Yackel TR, Logan JR, Bowen JL, Cooney TG, Gorman PN. Impacts of computerized physician documentation in a teaching hospital: perceptions of faculty and resident physicians. J Am Med Inform Assoc 2004; 11 (04) 300-309
• 64 Holden RJ. Cognitive performance-altering effects of electronic medical records: an application of the human factors paradigm for patient safety. Cogn Technol Work 2011; 13 (01) 11-29
• 65 Santell JP, Kowiatek JG, Weber RJ, Hicks RW, Sirio CA. Medication errors resulting from computer entry by nonprescribers. Am J Health Syst Pharm 2009; 66 (09) 843-853
• 66 Hoonakker PL, Cartmill RS, Carayon P, Walker JM. Development and psychometric qualities of the SEIPS survey to evaluate CPOE/EHR implementation in ICUs. Int J Healthc Inf Syst Inform 2011; 6 (01) 51-69
• 67 Love JS, Wright A, Simon SR. et al. Are physicians' perceptions of healthcare quality and practice satisfaction affected by errors associated with electronic health record use?. J Am Med Inform Assoc 2012; 19 (04) 610-614