Download PDF
Methods Inf Med 2017; 56(01): 1-12
DOI: 10.3414/ME16-01-0097
Paper
Schattauer GmbH

FIN-TIERA: A Tool for Assessing Technology Induced Errors[*]

Sari Palojoki
1   Faculty of Social Sciences and Business Studies, University of Eastern Finland, Kuopio, Finland
2   Group Administration, Helsinki and Uusimaa Hospital District and Helsinki University Hospital, Helsinki, Finland
,
Tuuli Pajunen
2   Group Administration, Helsinki and Uusimaa Hospital District and Helsinki University Hospital, Helsinki, Finland
,
Lasse Lehtonen
2   Group Administration, Helsinki and Uusimaa Hospital District and Helsinki University Hospital, Helsinki, Finland
,
Kaija Saranto
1   Faculty of Social Sciences and Business Studies, University of Eastern Finland, Kuopio, Finland
› Author AffiliationsThis work benefited from the support of Finnish governmental research funding (study grant TYH2014224).
Further Information

Publication History

received: 14 August 2016

accepted in revised form: 15 November 2016

Publication Date:
22 January 2018 (online)

    Permissions and Reprints

Summary

Background: Due to the complexity of healthcare processes, the potential for Health Information Systems (HIS) to cause technology-induced errors is a growing concern. Health Information Technology (HIT) errors nearly always threaten good patient care and can lead to patient harm. Instruments to allow hospitals to proactively identify areas of Electronic Health Records (EHR) safety, to set priorities and to intervene before incidents occur are currently underdeveloped.

Objectives: The aim was to design a Finnish questionnaire to measure EHR users’ perceptions of common EHR-related safety concerns in a specialized hospital district context through the lens of the theory of socio-technical dimensions. Moreover, the aim was to measure its reliability by assessing its internal consistency and validity, namely its content and construct validity.

Methods: We constructed the instrument, based on the socio-technical theory and Sittig and Singh’s study findings, through a multi-stage process, and expert panels evaluated it to ensure its content validity. The final questionnaire consisted of eight error types to be assessed on a qualitative risk matrix scale. We used a cross-sectional design to test its psychometric properties. Application of the FIN-TIERA Questionnaire to a sample of 2864 clinicians in 2015 then served to evaluate the instrument’s reliability as well as its construct validity.

Results: All eight multi-item scales showed high internal consistency (range α > 0.798-0.932 and CR 0.845-0.983). The average variance extracted (AVE) served to assess the confirmatory factor analysis (CFA). The results of the model fit with AGFI = .86, CFI = .898, RMSEA = .052, SRMR = .048 were deemed acceptable. For all factors, AVE yielded values > 0.5, which indicates adequate convergence and supports convergent validity. Discriminant validity was established for five out of a total of eight latent variables.

Conclusions: FIN-TIERA is a new multi-dimensional instrument which may be a useful tool for assessing risk in EHR. Our testing shows its potential for use in-hospital settings: the involvement of EHR users demonstrated initial reliability and validity. Further research is recommended to assess the instrument’s psychometric properties.

Keywords

Electronic health records - patient safety, psychometrics - socio-technical research - questionnaire

* Supplementary material published on our website https://doi.org/10.3414/ME16-01-0097


 

  • References

  • 1 Sittig DF, Gonzalez D, Singh H. Contingency planning for electronic health record-based care continuity: a survey of recommended practices. Int J Med Inform. 2014; 83 (11) 797-804. doi: 10.1016/ j.ijmedinf.2014.07.007. PubMed PMID: 25200197.
    CrossrefPubMedGoogle Scholar
  • 2 Shekelle PG, Goldzweig CL. Costs and benefits of health technology information: an updated systematic review. London, UK: The Health Foundation, London; 2009
    Google Scholar
  • 3 Sittig DF, Singh H. Electronic health records and national patient-safety goals. N Engl J Med. 2012; 367 (19) 1854-1860. doi: 10.1056/NEJMsb120 5420. PMID: 23134389.
    CrossrefPubMedGoogle Scholar
  • 4 Bellwood P, Borycki EM, Kushniruk AW. Awareness of technology-induced errors and processes for identifying and preventing such errors. Stud Health Technol Inform. 2015; 208: 61-65. PubMed PMID: 25676948.
    PubMedGoogle Scholar
  • 5 Borycki EM, Kushniruk AW, Bellwood P, Brender J. Technology-induced errors. The current use of frameworks and models from the biomedical and life sciences literatures. Methods Inf Med. 2012; 51 (02) 95-103. doi: 10.3414/ME11-02-0009. PubMed PMID: 22101488.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Nøhr C, Borycki EM, Kushniruk AW, Kuziemsky CE. Theories and Methods for Context Sensitive Health Informatics. Stud Health Technol Inform. 2015; 218: 40587. PubMed PMID: 26262517.
    PubMedGoogle Scholar
  • 7 IOM (Institute of Medicine). Health IT and patient safety: building safer systems for safer care. Washington, DC: The National Academies Press; 2012
    Google Scholar
  • 8 Braithwaite J, Wears RL, Hollnagel E. Resilient health care: turning patient safety on its head. Int J Qual Health Care. 2015; 27 (05) 418-420. doi: 10.1093/intqhc/mzv063. PubMed PMID: 26294 709.
    CrossrefPubMedGoogle Scholar
  • 9 Öhrn A, Ericsson C, Andersson C, Elfström J. High Rate of Implementation of Proposed Actions for Improvement With the Healthcare Failure Mode Effect Analysis Method: Evaluation of 117 Analyses. J Patient Saf.. 2015 Feb 24. [Epub ahead of print]. PubMed PMID: 25719818.
    PubMedGoogle Scholar
  • 10 Smith MW, Ash JS, Sittig DF, Singh H. Resilient Practices in Maintaining Safety of Health Information Technologies. J Cogn Eng Decis Mak. 2014; 8 (03) 265-282. PubMed PMID: 25866492.
    CrossrefPubMedGoogle Scholar
  • 11 Sittig DF, Singh H. A red-flag-based approach to risk management of EHR-related safety concerns. J Healthc Risk Manag. 2013; 33 (02) 21-26. doi: 10.1002/jhrm.21123. PubMed PMID: 24078205.
    CrossrefPubMedGoogle Scholar
  • 12 Singh H, Sittig DF. Measuring and improving patient safety through health information technology: The Health IT Safety Framework. BMJ Qual Saf. 2016; 25 (04) 226-232. doi: 10.1136/bmjqs-2015-004486. PubMed PMID: 26369894.
    CrossrefPubMedGoogle Scholar
  • 13 Sittig DF. Grand challenges in medical informatics?. J Am Med Inform Assoc. 1994; 1 (05) 412-413. PubMed PMID: 7850565.
    CrossrefPubMedGoogle Scholar
  • 14 Sittig DF, Singh H. New Socio-technical Model for Studying Health Information Technology in Complex Adaptive Healthcare Systems. Qual Saf Health Care. 2010; 19 (Suppl. 03) i68-i74. doi: 10.1136/ qshc.2010.042085. PubMed PMID: 20959322.
    CrossrefPubMedGoogle Scholar
  • 15 Sittig DF, Singh H. Defining Health Information Technology-related Errors: New Developments Since To Err is Human. Arch Intern Med. 2011; 171 (14) 1281-1284. doi: 10.1001/archinternmed. 2011.327. PubMed PMID: 21788544.
    CrossrefPubMedGoogle Scholar
  • 16 Sittig DF, Singh H. SAFER Electronic Health Records: Safety Assurance Factors for EHR Resilience. Oakville, ON, Canada: Apple Academic Press; 2015
    Google Scholar
  • 17 Menon S, Singh H, Meyer AN, Belmont E, Sittig DF. Electronic health record-related safety concerns: a cross-sectional survey. J Healthc Risk Manag. 2014; 34 (01) 14-26. doi: 10.1002/jhrm. 21146. PubMed PMID: 25070253.
    PubMedGoogle Scholar
  • 18 Larizgoitia I, Bouesseau MC, Kelley E. WHO Efforts to Promote Reporting of Adverse Events and Global Learning. J Public Health Res. 2013; 2 (03) e29. doi: 10.4081/jphr.2013.e29. PubMed PMID: 25170500.
    CrossrefPubMedGoogle Scholar
  • 19 Meeks DW, Smith MW, Taylor L, Sittig DF, Scott JM, Singh H. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc. 2014; 21 (06) 1053-1059. doi: 10.1136/ amiajnl-2013-002578. PubMed PMID: 24951796.
    CrossrefPubMedGoogle Scholar
  • 20 Sittig DF, Ash JS, Singh H. The SAFER guides: empowering organizations to improve the safety and effectiveness of electronic health records. Am J Manag Care. 2014; 20 (05) 418-423. PubMed PMID: 25181570.
    PubMedGoogle Scholar
  • 21 Blumenthal D, Glaser JP. Information technology comes to medicine. N Engl J Med. 2007; 356 (24) 2527-2534. PubMed PMID: 17568035.
    CrossrefPubMedGoogle Scholar
  • 22 Palojoki S, Mäkelä M, Lehtonen L, Saranto K. An analysis of electronic health record-related patient safety incidents. Health Informatics J.. 2016 Mar 7. pii: 1460458216631072. [Epub ahead of print]. PubMed PMID: 26951568.
    PubMedGoogle Scholar
  • 23 Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform. 2009; 78 (Suppl. 01) S69-76. doi: 10.1016/j.ijmedinf. 2008.07.015. PubMed PMID: 18786852.
    CrossrefPubMedGoogle Scholar
  • 24 Streiner DL, Norman GR, Cairney J. Health Measurement Scales. A practical guide to their development and use.. 5th ed. Oxford: Oxford University Press; 2015
    Google Scholar
  • 25 Fowler FJ. Survey Research Methods. 5th ed. Boston: Sage Publications; 2014
    Google Scholar
  • 26 Presser S. Methods for Testing and Evaluating Survey Questionnaires. New Jersey: John Wiley & Sons; 2004
    Google Scholar
  • 27 Waltz C, Strickland O, Lenz E. Measurement in nursing and health research. 4th ed. New York: Springer; 2010
    Google Scholar
  • 28 Kline R. B. Principles and Practice of Structural Equation Modeling. 3rd ed. New York: The Guilford Press; 2011
    Google Scholar
  • 29 ISO 31010: Risk management – risk assessment techniques. Geneva, Switzerland: ISO Copyright Office; 2009. Reference number: IEC/FDIS 31010:2009(E).
  • 30 Renn O. Concepts of Risk. Krimsky S, Golding D. Social Theories of Risk.. Westport, CT, USA: Praeger Publishing; 1992: 52-79.
    Google Scholar
  • 31 Card AJ, Ward JR, Clarkson PJ. Trust-level risk evaluation and risk control guidance in the NHS East of England. Risk Anal. 2014; 34 (08) 1469-1481. doi: 10.1111/risa.12159. PubMed PMID: 24341726.
    CrossrefPubMedGoogle Scholar
  • 32 Palojoki S, Pajunen T, Saranto K, Lehtonen L. Electronic Health Record-Related Safety Concerns: A Cross-Sectional Survey of Electronic Health Record Users. JMIR Med Inform. 2016; 4 (02) e13. doi: 10.2196/medinform.5238. PubMed PMID: 27154 599.
    CrossrefPubMedGoogle Scholar
  • 33 Medical Research Act 488/199. Finlex Data Bank. Available from: http://www.finlex.fi/fi/laki/kaannokset/1999/en19990488. Archived at: http://www.webcitation.org/6diXnx02M.
    PubMedGoogle Scholar
  • 34 R Foundation for Statistical Computing. R: A language and environment for statistical computing. Vienna, Austria: 2014. Available from: http://www.R-project.org/.ArchivedatWebCiteCacheID6cCVU4SXk.
    Google Scholar
  • 35 Hair JF. Multivariate Data Analysis. 7th ed. New Jersey: Pearson Prentice Hall; 2010
    Google Scholar
  • 36 Ong MS, Magrabi F, Coiera E. Syndromic surveillance for health information system failures: a feasibility study. J Am Med Inform Assoc. 2013; 20 (03) 506-512. doi: 10.1136/amiajnl-2012-00 1144. PubMed PMID: 23184193.
    CrossrefPubMedGoogle Scholar
  • 37 The Joint Commission Resources Inc. Sentinal Event Alert: Behaviors That Undermine a Culture of Safety. Washington, DC: 2008. Available from: https://www.jointcommission.org/sentinel_event_alert_issue_40_behaviors_that_undermine_a_culture_of_safety/ Archived at http://www.webcitation.org/6lFmYtioq.
    Google Scholar
  • 38 Alamry A, Al Owais SM, Marini AM, Al-Dorzi H, Alsolamy S, Arabi Y. Application of Failure Mode Effect Analysis to Improve the Care of Septic Patients Admitted Through the Emergency Department. J Patient Saf.. 2014 Aug 12. [Epub ahead of print]. PubMed PMID: 25119786.
    PubMedGoogle Scholar
  • 39 Joint Commission Resources Inc. Failure Mode and Effects Analysis in Health Care: Proactive Risk Reduction. 3rd ed. Oakbrook, IL: Joint Commission Resources Inc; 2010
    Google Scholar
  • 40 Shebl NA, Franklin BD, Barber N. Failure mode and effects analysis outputs: are they valid?. BMC Health Serv Res. 2012; 12: 150. doi:10.1186/1472-6963-12-150. PubMed PMID: 22682433.
    CrossrefPubMedGoogle Scholar
  • 41 Arat S, Van den Zegel A, Van Rillaer M, Moons P, Vandenberghe J, De Langhe E. et al. Development and preliminary evaluation of the validity and reliability of a revised illness perception questionnaire for healthcare professionals. BMC Nurs. 2016; 15: 34. doi: 10.1186/s12912-016-0156-4. PubMed PMID: 27252595.
    CrossrefPubMedGoogle Scholar
  • 42 Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. 2003; 88 (05) 879-903. PubMed PMID: 14516251.
    CrossrefPubMedGoogle Scholar
  • 43 Hernán MA, Hernández-Díaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004; 15 (05) 615-625. PubMed PMID: 15308962.
    CrossrefPubMedGoogle Scholar
  • 44 Lakes KD. Restricted sample variance reduces generalizability. Psychol Assess. 2013; 25 (02) 643-650. doi: 10.1037/a0030912. PubMed PMID: 23205627.
    CrossrefPubMedGoogle Scholar
  • 45 Clark LA, Watson D. Constructing validity: Basic issues in objective scale development. Psychol Assess. 1995; 07: 309-319. doi: http://dx.doi.org/10.1037/1040-3590.7.3.309.
    CrossrefPubMedGoogle Scholar
  • 46 Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J. et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009; 339: b4012. doi: 10.1136/bmj.b4012. PubMed PMID:19841007.
    CrossrefPubMedGoogle Scholar