Appl Clin Inform 2018; 09(02): 326-335
DOI: 10.1055/s-0038-1649509
Research Article
Schattauer GmbH Stuttgart

Evaluating the Efficiency and Safety of Speech Recognition within a Commercial Electronic Health Record System: A Replication Study

Tobias Hodgson
1  Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, New South Wales, Australia
,
Farah Magrabi
1  Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, New South Wales, Australia
,
Enrico Coiera
1  Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, New South Wales, Australia
› Author Affiliations
Funding This work was supported by the NHMRC Centre for Research Excellence in eHealth (APP1032664).
Further Information

Publication History

24 December 2017

24 March 2018

Publication Date:
16 May 2018 (online)

Abstract

Objective To conduct a replication study to validate previously identified significant risks and inefficiencies associated with the use of speech recognition (SR) for documentation within an electronic health record (EHR) system.

Methods Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR using a commercial EHR system. The experiment design, setting, and tasks (E2) replicated an earlier study (E1), while technical integration issues that may have led to poorer SR performance were addressed.

Results Complex tasks were significantly slower to complete using SR (16.94%) than KBM (KBM: 191.9 s, SR: 224.4 s; p = 0.009; CI, 11.9–48.3), replicating task completion times observed in the earlier experiment. Errors (non-typographical) were significantly higher with SR compared with KBM for both simple (KBM: 3, SR: 84; p < 0.001; CI, 1.5–2.5) and complex tasks (KBM: 23, SR: 53; p = 0.001; CI, 0.5–1.0), again replicating earlier results (E1: 170, E2: 163; p = 0.660; CI, 0.0–0.0). Typographical errors were reduced significantly in the new study (E1: 465, E2: 150; p < 0.001; CI, 2.0–3.0).

Discussion The results of this study replicate those reported earlier. The use of SR for clinical documentation within an EHR system appears to be consistently associated with decreased time efficiencies and increased errors. Modifications implemented to optimize SR integration in the EHR seem to have resulted in minor improvements that did not fundamentally change overall results.

Conclusion This replication study adds further evidence for the poor performance of SR-assisted clinical documentation within an EHR. Replication studies remain rare in informatics literature, especially where study results are unexpected or have significant implication; such studies are clearly needed to avoid overdependence on the results of a single study.

Authors' Contributions

T.H., E.C., and F.M. conceived the study and its design. T.H. conducted the research, the primary analysis, and the initial drafting of the manuscript. E.C. and F.M. contributed to the analysis and drafting of the manuscript, and T.H., E.C., and F.M. approved the final manuscript. T.H. is the corresponding author.


Protection of Human and Animal Subjects

This study is approved by the Sydney Local Health District Local Health District Human Ethics Committee—Concord Repatriation and General Hospital (CRGH) (LNR/14/CRGH/272).


Supplementary Material