Appl Clin Inform 2018; 09(03): 541-552
DOI: 10.1055/s-0038-1666844
Review Article
Georg Thieme Verlag KG Stuttgart · New York

Electronic Health Record Interactions through Voice: A Review

Yaa A. Kumah-Crystal
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States
,
Claude J. Pirtle
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States
,
Harrison M. Whyte
2   Department of Computer Science, Vanderbilt University College of Arts and Science, Vanderbilt University, Nashville, Tennessee, United States
,
Edward S. Goode
2   Department of Computer Science, Vanderbilt University College of Arts and Science, Vanderbilt University, Nashville, Tennessee, United States
,
Shilo H. Anders
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States
3   Department of Anesthesiology, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States
,
Christoph U. Lehmann
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, United States
› Author Affiliations
Further Information

Publication History

25 February 2018

30 May 2018

Publication Date:
18 July 2018 (online)

Abstract

Background Usability problems in the electronic health record (EHR) lead to workflow inefficiencies when navigating charts and entering or retrieving data using standard keyboard and mouse interfaces. Voice input technology has been used to overcome some of the challenges associated with conventional interfaces and continues to evolve as a promising way to interact with the EHR.

Objective This article reviews the literature and evidence on voice input technology used to facilitate work in the EHR. It also reviews the benefits and challenges of implementation and use of voice technologies, and discusses emerging opportunities with voice assistant technology.

Methods We performed a systematic review of the literature to identify articles that discuss the use of voice technology to facilitate health care work. We searched MEDLINE and the Google search engine to identify relevant articles. We evaluated articles that discussed the strengths and limitations of voice technology to facilitate health care work. Consumer articles from leading technology publications addressing emerging use of voice assistants were reviewed to ascertain functionalities in existing consumer applications.

Results Using a MEDLINE search, we identified 683 articles that were reviewed for inclusion eligibility. The references of included articles were also reviewed. Sixty-one papers that discussed the use of voice tools in health care were included, of which 32 detailed the use of voice technologies in production environments. Articles were organized into three domains: Voice for (1) documentation, (2) commands, and (3) interactive response and navigation for patients. Of 31 articles that discussed usability attributes of consumer voice assistant technology, 12 were included in the review.

Conclusion We highlight the successes and challenges of voice input technologies in health care and discuss opportunities to incorporate emerging voice assistant technologies used in the consumer domain.

Protection of Human and Animal Subjects

Human and/or animal subjects were not included in the work.


 
  • References

  • 1 King J, Patel V, Jamoom EW, Furukawa MF. Clinical benefits of electronic health record use: national findings. Health Serv Res 2014; 49 (1 Pt 2): 392-404
  • 2 Joseph S, Sow M, Furukawa MF, Posnack S, Chaffee MA. HITECH spurs EHR vendor competition and innovation, resulting in increased adoption. Am J Manag Care 2014; 20 (09) 734-740
  • 3 Middleton B, Bloomrosen M, Dente MA. , et al; American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8
  • 4 Friedberg MW, Chen PG, Van Busum KR. , et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Q 2014; 3 (04) 1
  • 5 Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform 2011; 44 (06) 1056-1067
  • 6 Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy 2011; 4: 47-55
  • 7 Natarajan K, Stein D, Jain S, Elhadad N. An analysis of clinical queries in an electronic health record search utility. Int J Med Inform 2010; 79 (07) 515-522
  • 8 Ray A. Implementation and Usability of Meaningful Use: Usability Panel. Certification Commission for Healthcare Information Technology (CCHIT); 2017 Available at: https://www.healthit.gov/sites/default/files/facas/iu_mu_raytestimony072313.pdf . Accessed November 12, 2017
  • 9 Ratwani RM, Benda NC, Hettinger AZ, Fairbanks RJ. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA 2015; 314 (10) 1070-1071
  • 10 Montague E, Asan O. Dynamic modeling of patient and physician eye gaze to understand the effects of electronic health records on doctor-patient communication and attention. Int J Med Inform 2014; 83 (03) 225-234
  • 11 Kazmi Z. Effects of exam room EHR use on doctor-patient communication: a systematic literature review. Inform Prim Care 2013; 21 (01) 30-39
  • 12 Linder JA, Schnipper JL, Middleton B. Method of electronic health record documentation and quality of primary care. J Am Med Inform Assoc 2012; 19 (06) 1019-1024
  • 13 Tsou AY, Lehmann CU, Michel J, Solomon R, Possanza L, Gandhi T. Safe practices for copy and paste in the EHR. Systematic review, recommendations, and novel model for health IT collaboration. Appl Clin Inform 2017; 8 (01) 12-34
  • 14 Edwards ST, Neri PM, Volk LA, Schiff GD, Bates DW. Association of note quality and quality of care: a cross-sectional study. BMJ Qual Saf 2014; 23 (05) 406-413
  • 15 Sittig DF, Singh H. Eight rights of safe electronic health record use. JAMA 2009; 302 (10) 1111-1113
  • 16 Anderson C, Henner T, Burkey J. Tablet computers in support of rural and frontier clinical practice. Int J Med Inform 2013; 82 (11) 1046-1058
  • 17 Saleem JJ, Savoy A, Etherton G, Herout J. Investigating the need for clinicians to use tablet computers with a newly envisioned electronic health record. Int J Med Inform 2018; 110: 25-30
  • 18 Walsh C, Stetson P. EHR on the move: resident physician perceptions of iPads and the clinical workflow. AMIA Annu Symp Proc 2012; 2012: 1422-1430
  • 19 Hirsch EB, Raux BR, Lancaster JW, Mann RL, Leonard SN. Surface microbiology of the iPad tablet computer and the potential to serve as a fomite in both inpatient practice settings as well as outside of the hospital environment. PLoS One 2014; 9 (10) e111250
  • 20 Hockett CF. The origin of speech. Sci Am 1960; 203: 89-96
  • 21 Zuberbühler K. Linguistic capacity of non-human animals. Wiley Interdiscip Rev Cogn Sci 2015; 6 (03) 313-321
  • 22 Williams S. Can Dragon Speech Recognition Beat the World Touch Typing Record?. IT Pro Portal; 2015
  • 23 Bledsoe Jr D. Handwriting speed in an adult population. Adv Occup Ther Pract 2011; 27 (22) 10
  • 24 Issenman RM, Jaffer IH. Use of voice recognition software in an outpatient pediatric specialty practice. Pediatrics 2004; 114 (03) e290-e293
  • 25 Garrett J. , et al. Using speech recognition software to increase writing fluency for individuals with physical disabilities. J Spec Educ Technol 2011; 26 (01) 25-41
  • 26 Ajami S. Use of speech-to-text technology for documentation by healthcare providers. Natl Med J India 2016; 29 (03) 148-152
  • 27 Hoyt R, Yoshihashi A. Lessons learned from implementation of voice recognition for documentation in the military electronic health record system. Perspect Health Inf Manag 2010; 7: 1e
  • 28 Haas JS, Iyer A, Orav EJ, Schiff GD, Bates DW. Participation in an ambulatory e-pharmacovigilance system. Pharmacoepidemiol Drug Saf 2010; 19 (09) 961-969
  • 29 AHIMA. Speech Recognition in the Electronic Health Record. AHIMA Practice Brief; 2013
  • 30 Leeming BW, Porter D, Jackson JD, Bleich HL, Simon M. Computerized radiologic reporting with voice data-entry. Radiology 1981; 138 (03) 585-588
  • 31 Massey BT, Geenen JE, Hogan WJ. Evaluation of a voice recognition system for generation of therapeutic ERCP reports. Gastrointest Endosc 1991; 37 (06) 617-620
  • 32 Robbins AH, Horowitz DM, Srinivasan MK. , et al. Speech-controlled generation of radiology reports. Radiology 1987; 164 (02) 569-573
  • 33 Thumann P, Topf S, Feser A, Erfurt C, Schuler G, Mahler V. Digital speech recognition in dermatology: a pilot study with regard to medical and economic aspects [in German]. Hautarzt 2008; 59 (02) 131-134
  • 34 Williams DR, Kori SK, Williams B. , et al. Journal Club: voice recognition dictation: analysis of report volume and use of the send-to-editor function. AJR Am J Roentgenol 2013; 201 (05) 1069-1074
  • 35 Hart JL, McBride A, Blunt D, Gishen P, Strickland N. Immediate and sustained benefits of a “total” implementation of speech recognition reporting. Br J Radiol 2010; 83 (989) 424-427
  • 36 Kang HP, Sirintrapun SJ, Nestler RJ, Parwani AV. Experience with voice recognition in surgical pathology at a large academic multi-institutional center. Am J Clin Pathol 2010; 133 (01) 156-159
  • 37 Bhan SN, Coblentz CL, Norman GR, Ali SH. Effect of voice recognition on radiologist reporting time. Can Assoc Radiol J 2008; 59 (04) 203-209
  • 38 Kauppinen T, Koivikko MP, Ahovuo J. Improvement of report workflow and productivity using speech recognition--a follow-up study. J Digit Imaging 2008; 21 (04) 378-382
  • 39 Rana DS, Hurst G, Shepstone L, Pilling J, Cockburn J, Crawford M. Voice recognition for radiology reporting: is it good enough?. Clin Radiol 2005; 60 (11) 1205-1212
  • 40 Chapman WW, Aronsky D, Fiszman M, Haug PJ. Contribution of a speech recognition system to a computerized pneumonia guideline in the emergency department. Proc AMIA Symp 2000; 131-135
  • 41 Lemme PJ, Morin RL. The implementation of speech recognition in an electronic radiology practice. J Digit Imaging 2000; 13 (02) (Suppl. 01) 153-154
  • 42 Simmons J. What works: speech recognition. Talking it through. Busy physical therapy practice converts from manual transcription to voice recognition. Health Manag Technol 2002; 23 (02) 38
  • 43 LLC. C.M.G. US Inflation Calculator; 2015. Available at: http://www.usinflationcalculator.com/ . Accessed June 14, 2018
  • 44 Green HD. Adding user-friendliness and ease of implementation to continuous speech recognition technology with speech macros: case studies. J Healthc Inf Manag 2004; 18 (04) 40-48
  • 45 Pezzullo JA, Tung GA, Rogg JM, Davis LM, Brody JM, Mayo-Smith WW. Voice recognition dictation: radiologist as transcriptionist. J Digit Imaging 2008; 21 (04) 384-389
  • 46 Hodgson T, Magrabi F, Coiera E. Efficiency and safety of speech recognition for documentation in the electronic health record. J Am Med Inform Assoc 2017; 24 (06) 1127-1133
  • 47 Motyer RE, Liddy S, Torreggiani WC, Buckley O. Frequency and analysis of non-clinical errors made in radiology reports using the National Integrated Medical Imaging System voice recognition dictation software. Ir J Med Sci 2016; 185 (04) 921-927
  • 48 Ringler MD, Goss BC, Bartholmai BJ. Syntactic and semantic errors in radiology reports associated with speech recognition software. Stud Health Technol Inform 2015; 216: 922
  • 49 du Toit J, Hattingh R, Pitcher R. The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital. BMC Med Imaging 2015; 15: 8
  • 50 Basma S, Lord B, Jacks LM, Rizk M, Scaranelo AM. Error rates in breast imaging reports: comparison of automatic speech recognition and dictation transcription. AJR Am J Roentgenol 2011; 197 (04) 923-927
  • 51 Chang CA, Strahan R, Jolley D. Non-clinical errors using voice recognition dictation software for radiology reports: a retrospective audit. J Digit Imaging 2011; 24 (04) 724-728
  • 52 Quint LE, Quint DJ, Myles JD. Frequency and spectrum of errors in final radiology reports generated with automatic speech recognition technology. J Am Coll Radiol 2008; 5 (12) 1196-1199
  • 53 McGurk S, Brauer K, Macfarlane TV, Duncan KA. The effect of voice recognition software on comparative error rates in radiology reports. Br J Radiol 2008; 81 (970) 767-770
  • 54 Zick RG, Olsen J. Voice recognition software versus a traditional transcription service for physician charting in the ED. Am J Emerg Med 2001; 19 (04) 295-298
  • 55 Al-Aynati MM, Chorneyko KA. Comparison of voice-automated transcription and human transcription in generating pathology reports. Arch Pathol Lab Med 2003; 127 (06) 721-725
  • 56 Ramaswamy MR, Chaljub G, Esch O, Fanning DD, vanSonnenberg E. Continuous speech recognition in MR imaging reporting: advantages, disadvantages, and impact. AJR Am J Roentgenol 2000; 174 (03) 617-622
  • 57 Vogel M, Kaisers W, Wassmuth R, Mayatepek E. Analysis of documentation speed using web-based medical speech recognition technology: randomized controlled trial. J Med Internet Res 2015; 17 (11) e247
  • 58 Hanna TN, Shekhani H, Maddu K, Zhang C, Chen Z, Johnson JO. Structured report compliance: effect on audio dictation time, report length, and total radiologist study time. Emerg Radiol 2016; 23 (05) 449-453
  • 59 Hawkins CM, Hall S, Hardin J, Salisbury S, Towbin AJ. Prepopulated radiology report templates: a prospective analysis of error rate and turnaround time. J Digit Imaging 2012; 25 (04) 504-511
  • 60 Bruce C, Edmundson A, Coleman M. Writing with voice: an investigation of the use of a voice recognition system as a writing aid for a man with aphasia. Int J Lang Commun Disord 2003; 38 (02) 131-148
  • 61 Liu F, Tur G, Hakkani-Tür D, Yu H. Towards spoken clinical-question answering: evaluating and adapting automatic speech-recognition systems for spoken clinical questions. J Am Med Inform Assoc 2011; 18 (05) 625-630
  • 62 Nuance Communications, Inc. Nuance's Dragon Medical One Accelerates Adoption among Clinicians for Clinical Documentation. Nasdaq Globewire; 2017. Available at https://globenewswire.com/news-release/2017/08/24/1100145/0/en/Nuance-s-Dragon-Medical-One-Accelerates-Adoption-among-Clinicians-for-Clinical-Documentation.html . Accessed November 12, 2017
  • 63 Cai ZG, Gilbert RA, Davis MH. , et al. Accent modulates access to word meaning: Evidence for a speaker-model account of spoken word recognition. Cognit Psychol 2017; 98: 73-101
  • 64 Kat LWF. Pascale, Fast accent identification and accented speech recognition. In 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. IEEE; 1999 :221–224
  • 65 Nuance Communications Inc. Nuance Launches New Version of Dragon Medical Practice Edition [Press release]; 2013. Available at: http://investors.nuance.com/news-releases/news-release-details/nuance-launches-new-version-dragon-medical-practice-edition . Accessed November 12, 2017
  • 66 Meeker M. Internet Trends 2017. Kliener Perkins; 2017. Available at: http://www.kpcb.com/blog/internet-trends-report-2017 ; 1–355. Accessed November 12, 2017
  • 67 Weller C. IBM speech recognition is on the verge of super-human accuracy. Business Insider; 2017
  • 68 Guide BNL. Capturing high quality electronic health records data to support performance improvement, in Implementation Objective. Office of the National Coordinator for Health Information Technology; 2013
  • 69 Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc 2011; 18 (02) 181-186
  • 70 Husnjak S, Perakovic D, Jovovic I. Possibilities of using speech recognition systems of smart terminal devices in traffic environment. Procedia Eng 2014; 69: 778-787
  • 71 Grasso M, Ebert D, Finin T. The effect of perceptual structure on multimodal speech recognition interfaces. Proc AMIA Annu Fall Symp; 1997
  • 72 Klatt EC. Voice-activated dictation for autopsy pathology. Comput Biol Med 1991; 21 (06) 429-433
  • 73 Henricks WH, Roumina K, Skilton BE, Ozan DJ, Goss GR. The utility and cost effectiveness of voice recognition technology in surgical pathology. Mod Pathol 2002; 15 (05) 565-571
  • 74 Sistrom CL, Honeyman JC, Mancuso A, Quisling RG. Managing predefined templates and macros for a departmental speech recognition system using common software. J Digit Imaging 2001; 14 (03) 131-141
  • 75 Langer S. Radiology speech recognition: workflow, integration, and productivity issues. Curr Probl Diagn Radiol 2002; 31 (03) 95-104
  • 76 Green HD. Speech recognition technology for the medical field. J Am Acad Bus Camb 2003; 2 (02) 299-303
  • 77 Nagy M, Hanzlicek P, Zvarova J. , et al. Voice-controlled data entry in dental electronic health record. Stud Health Technol Inform 2008; 136: 529-534
  • 78 Wolf DM, Kapadia A, Kintzel J, Anton BB. Nurses using futuristic technology in today's healthcare setting. Stud Health Technol Inform 2009; 146: 59-63
  • 79 Friend TH, Jennings SJ, Levine WC. Communication patterns in the perioperative environment during epic electronic health record system implementation. J Med Syst 2017; 41 (02) 22
  • 80 Friend TH, Jennings SJ, Copenhaver MS, Levine WC. Implementation of the Vocera communication system in a quaternary perioperative environment. J Med Syst 2017; 41 (01) 6
  • 81 Richardson JE, Ash JS. The effects of hands free communication devices on clinical communication: balancing communication access needs with user control. AMIA Annu Symp Proc 2008; 621-625
  • 82 Fang DZ, Patil T, Belitskaya-Levy I, Yeung M, Posley K, Allaudeen N. Use of a hands free, instantaneous, closed-loop communication device improves perception of communication and workflow integration in an academic teaching hospital: a pilot study. J Med Syst 2017; 42 (01) 4
  • 83 Mewes A, Hensen B, Wacker F, Hansen C. Touchless interaction with software in interventional radiology and surgery: a systematic literature review. Int J CARS 2017; 12 (02) 291-305
  • 84 Salama IA, Schwaitzberg SD. Utility of a voice-activated system in minimally invasive surgery. J Laparoendosc Adv Surg Tech A 2005; 15 (05) 443-446
  • 85 Radziwill N, Benton N. Evaluating Quality of Chatbots and Intelligent Conversational Agents. Cornell University Library. Computers and Society; 2017
  • 86 Bauermeister J, Giguere R, Leu CS. , et al. Interactive voice response system: data considerations and lessons learned during a rectal microbicide placebo adherence trial for young men who have sex with men. J Med Internet Res 2017; 19 (06) e207
  • 87 Lalanne DK. Juerg Human Machine Interaction: Research Results of the MMI Program. Lecture Notes in Computer Science (Book 5440). Berlin Heidelberg: Springer; 2009
  • 88 Reidel K, Tamblyn R, Patel V, Huang A. Pilot study of an interactive voice response system to improve medication refill compliance. BMC Med Inform Decis Mak 2008; 8: 46
  • 89 Krenzelok EP, Mrvos R. A regional poison information center IVR medication identification system: does it accomplish its goal?. Clin Toxicol (Phila) 2011; 49 (09) 858-861
  • 90 Siwicki B. Special Report: AI voice assistants have officially arrived in healthcare. Heathcare IT News; 2018
  • 91 Harnad S. The annotation game: On Turing (1950) on computing, machinery, and intelligence. In The Turing test sourcebook: philosophical and methodological issues in the quest for the thinking computer. Dordrecht, Netherlands: Kluwer; 2006
  • 92 Cadario F, Binotti M, Brustia M. , et al. Telecare for teenagers with type 1 diabetes: a trial. Minerva Pediatr 2007; 59 (04) 299-305
  • 93 iRunway. Speech Recognition: Technology & Patent Landscape; 2015. Available at: http://www.i-runway.com/ . Accessed June 14, 2018
  • 94 Staff DT. Virtual Assistant Comparison: Cortana, Google Assistant, Siri, Alexa, Bixby. Digital Trends; 2017
  • 95 Habib O. Conversational Technology: Siri, Alexa, Cortana, and the Google Assistant. App Dynamics; 2017
  • 96 Carey S. No, Amazon Alexa Isn't Listening to Your Conversations. Tech World; 2017
  • 97 Hinton G, Deng L, Deng Y. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 2012; 29 (06) 82-97
  • 98 Graves A, Mohamed AR, Hinton G. Speech recognition with deep recurrent neural networks. Acoustics, Speech. Signal Process 2013; 01: 6645-6649
  • 99 Demetriou GC, Atwell ES. Semantics in Speech Recognition and Understanding: A Survey. Proceedings of the 1994 AISB Workshop on Computational Linguistics for Speech and Handwriting Recognition. 1994 AISB Workshop on Computational Linguistics for Speech and Handwriting Recognition, April 12, 1994; United Kingdom: University of Leeds
  • 100 Annas GJ. HIPAA regulations - a new era of medical-record privacy?. N Engl J Med 2003; 348 (15) 1486-1490
  • 101 Services AW. Architecting for HIPAA Security and Compliance on Amazon Web Services. Amazon Web Services; 2017
  • 102 Amazon Web Services HIPAA Compliance; 2017 . Available at https://aws.amazon.com/compliance/hipaa-compliance/ . Accessed November 12, 2017
  • 103 Hiner C. Is your Cloud-based Speech Recognition HIPAA Compliant?. NVOQ; 2017
  • 104 Faas R. If You Use The New iPad's Dictation Feature For Work, You Could Be Breaking The Law. Cult of Mac; 2012
  • 105 Glatter R. Why Hospitals Are In Pursuit of the Ideal Amazon Alexa App. Forbes; 2017
  • 106 Dahm MR, O'Grady C, Yates L, Roger P. Into the spotlight: exploring the use of the dictaphone during surgical consultations. Health Commun 2015; 30 (05) 513-520
  • 107 Ahmed J, Roy A, Abed T, Kotecha B. Does dictating the letter to the GP in front of a follow-up patient improve satisfaction with the consult? A randomised controlled trial. Eur Arch Otorhinolaryngol 2010; 267 (04) 619-623
  • 108 Hamilton W, Round A, Taylor P. Dictating clinic letters in front of the patient. Letting patients see copy of consultant's letter is being studied in trial. BMJ 1997; 314 (7091): 1416
  • 109 de Silva R, Misbahuddin A, Mikhail S, Grayson K. Do patients wish to ‘listen in’ when doctors dictate letters to colleagues?. JRSM Short Rep 2010; 1 (06) 47
  • 110 Lloyd BW. A randomised controlled trial of dictating the clinic letter in front of the patient. BMJ 1997; 314 (7077): 347-348
  • 111 Waterston T. Dictating clinic letters in front of the patient. Further research should be qualitative. BMJ 1997; 314 (7091): 1416
  • 112 Nuance Communications, Inc. Nuance and Epic Join Forces on Artificial Intelligence to Revolutionize Disabled Veterans Accessibility to Health IT; 2017 [Press release]. Available at https://www.nuance.com/about-us/newsroom/press-releases/nuance-and-epic-improve-accessibility-for-veterans.html . Accessed June 26, 2018
  • 113 McCluskey PD. Meet Eva, the voice-activated ‘assistant’ for doctors. Boston Globe; 2018
  • 114 Dela Cruz JE, Shabosky JC, Albrecht M. , et al. Typed versus voice recognition for data entry in electronic health records: emergency physician time use and interruptions. West J Emerg Med 2014; 15 (04) 541-547