Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients

Michelle A. Jahn; Brian W. Porter; Himalaya Patel; Alan J. Zillich; Steven R. Simon; Alissa L. Russ

doi:10.1055/s-0038-1660521

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2018; 09(02): 467-477
DOI: 10.1055/s-0038-1660521

Research Article

Schattauer GmbH Stuttgart

Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients

Authors

Michelle A. Jahn

¹Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States

²School of Industrial Engineering, Purdue University, West Lafayette, Indiana, United States
Brian W. Porter

¹Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
Himalaya Patel

¹Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
Alan J. Zillich

³Department of Pharmacy Practice, College of Pharmacy, Purdue University, West Lafayette, Indiana, United States
Steven R. Simon

⁴Geriatrics and Extended Care Service, VA Boston Healthcare System and Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, Massachusetts, United States
Alissa L. Russ

¹Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States

³Department of Pharmacy Practice, College of Pharmacy, Purdue University, West Lafayette, Indiana, United States

⁵Regenstrief Institute Inc., Indianapolis, Indiana, United States

Abstract

Background Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing.

Objective This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents.

Methods We conducted usability testing with providers and patients in a human–computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction.

Results Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8–12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2–15.0) minutes; retrieved the clinical document within 0.8 (0.5–5.7) minutes; and sent the attached clinical document in 6.3 (1.5–18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0–7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks.

Conclusion This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing.

Keywords

personal health records - secure messaging - communication - interfaces and usability - electronic health records and systems

Full Text

References

References
1 Cronin RM, Davis SE, Shenson JA, Chen Q, Rosenbloom ST, Jackson GP. Growth of secure messaging through a patient portal as a form of outpatient interaction across clinical specialties. Appl Clin Inform 2015; 6 (02) 288-304
2 Heyworth L, Clark J, Marcello TB. , et al. Aligning medication reconciliation and secure messaging: qualitative study of primary care providers' perspectives. J Med Internet Res 2013; 15 (12) e264
3 Shenson JA, Cronin RM, Davis SE, Chen Q, Jackson GP. Rapid growth in surgeons' use of secure messaging in a patient portal. Surg Endosc 2016; 30 (04) 1432-1440
4 Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res 2015; 17 (02) e40
5 Haun JN, Patel NR, Lind JD, Antinori N. Large-scale survey findings inform patients' experiences in using secure messaging to engage in patient-provider communication and self-care management: a quantitative assessment. J Med Internet Res 2015; 17 (12) e282
6 Henry J, Pylypchuk Y, Patel V. Electronic capabilities for patient engagement among US non-federal acute care hospitals: 2012–2015. The Office of the National Coordinator for Health Information Technology; 2016
7 Schnipper JL, Gandhi TK, Wald JS. , et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008; 16 (02) 147-155
8 Heyworth L, Paquin AM, Clark J. , et al. Engaging patients in medication reconciliation via a patient portal following hospital discharge. J Am Med Inform Assoc 2014; 21 (e1): e157-e162
9 Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006; 13 (02) 121-126
10 Chow CK, Redfern J, Hillis GS. , et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial. JAMA 2015; 314 (12) 1255-1263
11 Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res 2013; 15 (07) e133
12 Kuo A, Dang S. Secure messaging in electronic health records and its impact on diabetes clinical outcomes: a systematic review. Telemed J E Health 2016; 22 (09) 769-777
13 Green BB, Cook AJ, Ralston JD. , et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008; 299 (24) 2857-2867
14 Raghu TS, Frey K, Chang Y-H, Cheng M-R, Freimund S, Patel A. Using secure messaging to update medications list in ambulatory care setting. Int J Med Inform 2015; 84 (10) 754-762
15 Haun JN, Lind JD, Shimada SL. , et al. Evaluating user experiences of the secure messaging tool on the Veterans Affairs' patient portal system. J Med Internet Res 2014; 16 (03) e75
16 Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015; 17 (06) e148
17 Ferranti JM, Musser RC, Kawamoto K, Hammond WE. The clinical document architecture and the continuity of care record: a critical analysis. J Am Med Inform Assoc 2006; 13 (03) 245-252
18 Choi M, Lee HS, Park JH. Usability of academic electronic medical record application for nursing students' clinical practicum. Healthc Inform Res 2015; 21 (03) 191-195
19 Russ AL, Jahn MA, Patel H. , et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users' detection of medication discrepancies. J Biomed Inform 2018;
20 Russ AL, Weiner M, Russell SA, Baker DA, Fahner WJ, Saleem JJ. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf 2012; 38 (12) 531-540
21 Nielsen J. Demonstrate Thinking Aloud by Showing Users a Video. Available at: https://www.nngroup.com/articles/thinking-aloud-demo-video2014 . Accessed February 23, 2017
22 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
23 Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive impairment among potential subjects for clinical research. Med Care 2002; 40 (09) 771-781
24 Nielsen J. Why you only need to test with 5 users. Useit Com Alertbox; 2000
25 FDA. Applying Human Factors and Usability Engineering to Medical Devices. Silver Spring, MA: Food and Drug Administration; 2016
26 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
27 Cooper A, Reimann R, Cronin D. About face 3: the essentials of interaction design. John Wiley & Sons; 2007
28 The VA and Academic Medicine: Partners in Health Care, Training, and Research. (2014). Available at: https://www.aamc.org/download/385612/data/07182014.pdf
29 Thompson MJ, Reilly JD, Valdez RS. Work system barriers to patient, provider, and caregiver use of personal health records: A systematic review. Appl Ergon 2016; 54: 218-242
30 Urowitz S, Wiljer D, Dupak K. , et al. Improving diabetes management with a patient portal: a qualitative study of diabetes self-management portal. J Med Internet Res 2012; 14 (06) e158
31 Jahn M, Caldwell BS. Developing Distributed Expertise Coordination Models of Diabetes Care Teams. In: Proceedings of 2018 Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA (In Press)
32 U.S. Department of Veterans Affairs. VA Utilization Profile. Web 2017; 2017: 20