Download PDF
Appl Clin Inform 2018; 09(02): 467-477
DOI: 10.1055/s-0038-1660521
Research Article
Schattauer GmbH Stuttgart

Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients

Michelle A. Jahn
1   Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
2   School of Industrial Engineering, Purdue University, West Lafayette, Indiana, United States
,
Brian W. Porter
1   Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
,
Himalaya Patel
1   Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
,
Alan J. Zillich
3   Department of Pharmacy Practice, College of Pharmacy, Purdue University, West Lafayette, Indiana, United States
,
Steven R. Simon
4   Geriatrics and Extended Care Service, VA Boston Healthcare System and Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, Massachusetts, United States
,
Alissa L. Russ
1   Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service, Center for Health Information and Communication, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana, United States
3   Department of Pharmacy Practice, College of Pharmacy, Purdue University, West Lafayette, Indiana, United States
5   Regenstrief Institute Inc., Indianapolis, Indiana, United States
› Author Affiliations Funding Primary funding was provided by VA HSR&D grant IIR #14–059 (PI: Simon). Dr. Russ was supported in part by a VA HSR&D Research Career Development Award (CDA 11–214). Michelle Jahn was supported in part by the Purdue University Doctoral Fellowship and the National Science Foundation Graduate Research Fellowship (#DGE-1333468).
Further Information

Publication History

26 September 2017

02 May 2018

Publication Date:
27 June 2018 (online)

   Permissions and Reprints

Abstract

Background Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing.

Objective This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents.

Methods We conducted usability testing with providers and patients in a human–computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction.

Results Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8–12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2–15.0) minutes; retrieved the clinical document within 0.8 (0.5–5.7) minutes; and sent the attached clinical document in 6.3 (1.5–18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0–7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks.

Conclusion This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing.

Keywords

personal health records - secure messaging - communication - interfaces and usability - electronic health records and systems

Protection of Human and Animal Subjects

This study was performed in compliance with standards set by the Institutional Review Boards at Indiana University and the VA Research and Development Committee. These organizations reviewed and approved the research methods.


Authors' Contributions

A.R. led the grant writing and study design, with contributions from A.Z., A.L., and S.S. M.J. developed the tasks scenarios with input from coauthors; moderated most usability sessions; conducted the analyses for use errors; and drafted the initial manuscript. B.P. recruited patient participants; assisted with demographic, task completion, and efficiency data analyses; and drafted most figures and tables. H.P. moderated some usability sessions and analyzed the satisfaction data. All authors contributed to and approved the final version of the manuscript.


 

  • References

  • 1 Cronin RM, Davis SE, Shenson JA, Chen Q, Rosenbloom ST, Jackson GP. Growth of secure messaging through a patient portal as a form of outpatient interaction across clinical specialties. Appl Clin Inform 2015; 6 (02) 288-304
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Heyworth L, Clark J, Marcello TB. , et al. Aligning medication reconciliation and secure messaging: qualitative study of primary care providers' perspectives. J Med Internet Res 2013; 15 (12) e264
    CrossrefPubMedGoogle Scholar
  • 3 Shenson JA, Cronin RM, Davis SE, Chen Q, Jackson GP. Rapid growth in surgeons' use of secure messaging in a patient portal. Surg Endosc 2016; 30 (04) 1432-1440
    CrossrefPubMedGoogle Scholar
  • 4 Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res 2015; 17 (02) e40
    CrossrefPubMedGoogle Scholar
  • 5 Haun JN, Patel NR, Lind JD, Antinori N. Large-scale survey findings inform patients' experiences in using secure messaging to engage in patient-provider communication and self-care management: a quantitative assessment. J Med Internet Res 2015; 17 (12) e282
    CrossrefPubMedGoogle Scholar
  • 6 Henry J, Pylypchuk Y, Patel V. Electronic capabilities for patient engagement among US non-federal acute care hospitals: 2012–2015. The Office of the National Coordinator for Health Information Technology; 2016
    Google Scholar
  • 7 Schnipper JL, Gandhi TK, Wald JS. , et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008; 16 (02) 147-155
    PubMedGoogle Scholar
  • 8 Heyworth L, Paquin AM, Clark J. , et al. Engaging patients in medication reconciliation via a patient portal following hospital discharge. J Am Med Inform Assoc 2014; 21 (e1): e157-e162
    CrossrefPubMedGoogle Scholar
  • 9 Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc 2006; 13 (02) 121-126
    CrossrefPubMedGoogle Scholar
  • 10 Chow CK, Redfern J, Hillis GS. , et al. Effect of lifestyle-focused text messaging on risk factor modification in patients with coronary heart disease: a randomized clinical trial. JAMA 2015; 314 (12) 1255-1263
    CrossrefPubMedGoogle Scholar
  • 11 Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res 2013; 15 (07) e133
    CrossrefPubMedGoogle Scholar
  • 12 Kuo A, Dang S. Secure messaging in electronic health records and its impact on diabetes clinical outcomes: a systematic review. Telemed J E Health 2016; 22 (09) 769-777
    CrossrefPubMedGoogle Scholar
  • 13 Green BB, Cook AJ, Ralston JD. , et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008; 299 (24) 2857-2867
    CrossrefPubMedGoogle Scholar
  • 14 Raghu TS, Frey K, Chang Y-H, Cheng M-R, Freimund S, Patel A. Using secure messaging to update medications list in ambulatory care setting. Int J Med Inform 2015; 84 (10) 754-762
    CrossrefPubMedGoogle Scholar
  • 15 Haun JN, Lind JD, Shimada SL. , et al. Evaluating user experiences of the secure messaging tool on the Veterans Affairs' patient portal system. J Med Internet Res 2014; 16 (03) e75
    CrossrefPubMedGoogle Scholar
  • 16 Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015; 17 (06) e148
    CrossrefPubMedGoogle Scholar
  • 17 Ferranti JM, Musser RC, Kawamoto K, Hammond WE. The clinical document architecture and the continuity of care record: a critical analysis. J Am Med Inform Assoc 2006; 13 (03) 245-252
    CrossrefPubMedGoogle Scholar
  • 18 Choi M, Lee HS, Park JH. Usability of academic electronic medical record application for nursing students' clinical practicum. Healthc Inform Res 2015; 21 (03) 191-195
    CrossrefPubMedGoogle Scholar
  • 19 Russ AL, Jahn MA, Patel H. , et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users' detection of medication discrepancies. J Biomed Inform 2018; DOI: 10.1016/j.jbi.2018.05.002.
    CrossrefPubMedGoogle Scholar
  • 20 Russ AL, Weiner M, Russell SA, Baker DA, Fahner WJ, Saleem JJ. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf 2012; 38 (12) 531-540
    PubMedGoogle Scholar
  • 21 Nielsen J. Demonstrate Thinking Aloud by Showing Users a Video. Available at: https://www.nngroup.com/articles/thinking-aloud-demo-video2014 . Accessed February 23, 2017
    PubMedGoogle Scholar
  • 22 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
    CrossrefPubMedGoogle Scholar
  • 23 Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive impairment among potential subjects for clinical research. Med Care 2002; 40 (09) 771-781
    CrossrefPubMedGoogle Scholar
  • 24 Nielsen J. Why you only need to test with 5 users. Useit Com Alertbox; 2000
    Google Scholar
  • 25 FDA. Applying Human Factors and Usability Engineering to Medical Devices. Silver Spring, MA: Food and Drug Administration; 2016
    Google Scholar
  • 26 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
    CrossrefPubMedGoogle Scholar
  • 27 Cooper A, Reimann R, Cronin D. About face 3: the essentials of interaction design. John Wiley & Sons; 2007
    Google Scholar
  • 28 The VA and Academic Medicine: Partners in Health Care, Training, and Research. (2014). Available at: https://www.aamc.org/download/385612/data/07182014.pdf
    PubMedGoogle Scholar
  • 29 Thompson MJ, Reilly JD, Valdez RS. Work system barriers to patient, provider, and caregiver use of personal health records: A systematic review. Appl Ergon 2016; 54: 218-242
    CrossrefPubMedGoogle Scholar
  • 30 Urowitz S, Wiljer D, Dupak K. , et al. Improving diabetes management with a patient portal: a qualitative study of diabetes self-management portal. J Med Internet Res 2012; 14 (06) e158
    CrossrefPubMedGoogle Scholar
  • 31 Jahn M, Caldwell BS. Developing Distributed Expertise Coordination Models of Diabetes Care Teams. In: Proceedings of 2018 Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA (In Press)
    PubMedGoogle Scholar
  • 32 U.S. Department of Veterans Affairs. VA Utilization Profile. Web 2017; 2017: 20
    PubMedGoogle Scholar