Appl Clin Inform 2023; 14(04): 632-643
DOI: 10.1055/s-0043-1770900
Research Article

A Multiyear Survey Evaluating Clinician Electronic Health Record Satisfaction

Pamela M. Garabedian
1   Clinical Quality and IS Analysis, Mass General Brigham, Inc., Somerville, Massachusetts, United States
Angela Rui
2   Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Massachusetts, United States
Lynn A. Volk
1   Clinical Quality and IS Analysis, Mass General Brigham, Inc., Somerville, Massachusetts, United States
Bridget A. Neville
2   Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Massachusetts, United States
Stuart R. Lipsitz
2   Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Massachusetts, United States
3   Harvard Medical School, Harvard University, Ariadne Labs, Boston, Massachusetts, United States
Michael J. Healey
2   Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Massachusetts, United States
4   Harvard Medical School, Boston, Massachusetts, United States
David W. Bates
2   Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, Massachusetts, United States
4   Harvard Medical School, Boston, Massachusetts, United States
5   Harvard School of Public Health, Harvard University, Boston, Massachusetts
› Author Affiliations


Objectives We assessed how clinician satisfaction with a vendor electronic health record (EHR) changed over time in the 4 years following the transition from a homegrown EHR system to identify areas for improvement.

Methods We conducted a multiyear survey of clinicians across a large health care system after transitioning to a vendor EHR. Eligible clinicians from the first institution to transition received a survey invitation by email in fall 2016 and then eligible clinicians systemwide received surveys in spring 2018 and spring 2019. The survey included items assessing ease/difficulty of completing tasks and items assessing perceptions of the EHR's value, usability, and impact. One item assessing overall satisfaction and one open-ended question were included. Frequencies and means were calculated, and comparison of means was performed between 2018 and 2019 on all clinicians. A multivariable generalized linear model was performed to predict the outcome of overall satisfaction.

Results Response rates for the surveys ranged from 14 to 19%. The mean response from 3 years of surveys for one institution, Brigham and Women's Hospital, increased for overall satisfaction between 2016 (2.85), 2018 (3.01), and 2019 (3.21, p < 0.001). We found no significant differences in mean response for overall satisfaction between all responders of the 2018 survey (3.14) and those of the 2019 survey (3.19). Systemwide, tasks rated the most difficult included “Monitoring patient medication adherence,” “Identifying when a referral has not been completed,” and “Making a list of patients based on clinical information (e.g., problem, medication).” Clinicians disagreed the most with “The EHR helps me focus on patient care rather than the computer” and “The EHR allows me to complete tasks efficiently.”

Conclusion Survey results indicate room for improvement in clinician satisfaction with the EHR. Usability of EHRs should continue to be an area of focus to ease clinician burden and improve clinician experience.

Protection of Human and Animal Subjects

This project was undertaken as a quality improvement initiative at Mass General Brigham and as such was not formally supervised by the Institutional Review Board per their policies.

Publication History

Received: 05 January 2023

Accepted: 12 May 2023

Article published online:
16 August 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Office of the National Coordinator for Health Information Technology. Adoption of Electronic Health Records by Hospital Service Type 2019–2021, Health IT Quick Stat #60. Washington: U.S.: Department of Health and Human Services; 2022
  • 2 Burke HB, Sessums LL, Hoang A. et al. Electronic health records improve clinical note quality. J Am Med Inform Assoc 2015; 22 (01) 199-205
  • 3 Cimino JJ. Improving the electronic health record–are clinicians getting what they wished for?. JAMA 2013; 309 (10) 991-992
  • 4 Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform 2016; l (Suppl 1, Suppl 1): S48-S61
  • 5 Lawrence JE, Cundall-Curry D, Stewart ME, Fountain DM, Gooding CR. The use of an electronic health record system reduces errors in the National Hip Fracture Database. Age Ageing 2019; 48 (02) 285-290
  • 6 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
  • 7 Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 319 (12) 1276-1278
  • 8 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
  • 9 U.S. General Services Administration. System Usability Scale (SUS). Washington: U.S. Government Printing Office; 2006. .
  • 10 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
  • 11 Gomes KM, Ratwani RM. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Netw Open 2019; 2 (12) e1916651
  • 12 Krousel-Wood M, McCoy AB, Ahia C. et al. Implementing electronic health records (EHRs): health care provider perceptions before and after transition from a local basic EHR to a commercial comprehensive EHR. J Am Med Inform Assoc 2018; 25 (06) 618-626
  • 13 Huang C, Koppel R, McGreevey III JD, Craven CK, Schreiber R. Transitions from one electronic health record to another: challenges, pitfalls, and recommendations. Appl Clin Inform 2020; 11 (05) 742-754
  • 14 Hanauer DA, Branford GL, Greenberg G. et al. Two-year longitudinal assessment of physicians' perceptions after replacement of a longstanding homegrown electronic health record: does a J-curve of satisfaction really exist?. J Am Med Inform Assoc 2017; 24 (e1): e157-e165
  • 15 Ehrlich JR, Michelotti M, Blachley TS. et al. A two-year longitudinal assessment of ophthalmologists' perceptions after implementing an electronic health record system. Appl Clin Inform 2016; 7 (04) 930-945
  • 16 Kjeldskov J, Skov MB, Stage J. A longitudinal study of usability in health care: does time heal?. Int J Med Inform 2010; 79 (06) e135-e143
  • 17 Tutty MA, Carlasare LE, Lloyd S, Sinsky CA. The complex case of EHRs: examining the factors impacting the EHR user experience. J Am Med Inform Assoc 2019; 26 (07) 673-677
  • 18 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
  • 19 Williams DC, Warren RW, Ebeling M, Andrews AL, Teufel Ii RJ. Physician use of electronic health records: survey study assessing factors associated with provider reported satisfaction and perceived patient impact. JMIR Med Inform 2019; 7 (02) e10949
  • 20 Tajirian T, Stergiopoulos V, Strudwick G. et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res 2020; 22 (07) e19274
  • 21 Yan Q, Jiang Z, Harbin Z, Tolbert PH, Davies MG. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc 2021; 28 (05) 1009-1021
  • 22 Meyerhoefer CD, Sherer SA, Deily ME. et al. Provider and patient satisfaction with the integration of ambulatory and hospital EHR systems. J Am Med Inform Assoc 2018; 25 (08) 1054-1063
  • 23 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
  • 24 Harris PA, Taylor R, Minor BL. , et al. ; The REDCap Consortium. Building an international community of software partners. J Biomed Inform 2019; 95: 103208
  • 25 Limesurvey Gmb H. . LimeSurvey: An OpenSource survey tool. Hamburg, Germany: LimeSurvey, 2006.
  • 26 Agency for Healthcare Research and Quality. Primary Care Information Project (PCIP) Post-Electronic Health Record Implementation: Survey of Providers. New York, NY: New York City Department of Health and Mental Hygiene; 2010
  • 27 Adler KG, Edsall RL. The 2012 FPM survey of user satisfaction with EHR systems. Fam Pract Manag 2012; 19 (03) 19-20
  • 28 Marcilly R, Ammenwerth E, Roehrer E, Niès J, Beuscart-Zéphir MC. Evidence-based usability design principles for medication alerting systems. BMC Med Inform Decis Mak 2018; 18 (01) 69
  • 29 Marien S, Legrand D, Ramdoyal R. et al. A user-centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network. Int J Med Inform 2019; 126: 138-146
  • 30 Horsky J, Phansalkar S, Desai A, Bell D, Middleton B. Design of decision support interventions for medication prescribing. Int J Med Inform 2013; 82 (06) 492-503
  • 31 Garabedian PM, Wright A, Newbury I. et al. Comparison of a prototype for indications-based prescribing with 2 commercial prescribing systems. JAMA Netw Open 2019; 2 (03) e191514
  • 32 Wang L, Blackley SV, Blumenthal KG. et al. A dynamic reaction picklist for improving allergy reaction documentation in the electronic health record. J Am Med Inform Assoc 2020; 27 (06) 917-923
  • 33 Nanji KC, Garabedian PM, Langlieb ME. et al. Usability of a perioperative medication-related clinical decision support software application: a randomized controlled trial. J Am Med Inform Assoc 2022; 29 (08) 1416-1424
  • 34 Chokshi SK, Belli HM, Troxel AB. et al. Designing for implementation: user-centered development and pilot testing of a behavioral economic-inspired electronic health record clinical decision support module. Pilot Feasibility Stud 2019; 5: 28
  • 35 McCoy AB, Russo EM, Johnson KB. et al. Clinician collaboration to improve clinical decision support: the Clickbusters initiative. J Am Med Inform Assoc 2022; 29 (06) 1050-1059
  • 36 English EF, Holmstrom H, Kwan BW. et al. Virtual sprint outpatient electronic health record training and optimization effect on provider burnout. Appl Clin Inform 2022; 13 (01) 10-18
  • 37 Ratwani RM, Sinsky CA, Melnick ER. . Closing the Electronic Health Record Usability Gap. Bill of Health, Harvard Law School (; 2020
  • 38 Pierce RP, Eskridge BR, Ross B, Day MA, Dean B, Belden JL. Improving the user experience with discount site-specific user testing. Appl Clin Inform 2022; 13 (05) 1040-1052
  • 39 Rizvi RF, Marquard JL, Hultman GM, Adam TJ, Harder KA, Melton GB. Usability evaluation of electronic health record system around clinical notes usage-an ethnographic study. Appl Clin Inform 2017; 8 (04) 1095-1105
  • 40 Hettinger AZ, Melnick ER, Ratwani RM. Advancing electronic health record vendor usability maturity: progress and next steps. J Am Med Inform Assoc 2021; 28 (05) 1029-1031
  • 41 Kohavi R, Thomke S. The Surprising Power of Online Experiments. Harvard business review 2017; 95 (05) 74-82
  • 42 Austrian J, Mendoza F, Szerencsy A. et al. Applying A/B testing to clinical decision support: rapid randomized controlled trials. J Med Internet Res 2021; 23 (04) e16651
  • 43 Kohavi R, Tang D, Xu Y, Hemkens LG, Ioannidis JPA. Online randomized controlled experiments at scale: lessons and extensions to medicine. Trials 2020; 21 (01) 150