Local Investment in Training Drives Electronic Health Record User Satisfaction

Background and Significance

Despite decades of effort and billions of dollars of investment, the electronic health record (EHR) has not lived up to its potential to improve care, reduce costs, or revolutionize the experience for caregivers.[1] Many people point to poor technical usability as a root cause of these failings.[2] To find solutions to these challenges, the Arch Collaborative organizations (signed below) are working together to jointly study the feedback of their EHR users. After collecting responses from over 72,000 physicians, nurses, advanced practice professionals, and residents across 156 provider organizations, we are identifying key opportunities to derive greater value from the EHR investments that our organizations have collectively made (See details of survey methods in the [supplementary Material], available in the online version).

The extensive feedback from tens of thousands of users reveals critical gaps in users' understanding of how to optimize their EHR. Therefore, we as an industry have an opportunity to improve EHR adoption by investing in EHR learning and personalization support for caregivers. If health care organizations offered higher-quality educational opportunities for their care providers—and if providers were expected to develop greater mastery of EHR functionality—many of the current EHR challenges would be ameliorated.[3]

We came to this conclusion after discovering the wide variation in EHR experience that exists within all EHR customer bases (see [Table 1]). This variation cannot be ignored as it is not caused by differences in regulatory burden or programing design. We express concern that user competency often does not receive the strong focus it needs. These findings do not negate the need for EHR developers to continue to improve their user interfaces to be more intuitive, nor do they negate the critical need to reexamine the current regulatory and billing requirements that drive so much of the clinical documentation burden faced by providers today, but we believe that a greater focus on education and training is the overlooked opportunity that could enable EHR technology to drive substantial gains in the quadruple aim.[4] [5]

Table 1 Variation in experience by EHR

	Number of organizations with vendor deployed (and >10 surveys collected)	Lowest organization net EHR experience score	Highest organization net EHR experience score
Vendor 1	104	−13	73
Vendor 2	26	−51	43
Vendor 3	13	−58	31
Vendor 4	12	−41	54
Vendor 5	7	−26	42
Vendor 6	5	−15	21
Vendor 7	5	−60	−42

Abbreviation: EHR, electronic health record.

Variation Driven by Differences in User Experience More than Software

Multiple studies have indicated that users of the same EHR software can have significantly different experiences, from safety performance[6] [7] to mortality outcomes.[8] [9] Collaborative experience was decomposed to identify the variation that is explained at the EHR, organization, specialty, and user level (see [Table 2]). Less than 20% of all variation was explainable by the EHR in use, with over 50% of variation explained at the physician user level. Similarly, within the seven EHR solutions measured, a very unsuccessful provider organization was identified in each customer base, and a successful customer was identified in six of the seven customer bases (see [Table 1]).

Table 2 Variation decomposition by measurement unit

Group	Definition	Average net EHR experience score difference	Percent of total variation
Variation by EHR	Difference between collaborative average and average for each EHR	16.61 points	19.8
Variation by organization	Difference between average for each EHR and organizations using that EHR	12.74 points	15.1
Variation by specialty	Difference between average for each organization by EHR and specialties within that organization	12.1 points	14.4
Variation by user	Difference between average specialty experience in an organization with an EHR and individual user experiences within that specialty.	42.42 points	50.6

Abbreviation: EHR, electronic health record.

This variation at a user level in every software customer base indicates that no current enterprise software solution (as every major U.S. commercial EHR software solution was measured) has been identified that is so user friendly that it removes individual user variation in experience. Provider organizations seeking to create a positive user experience cannot expect the usability of the software to create consistent physician user success. Instead, factors unique to individual physician users must be identified to reduce negative variation.

EHR Training/Education is the Major Predictor of Positive User Experience

In the Arch Collaborative large dataset, the single greatest predictor of user experience is not which EHR a provider uses nor what percent of an organization's operating budget is spent on information technology, but how users rate the quality of the EHR-specific training they received. These data are consistent with multiple anecdotal case studies.[10] [11] [12] [13] [14] Across collaborative organizations, we have observed 475 instances in which two physicians of the same specialty using the same EHR in the same organization reported antipodal responses as to whether their EHR enables them to deliver high-quality health care (in each instance, one physician “strongly agreed” and the other “strongly disagreed”). In over 89% of these instances, the physician who strongly agreed also reported better training, more training efforts, or more effort expended in setting up EHR personalization.

Physicians Indicate Higher Quality EHR Training Drives Better Care

For a physician, feeling safe with the tools of medicine is about more than just the user interface. Physicians who report poor training are over 3.5 times more likely to report that their EHR does not enable them to deliver quality care (see [Table 3]).

Table 3 Training feedback on arch collaborative survey

	N	Percent of care providers who disagree or strongly disagree with the statement “This EHR enables me to deliver high-quality care”
All care providers who agree or strongly agree that their “initial training prepared them well”	15,425	10.3
All care providers who disagree or strongly disagree that their “initial training prepared them well”	9,739	38.4
Providers with scribes who agree or strongly agree that their “initial training prepared them well”	796	15.1
Providers with scribes who disagree or strongly disagree that their “initial training prepared them well”	562	44.4

Abbreviation: EHR, electronic health record.

While user interface matters, practice does also. A scalpel is a tool that has a very simple interface and use, but using it with confidence and safety requires knowledge of anatomy and surgical techniques coupled with practice to use it skillfully. In other industries, it is well recognized that education and training are of paramount importance to the successful use of professional-grade software. We need to recognize that this also holds true for EHRs and the practice of medicine. While documentation is a common burden, expert EHR users report a greater ability to find critical clinical information in the ever-growing pool of data available to caregivers today.

Standards Needed for EHR Education

When it comes to EHR education, it is critical to consider both the quantity and quality of the educational opportunities. Responses to the survey given to collaborative participants show that significant jumps in users' overall satisfaction with the EHR experience occur for every additional hour of initial EHR education they receive. Organizations requiring less than 4 hours of education for new providers appear to be creating a frustrating experience for their clinicians (see [Table 4]). These organizations have lower training satisfaction, lower self-reported proficiency, and are less likely to report that their EHR enables them to deliver quality care.

Table 4 Hours of training versus average next EHR experience score

Hours of required new-provider training	Organizations	Average net EHR experience score
<4 h	11	6
4 h	21	17.2
5–6 h	13	20.6
7–8 h	20	16.7
10–16 h	12	27.5

Abbreviation: EHR, electronic health record.

Table 5 Correlation between personalization adoption and net EHR experience score for users

Personalization/optimization	Number of personalization adopters	Net EHR experience score for personalization adopters	Number of personalization nonadopters	Net EHR experience score for personalization nonadopters	Percent adoption	NEES difference
Template personalization	25,240	29.3	6,196	−1.1	80	30.4
Order list personalization	18,583	31.2	11,468	10.3	62	20.8
Order set optimization	15,229	29.3	12,786	14.3	54	15.0
Navigation macro personalization	15,011	29.4	14,518	15.6	51	13.8
Filter personalization	14,289	35.3	16,298	11.9	47	23.4
Personalization of shortcuts	12,596	37.4	17,062	14.2	42	23.2
Layouts personalization	10,357	39.4	17,368	15.3	37	24.1
Personalization of report views	9,316	40.9	19,232	18.4	33	22.5
Personalization of sort orders for lists	6,664	40.2	19,450	17.7	26%	22.5

Abbreviations: EHR, electronic health record; NEES, net EHR experience score.

Table 6 Correlation between personalization adoption and NEES for organizations

Percent of personalization adopted by providers	Average net EHR experience score for organizations	Number of organizations
10–20	−21.5	3
20–30	−29-	6
30–40	−21.1	15
40–50	15.7	44
50–60	27.3	54
60–70	25.1	10

Abbreviations: EHR, electronic health record; NEES, net EHR experience score.

Regarding the quality of EHR education, there is huge variation between the collaborative organizations in terms of how they have structured training and educational programs and user perceptions of the quality of these programs, for both initial and ongoing training. Much has been published regarding educational best practices.[11] [12] [13] [14] This research indicates high variation between organizations in the quality of the EHR education in the outcomes this variation creates. While outside the scope of this research, significant variation exists between organizations in the quality and quantity of EHR education that is provided to users, resulting in significant variation in user experiences between organizations.

EHR Personalization Tools—A Key to Success

One key aspect of EHR use that we find significantly underutilized in EHR training is the power of user personalization. Personalization features are common with nearly every consumer technology (phones, computers, web browsers, and car computers) and help users get from a common technology what they want and need for their specific situations. One of the most consistent observations seen across the collaborative organizations is how powerful EHR personalization can be and how much adoption is lacking today.

Focusing organizational support and resources on ongoing education that helps providers personalize the retrieval and presentation of data leads to marked improvement in physician satisfaction.

The Future—More Investment in EHR Training Needed

Looking forward, we foresee that increasingly sophisticated decision support will be integrated into EHRs, positively affecting patient care in a dramatic way.[6] For this vision to become a reality, physicians will need to know the limits of their technology's advice in the same way that pilots know the limits of a plane's autopilot. Without clearly understanding the EHR's limits or how to use the technology, care providers will not trust the technology they work with.

While the Arch Collaborative research has convinced us that the greatest opportunity for progressing the value of the EHR currently lies in improved user training, this approach clearly needs to be balanced with a parallel focus on better designed and smarter software that can better meet nuanced needs of health care. For EHR software to revolutionize health care, both the software and the use of that complicated software need to progress in parallel. As users redouble their efforts to understand and utilize the full functionality available to them, EHR vendors can better anticipate the needs of end users trying to leverage the strengths of an intelligent electronic system.

We invite care providers to consider our claims that (1) delivering high-quality care in the 21st century requires caregivers to be well educated on the technology they utilize daily and (2) caregivers who do not understand EHR technology are a threat to quality care and will likely not realize any efficiency gains in using the EHR nor be able to use the technology fully to advance care quality.

Given these results, we advocate for health care delivery organizations to increase the EHR/technology education and support they make available to their providers. We also advocate for caregivers to adopt EHR technology expertise as a core competency of their profession. We are collectively encouraged that these changes, along with future improvements in EHR interfaces and technology, will unlock the potential of care providers working with information technology to revolutionize care quality and efficiency.

Clinical Relevance Statement

Dissatisfaction with the EHR is often associated with physician burnout. Physicians who reported receiving strong EHR training were significantly more likely to report feeling that the EHR enables them to deliver high-quality care (p < .01). Health care delivery organizations should invest in EHR training with a focus on adoption of EHR personalization tools.

Multiple Choice Questions

1. The single greatest predictor of EHR satisfaction at a given health care delivery organization is:

   • IT budget.

   • EHR training quality.

   • EHR vendor software choice.

   • EHR vendor software version.

   Correct Answer: The correct answer is option b. In this large dataset, user perception of EHR training quality was strongly associated with EHR satisfaction, more so than vendor choice or IT budget.

2. One EHR training best practice for achieving physician EHR mastery:

   • Classroom training of at least 8 hours.

   • Classroom training of at least 4 hours.

   • EHR personalization.

   • Vendor demos.

   Correct Answer: The correct answer is option c. In this large dataset, EHR personalization was strongly associated with a perception of high-quality training.

Conflict of Interest

T.D. is employed by KLAS, which founded and coordinates the Arch Learning Collaborative. The remaining authors have no real or apparent conflict of interest to disclose.

Protection of Human and Animal Subjects

This study is considered exempt from the Institutional Review Board review as defined by 45 CFR 46.101(b).

^* The list of Arch Collaborative members appears in [Supplementary Material] (available in the online version).

• References

• 1 Soumerai SB, Mahjumdar SR. 2009. A bad $50 billion bet. Washington, DC: The Washington Post;
• 2 Rosenbaum L. Transitional chaos or enduring harm? The EHR and the disruption of medicine. N Engl J Med 2015; 373 (17) 1585-1588
• 3 Sittig DF, Belmont E, Singh H. Improving the safety of health information technology requires shared responsibility: it is time we all step up. Healthc (Amst) 2018; 6 (01) 7-12
• 4 Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008; 27 (03) 759-769
• 5 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 6 Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood) 2010; 29 (04) 655-663
• 7 Chaparro JD, Classen DC, Danforth M, Stockwell DC, Longhurst CA. National trends in safety performance of electronic health record systems in children's hospitals. J Am Med Inform Assoc 2017; 24 (02) 268-274
• 8 Han YY, Carcillo JA, Venkataraman ST. , et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116 (06) 1506-1512
• 9 Longhurst CA, Parast L, Sandborg CI. , et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010; 126 (01) 14-21
• 10 Stevens LA, Pantaleoni JL, Longhurst CA. The value of clinical teachers for EMR implementations and conversions. Appl Clin Inform 2015; 6 (01) 75-79
• 11 Pantaleoni JL, Stevens LA, Mailes ES, Goad BA, Longhurst CA. Successful physician training program for large scale EMR implementation. Appl Clin Inform 2015; 6 (01) 80-95
• 12 Stroup K, Sanders B, Bernstein B, Scherzer L, Pachter LM. A new EHR training curriculum and assessment for pediatric residents. Appl Clin Inform 2017; 8 (04) 994-1002
• 13 Pereira AG, Kim M, Seywerd M, Nesbitt B, Pitt MB. ; Minnesota Epic101 Collaborative. Collaborating for competency-A model for single electronic health record onboarding for medical students rotating among separate health systems. Appl Clin Inform 2018; 9 (01) 199-204
• 14 Stevens LA, DiAngi YT, Schremp JD. , et al. Designing an individualized EHR learning plan for providers. Appl Clin Inform 2017; 8 (03) 924-935
• 15 Elkin PL. Human factors engineering in HI: so what? who cares? and what's in it for you?. Healthc Inform Res 2012; 18 (04) 237-241