Conversion of Provider EMR Training from Instructor-Led Training to eLearning at an Academic Medical Center

Summary

This case study overviews the conversion of provider training of the electronic medical record (EMR) from an instructor-led training (ILT) program to eLearning at an Academic Medical Center (AMC). This conversion provided us with both a useful training tool and the opportunity to maximize efficiency within both our training and optimization team and organization. eLearning Development Principles were created and served as a guide to assist us with designing an eLearning curriculum using a five step process. The result was a new training approach that allowed learners to complete training at their own pace, and even test out of sections based on demonstrated competency. The information we have leads us to believe that a substantial return on our investment can be obtained from the conversion with positive impacts that have served as the foundation for the future of end user EMR training at our AMC.

Citation: Sharp K, Williams M, Aldrich A, Bogacz A, Denier S, McAlearney AS. Conversion of Provider EMR Training from Instructor Led Training to eLearning at an Academic Medical Center. Appl Clin Inform 2017; 8: 754–762 https://doi.org/10.4338/ACI-2017-03-CR-0040

Keywords

Electronic health records and systems - Knowledge translation - Internet and the Web technology - Strategies for health IT training - Employee orientation program

1. Background and Significance

The promise of electronic medical records (EMRs) to improve quality, productivity, and efficiency in health care organizations, as well as incentives and penalties for use and nonuse of such systems, has led more and more organizations to implement EMRs [[1], [2]]. However, this implementation has been challenging [[3], [4]], and subject to both individual and organization-level barriers for use [[5]–[9]]. Research has consistently demonstrated that training on health information technology can positively influence a provider’s willingness and ability to use EMRs [[10], [11]]. However, how to best train busy clinicians to incorporate and efficiently use an EMR remains an outstanding question.

A variety of factors can influence EMR use, including the timing of training, how well training is targeting user needs, and accessibility of on-site support [[12]–[14]]. A growing body of research examines the benefits of different training approaches [[13], [15]–[17]], but most of the described methods are instructor-led. Instructor-led training (ILT) typically involves a computer lab and limited session dates. ILT can be expensive and time-consuming [[18]–[20]], and this ‘one size fits all’ approach falls short for some end users.

The eLearning method of EMR training offers the potential for long-term cost savings as compared to ILT [[19]]. eLearning’ refers to user-directed online education, or the use of technology to deliver learning solutions with high levels of interactivity, flexibility, and communication [[21]]. Evidence demonstrating the effectiveness of eLearning is growing [[22]], yet the experience of converting EMR training from an instructor-led to an eLearning approach has not been described in the literature. We examined the impact and return on investment of conversion to an eLearning format at our large academic medical center (AMC).

2. Objectives

The system-wide implementation of the current EMR at The Ohio State University Wexner Medical Center (OSUWMC) occurred in 2011 [[23]], led by the organization’s Information Technology (IT) department. OSUWMC’s IT department includes a Training and Optimization team established to provide training for new end users, assist current staff in optimizing their EMR use, and ensure that end users remain current on evolving functionality. After the Go-Live event, this Training and Optimization team redesigned and streamlined the organization’s EMR education approach.

Under the revised ILT curriculum, providers new to OSUWMC experienced “layered” EMR training. This curriculum started with essential content and continued with additional training based on the user’s specialty. These ILT sessions were offered twice a month.

We were onboarding an increasing number of physicians and residents with prior EMR experience. Many of these physicians were already familiar with the system in use at OSUWMC when they arrived for training. Therefore, we were challenged to modify our training to meet needs based on varying levels of prior system experience.

eLearning introduces the ability to meet learners where they are by delivering content dependent on learners’ demonstrated EMR competency. We hypothesized that a transition from ILT to eLearning would increase user satisfaction and reduce training time, freeing more time for revenue-producing patient care.

3. Methods

This case study was conducted at The Ohio State University Wexner Medical Center (OSUWMC), an AMC comprised of six hospitals, two campuses, and 46 outpatient sites in Columbus, Ohio. Close to 24,000 staff members utilize the EMR system, including approximately 1,100 attending physicians, 800 residents and 600 advanced practice professionals (APPs). All of these physicians and APPs recently transitioned from ILT to eLearning for the training required prior to obtaining access to OSUWMC’s electronic medical record.

We designed our ILT using a competency-based approach, requiring learners to demonstrate knowledge about and skill in using the EMR. This foundation created an easier transition from ILT to self-paced eLearning. We also designed workflow and scenarios into our ILT to enhance transfer of training to patient care. In the transition to eLearning, we identified a set of eLearning Development Principles (►[Table 1]) to ensure an experience that flows and engages learners.

Table 1

Development Principles for eLearning Success

Guiding eLearning Principles
Make sure there is consistent presentation of material
Use a project management system for tracking development
Have leadership and operational support
Utilize a standard development process
Enforce stringent quality assurance reviews
Ensure staff have time, skills and software resources to be successful
Use a robust Learning Management System
Make smart choices about investments
Adhere to design requirements: Standard style guides and templates No audio due to muted computers in patient care areas and sound maintenance considerations Short and digestible content, with no more than 30 minutes per lesson Each course typically includes multiple lessons including guided and un-guided interactions

Development of eLearning requires specialized skills that are different from those needed for ILT development. We did not hire any new team members. Instead, we identified new roles and dedicated staff time to developing expertise in these roles (►[Table 2]).

Table 2

Roles, Duties and Skill Sets Required for eLearning Oversight and Production

Role (% of Total Development Time)	Primary Duties	Skill Set Required
eLearning Team Lead and Project Manager (3%)	Manages resources and overall project deliverables	Project management
Production Manager/ Instructional Design Lead (5%)	Project manages storyboard and Captivate production Project manages quality review process Instructional design review of storyboards Maintains templates for storyboards, Captivate and Lectora	eLearning design Instructional design Captivate development Lectora development
Principal Trainer (11%)	Owns content Determines design approach for curriculum Completes content and quality review of storyboards, interactive demos, final courses Designs test questions, exercise assessments	Subject matter expert
eLearning Designer/ Developer (78%)	Designs storyboards Uses storyboard blueprint, screen shots and Captivate to produce interactive demos Updates Captivate lessons per quality review until final version is accepted Uses Lectora template to produce courses	Storyboarding PowerPoint Captivate Lectora
Technical Editor (3%)	Owns writing standards Completes quality review of storyboards and Captivate files for standards adherence, structure and consistency May develop storyboard and Captivate files	Development and application of writing standards

Our eLearning curriculum design is modular. Each curriculum has a set of courses and each course has a set of lessons, making it easy for first time learning and subsequent review of specific topics. Courses are taken in workflow order. Each lesson is designed to teach EMR functionality and application in a typical scenario for a role (i.e. ambulatory or inpatient clinician). Lessons allow learners to interact with the content as if they are in the live EMR. Knowledge assessments are integrated into each course to ensure that progress within a curriculum is based on demonstrated knowledge. Additionally, learners are required to demonstrate their knowledge in a competency assessment during a 1:1 onboarding session with a trainer. The competency assessment process is the same as for ILT.

It was essential to create a structured eLearning design/development process that allowed for accurate conversion of a large number of classes in an efficient, economical manner. Our process included five steps.

Step 1: Analysis

In the first phase of each class conversion, we restructured ILT content to align with our eLearning Development Principles (►[Table 1]). The deliverable was a design approach document that mapped classroom topics to eLearning courses and lessons. Most classes mapped to between three and five eLearning courses.

Step 2: Design and Storyboarding

The eLearning Developer worked in close collaboration with the Principal Trainer, or content owner, to convert classroom content to visual storyboards for each lesson. A storyboard template provided a standard construct for developers and trainers to use in the design of each lesson while allowing for creativity in the presentation of content. Using the storyboard template for each lesson, the eLearning developer proceeded through six activities:

1. Follow a classroom script in the EMR to take screen shots;

2. Insert the screen shots in a Microsoft PowerPoint document;

3. Lay out the text for each slide, including important notes and callouts;

4. Look for opportunities to introduce learner engagement activities;

5. Work with the Principal Trainer to finalize each storyboard; and

6. Save final screen shots for production in Adobe Captivate.

When a storyboard draft was complete, the Principal Trainer, Project Manager, Instructional Design Lead and Technical Editor reviewed the draft for quality. They evaluated the content and ensured adherence to design principles and eLearning standards.

Step 3: Interactive Lesson Development

Once the storyboard was approved, eLearning developers converted it to Adobe Captivate, where the lessons came to life. Developers used a Captivate template that set design standards for all interactive demos.

When the Developer completed a Captivate file, the Principal Trainer, Instructional Designer and Technical Editor conducted a review to confirm that all content was transferred accurately from the storyboard, and the interactive demos were operating correctly.

Step 4: Course Development

When all of the interactive demos were complete, the Instructional Designer used Lectora to publish each course for use in a Learning Management System (LMS). The Lectora portion of each course includes information about audience and objectives, a menu of interactive demo lessons, and assessment.

As part of course development, the Principal Trainer designed assessment questions, including both multiple choice and scenario-based questions that test learners on applying what has been learned. Our assessments require learners to answer 80 percent of questions correctly to pass. If learners do not achieve 80 percent, they can review the course and attempt the assessment again until they pass. A course is ready for deployment after a final quality assurance checks by a subject matter expert, instructional designer, technical editor, and an expert in the EMR software system in use (Epic™).

Step 5: Deployment

New clinicians were enrolled in the curriculum via the LMS. For the first several months after deployment, we had clinicians complete eLearning in the classroom to make sure the courses worked as intended. The Principal Trainer facilitated these sessions to troubleshoot any issues that emerged. Once we were confident in the eLearning curriculum, we developed instructions so that clinicians could complete courses from any location, any time, using the password-protected LMS. Clinicians are able to call the Medical Center help desk with questions. Technical eLearning/LMS issues are addressed by the first line help desk staff, and content issues are routed to an on-call trainer.

During the initial rollout, several issues emerged related to coordination with approximately 100 staff across the medical center with responsibility for onboarding providers. We had to assure that these staff knew who to contact at the Training Center to have modules assigned to new providers, and that the Training Center staff had a detailed decision tree in place to accurately assign the correct training to providers based on their specialty and practice locations. Onboarding staff were also educated on the need to schedule time in the providers’ first days of clinic for 1:1 onboarding sessions and competency assessment.

4. Results

All new providers now use eLearning instead of ILT for EMR training. The new structure for basic provider training consists of four curricula: Basic, Standard, Specialty, and Personalized Content. The online structure allows users with previous EMR system experience to test out of Basic Content, saving physician time and organizational resources.

The cost of converting a course to eLearning quickly adds up, in our case, quickly exceeding an investment of $40,000 to $50,000 for the primary provider eLearning curriculum. However, the potential for savings comes from reduced non-productive time that clinicians spend in classroom training [[19]].

To maximize our return on investment, we were selective about where to invest in conversion of ILT to eLearning. Some classes continue to be taught in person due to specialized content, or because the class impacted so few users that investment in development time could not be recuperated. In deciding where to implement replace ILT with eLearning, we considered the volume of staff trained, the hourly rate of affected staff, and EMR workflow stability.

We train approximately 400 high hourly rate providers (physicians and advanced practice professionals) each year. Hours gained can be applied to revenue-producing activity, making our provider classes good choices for eLearning conversion. Our first eLearning courses showed a significant reduction in training time compared to the ILT format (►[Table 3]). Because eLearning is self-paced and can meet providers where they are, providers completed training via eLearning approximately 45% faster than classroom training. Given the volume of new providers in our organization, these time savings can be easily translated into additional patient care time and a significant revenue opportunity. This conversion also enabled us to return time to our trainers for 1:1 end user optimization work.

Table 3

Gains

Who	1 Year Gain (hours)	Available for
Faculty	700	Clinical Service (175,5 day clinics)
Residents	920	Clinical Service (230,5 day clinics)
Trainers	450	Support and Optimization

Our standard practice is to survey learners after ILT, and we continue to obtain feedback after eLearning courses. In addition to the return on investment, it is important that providers like the eLearning format. Learners were surveyed upon completion of the eLearning on how we could improve the training experience. Responses were positive overall, with providers reporting that eLearning was comprehensive and easy to navigate. Providers with previous experience using our EMR found the self-paced format especially beneficial because it allowed them to move more quickly or test out of training. A few clinicians noted that they would like to be able to ask questions, which is not directly possible within the LMS. We also noted that the hands-on practice exercises, when not programmed as requirements for module completion, were sometimes being neglected. Our developers are working to address these challenges in future eLearning updates.

5. Discussion

Successful conversion of our provider trainin from ILT to eLearning took considerable planning and resources. The goal was to deliver a high-quality product that would support our providers with practical EMR training in less time. The literature on continuing education in the health professions has shown that online format can be just as effective at achieving learning outcomes as classroom-based instruction [[24], [25]]. The flexibility of the online format is valued by busy clinicians [[26]–[28]]. We noted additional positive impacts on the organization as a result of this conversion, including time savings.

Many new clinicians arrive at OSUWMC with experience using the same EMR at other agencies. It is important to minimize onboarding time so clinicians can be productive as soon as they are on site. Completing training remotely via eLearning, even before arriving at our facility, allows clinicians to begin patient care earlier.

6. Conclusions

Converting provider training to eLearning impacted our organization in three major areas. First, our providers now spend less time in class and can learn through personalized, self-paced lessons. Learner satisfaction increased with this delivery method, which has also created a positive image for our organization during provider onboarding. Second, our Training and Optimization staff are out of the classroom and have time to provide more on-site end user optimization. Finally, at the organizational level, the return on our investmet in conversion was robust, reflected in both time and cost savings. Clinicians spend less time in training and more time in clinical roles.

Our eLearning development process will continue to be refined. We are currently pursuing the conversion to eLearning for other clinical roles. We plan to make our next conversion decisions based on the criteria we have established for return on investment and adherence to our eLearning Development Principles.

Multiple Choice Questions

1. In changing from ILT to eLearning, what are the key factors for an organization to include?

• A structured development and QA process

• Focus conversion effort on small specialty classes for maximum benefit

• The need to cover all policies and procedures in the training

• The importance of outsourcing the development process

Correct answer: A – While there are many benefits to eLearning, development of eLearning modules can be expensive and time consuming. The cost of poor quality, in terms of rework and/or incorrect training, is high and should be avoided. A structured development and QA process can assure that eLearning modules are high quality and correct. 2. Why would an organization consider switching to eLearning?

• Assure that training is “one size fits all”

• Assure that all learners complete training on site

• Reduce training time due to self-paced nature

• eLearning development is quick and inexpensive

Correct answer: C – Self-directed eLearning allows learners to complete training at their own pace, allowing those who have had experience with this particular software to go through the material more quickly, and those who need more time can complete it at their own pace without feeling rushed.

Conflict of Interest

The authors have no conflicts of interest to report.

Clinical Relevance Statement

Converting provider EMR training to an eLearning format has the potential to increase learner satisfaction while yielding considerable savings to the organization in terms of cost and clinical time savings.

Protection of Human Subjects

This activity did not involve research on human subjects.

• References

• 1 Joseph S, Sow M, Furukawa MF, Posnack S, Chaffee MA. HITECH spurs EHR vendor competition and innovation, resulting in increased adoption. The American journal of managed care 2014; 20 (09) 734-40.
• 2 Office of the National Coordinator for Health Information Technology.. Update on the adoption of health information technology and related efforts to facilitate the electronic use and exchange of health information. 2016 [cited 2016 January 5]; Available from: https://dashboard.healthit.gov/report-to-congress/2015-update-adoption-health-information-technology.php
• 3 Iveroth E, Fryk P, Rapp B. Information technology strategy and alignment issues in health care organizations. Health care management review 2013; 38 (03) 188-200.
• 4 Lapointe L, Rivard S. A Multilevel Model of Resistance to Information Technology Implementation. MIS Quarterly 2005; 29 (03) 461-91.
• 5 Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC health services research 2010; 10: 231.
• 6 Kaplan B, Harris-Salamone KD. Health IT success and failure: recommendations from literature and an AMIA workshop. JAMIA 2009; 16 (03) 291-9.
• 7 Kruse CS, Regier V, Rheinboldt KT. Barriers over time to full implementation of health information exchange in the United States. JMIR medical informatics 2014; 2 (02) e26.
• 8 Mennemeyer ST, Menachemi N, Rahurkar S, Ford EW. Impact of the HITECH Act on physicians’ adoption of electronic health records. Journal of the American Medical Informatics Association : JAMIA 2016; 23 (02) 375-9.
• 9 Miller RH, Sim I. Physicians’ use of electronic medical records: barriers and solutions. Health affairs (Project Hope) 2004; 23 (02) 116-26.
• 10 Anderson LK, Stafford CJ. The „big bang“ implementation: not for the faint of heart. Computers in nursing 2002; 20 (Suppl. 01) 14-20. quiz –2.
• 11 Bredfeldt CE, Awad EB, Joseph K, Snyder MH. Training providers: beyond the basics of electronic health records. BMC health services research 2013; 13: 503.
• 12 Lorenzi NM, Kouroubali A, Detmer DE, Bloomrosen M. How to successfully select and implement electronic health records (EHR) in small ambulatory practice settings. BMC medical informatics and decision making 2009; 9: 15.
• 13 Terry AL, Thorpe CF, Giles G, Brown JB, Harris SB, Reid GJ, Thind A, Stewart M. Implementing electronic health records: Key factors in primary care. Canadian family physician Medecin de famille canadien 2008; 54 (05) 730-6.
• 14 Whittaker AA, Aufdenkamp M, Tinley S. Barriers and facilitators to electronic documentation in a rural hospital. Journal of nursing scholarship: an official publication of Sigma Theta Tau International Honor Society of Nursing 2009; 41 (03) 293-300.
• 15 Carr DM. A team approach to EHR implementation and maintenance. Nursing management 2004; 35 (Suppl. 05) 15-6 24.
• 16 Dastagir MT, Chin HL, McNamara M, Poteraj K, Battaglini S, Alstot L. Advanced proficiency EHR training: effect on physicians’ EHR efficiency, EHR satisfaction and job satisfaction. AMIA Annual Symposium proceedings AMIA Symposium 2012; 2012: 136-43.
• 17 Nicklaus J, Kusser J, Zessin J, Amaya M. Transforming Education for Electronic Health Record Implementation. Journal of continuing education in nursing 2015; 46 (08) 359-63.
• 18 Fleming NS, Culler SD, McCorkle R, Becker ER, Ballard DJ. The financial and nonfinancial costs of implementing electronic health records in primary care practices. Health affairs (Project Hope) 2011; 30 (03) 481-9.
• 19 Jimenez A. E-learning supports EHR implementations. In addition to meaningful use, we need to define meaningful training. Health management technology 2010; 31 (11) 22-3.
• 20 Peck AD. EHR implementation: training pays dividends. Medical economics 2013; 90 (14) 53-6.
• 21 Macdonald CJ, Stodel EJ, Chambers LW. An online interprofessional learning resource for physicians, pharmacists, nurse practitioners, and nurses in long-term care: benefits, barriers, and lessons learned. Informatics for health & social care 2008; 33 (01) 21-38.
• 22 Bramble WJ, Panda SK. Economics of distance and online learning: theory, practice, and research. New York:: Routledge; 2008
• 23 Bentley T, Rizer M, McAlearney AS, Mekhjian H, Siedler M, Sharp K. et al. The journey from precontemplation to action: Transitioning between electronic medical record systems. Health care management review 2016; 41 (01) 22-31.
• 24 Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008; 300 (10) 1181-96.
• 25 Wutoh R, Boren SA, Balas EA. eLearning: a review of Internet-based continuing medical education. The Journal of continuing education in the health professions 2004; 24 (01) 20-30.
• 26 Curran VR, Fleet LJ, Kirby F. A comparative evaluation of the effect of Internet-based CME delivery format on satisfaction, knowledge and confidence. BMC medical education 2010; 10: 10.
• 27 Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC medical education 2010; 10: 12.
• 28 Young KJ, Kim JJ, Yeung G, Sit C, Tobe SW. Physician preferences for accredited online continuing medical education. The Journal of continuing education in the health professions 2011; 31 (04) 241-6.