Impact of Electronic Health Record Implementation on Ophthalmology Trainee Time Expenditures

Abstract

Objective Electronic health records (EHRs) are widely adopted, but the time demands of EHR use on ophthalmology trainees are not well understood. This study evaluated ophthalmology trainee time spent on clinical activities in an outpatient clinic undergoing EHR implementation.

Design Prospective, manual time-motion observations of ophthalmology trainees in 2018.

Participants Eleven ophthalmology residents and fellows observed during 156 patient encounters.

Methods Prospective time-motion study of ophthalmology trainees 2 weeks before and 6 weeks after EHR implementation in an academic ophthalmology department. Manual time-motion observations were conducted for 11 ophthalmology trainees in 6 subspecialty clinics during 156 patient encounters. Time spent documenting, examining, and talking with patients were recorded. Factors influencing time requirements were evaluated using linear mixed effects models.

Main Outcome Measures Total time spent by ophthalmology residents and fellows per patient, time spent on documentation, examination, and talking with patients.

Results Seven ophthalmology residents and four ophthalmology fellows with mean (standard deviation) postgraduate year of 3.7 (1.2) were observed during 156 patient encounters. Using paper charts, mean total time spent on each patient was 11.6 (6.5) minutes, with 5.4 (3.5) minutes spent documenting (48%). After EHR implementation, mean total time spent on each patient was 11.8 (6.9) minutes, with 6.8 (4.7) minutes spent documenting (57%). Total time expenditure per patient did not significantly change after EHR implementation (+0.17 minutes, 95% confidence interval [CI] for difference in means: –2.78, 2.45; p = 0.90). Documentation time did not change significantly after EHR implementation in absolute terms (+1.42 minutes, 95% CI: –3.13, 0.29; p = 0.10), but was significantly greater as a proportion of total time (48% on paper to 57% on EHR; +9%, 95% CI: 2.17, 15.83; p = 0.011).

Conclusion Total time spent per patient and absolute time spent on documentation was not significantly different whether ophthalmology trainees used paper charts or the recently implemented EHR. Percentage of total time spent on documentation increased significantly with early EHR use. Evaluating EHR impact on ophthalmology trainees may improve understanding of how trainees learn to use the EHR and may shed light on strategies to address trainee burnout.

Keywords

electronic health record - documentation time - talking time - examination time - time-motion - ophthalmology trainee - paper chart - burnout

Electronic health record (EHR) use has become commonplace in residency training programs across the United States. While much focus has been placed on the effects of EHR use on attending physician workload, how EHR use impacts residency training is not as well studied.[1] [2] Efficiency in navigating and documenting in the EHR is of paramount importance as work-hour restrictions are enforced across training programs and as more evidence emerges connecting EHR use to resident burnout.[3] [4] This discussion is increasingly relevant as more studies examine EHR use and ophthalmology, with recent data suggesting that the complexity and time associated with using an EHR are especially concerning to ophthalmologists.[5] [6]

Several studies have investigated the impact of an EHR on residents in specialties including otolaryngology, emergency medicine, internal medicine, pediatrics, psychiatry, orthopaedics, and general surgery.[7] [8] [9] [10] [11] [12] [13] Many of these studies focused their attention on how residents and fellows spent their time caring for patients in an inpatient setting, while some focused on the effect of implementation of a note template within the EHR.[14] [15] There are few studies investigating ophthalmology trainee workflow as it relates to an EHR, but these focus on how the presence of an ophthalmology trainee affects overall clinic flow in which an EHR is present or on the implementation of a template within the EHR.[16] [17]

The purpose of this study was to examine the ways in which ophthalmology residents and fellows (“trainees”) utilize the EHR within patient encounters while in an outpatient clinic setting. The EHR is a powerful tool that affects how trainees learn and practice ophthalmology and how they interact with patients. To understand the impact of the EHR on this training, we first need to understand how the EHR is used. Understanding how trainees utilize the EHR by evaluating the time dedicated to EHR use compared with other clinical activities may shed light on how to better optimize EHRs for both future trainees and physicians in general. Our study is unique in that it collected distinct time-motion data following trainees before and after implementation of an EHR, providing a look into how trainees spend their time in clinic on a second-to-second basis. Detailed information about time expenditure within individual patient encounters is important because clinical activities play a large role in trainee education. Analyzing this data allows for a granular analysis of how much time trainees spend on a variety of clinical tasks, providing us with an opportunity to optimize trainee education and patient care. The University of California San Diego (UCSD) Shiley Eye Institute and Viterbi Family Department of Ophthalmology recently underwent a transition from paper charts to an EHR for documenting outpatient clinical encounters. By conducting time-motion analyses of paper-based clinical workflows and of early post-EHR implementation workflows of ophthalmology trainees in outpatient clinical encounters, we are better able to understand how an EHR impacts ophthalmology trainee time expenditure in clinic.

Methods

The UCSD Shiley Eye Institute and Viterbi Family Department of Ophthalmology is an academic ophthalmology department that transitioned from paper charts to an EHR (Epic Kaleidoscope; Epic Systems, Verona, WI) for outpatient clinic encounters for half of its faculty in September 2018. The department had already previously implemented the EHR for patient registration and scheduling as well as in the surgical suite. This study adhered to the tenets of the Declaration of Helsinki and was approved by the UCSD Institutional Review Board with waiver of documented consent and HIPAA exemption.

This was a prospective time-motion study of outpatient encounters in the clinics of attending ophthalmologists who involved trainees in direct patient care. In the typical workflow of an attending clinic, the trainee performs an examination and assessment of the patient prior to the attending performing an examination and assessment of the patient. Time-motion observations were performed 2 to 3 weeks prior to EHR implementation in September 2018 and again 5 to 6 weeks after EHR implementation in November 2018, after all temporary onsite support staff involved in the initial implementation efforts were no longer regularly present in clinic. Time-motion data were collected by SLB, HEG, and trained student observers using a standardized electronic iPad-based data entry tool (Numbers; Apple, Inc, Cupertino, CA). A templated spreadsheet tool containing easy-to-use dropdown menus of time-stamped, observable activities was designed for this study to reduce interobserver variability and lag time when documenting activities (i.e., no time spent checking a watch or writing down the time). All observers underwent a 2-hour didactic orientation before the start of data collection during the pre-EHR phase, and an additional 2-hour training refresher session was conducted prior to data collection in the post-EHR phase. A pilot study was performed prior to formal data collection during seven half-day clinical training sessions. Parallel observations were performed between the corresponding author (SLB) and observers during the pilot phase to verify consistency, as measured by intraclass correlation coefficient exceeding 0.8 (calculated with the icc function in the psych [18] package in R). Data collected during the pilot training sessions were not included in the analysis. Observers minimized interactions with residents and fellows, attending ophthalmologists, other ancillary staff, and patients. Clinical providers were instructed to limit any interaction with observers.

Seven ophthalmology residents and four ophthalmology fellows were observed in the clinics of attending ophthalmologists from six divisions (comprehensive, cornea, glaucoma, pediatrics, retina, and oculoplastics). In the preimplementation study phase (documentation on paper), time-motion observations were conducted during 13 half-day clinic sessions. In the postimplementation study phase (documentation on Epic), time-motion observations were conducted for six half-day clinic sessions. The inclusion criterion was any ophthalmology resident or fellow currently working in an outpatient attending clinic at the UCSD Shiley Eye Institute. Of note, all residents and fellows included in the study population underwent a comprehensive, 4-hour long training session led by a UCSD Epic analyst that was specific to Epic Kaleidoscope, during which they learned how to navigate, operate, and practice using a “play” environment simulating real documentation and charting. During the training session, each trainee also had the opportunity to work with the analyst to develop customization and personalization features. Additionally, during the first week of implementation, trainees received support in clinic from Epic Kaleidoscope analysts on site, a centralized implementation command center accessible via telephone, and UCSD staff support. Trainees in this study did not include medical students. Due to changes in rotation schedules between the preimplementation study phase and the postimplementation study phase, each participating trainee may not have been observed in both study phases. Because the time spent on documentation, examination, and other activities was expected to vary substantially by subspecialty clinic based on differing clinical workflows, the decision was made to observe trainees based on specific clinics before and after EHR implementation, rather than by individual trainees, since individual trainees rotated to different subspecialty clinics between the two phases of observation. Demographic information such as age, self-reported gender, self-reported ethnicity, and primary language were recorded for each patient seen, as well as visit type (new patient, routine follow-up, or postoperative visit within the 90-day global period), and whether the patient was dilated. The number of exam rooms, technicians, and patients (“clinic volume”) were recorded for each clinic session.

Total time spent by the resident or fellow with the patient, time spent documenting, time spent examining, and time spent talking with the patient were documented. The protocol for collecting time-motion data was based on previously published methods.[19] Documenting was broadly defined to include reviewing notes and images, writing notes, and writing orders or prescriptions. Documenting on EHR included any use of electronics to document, including the use of a desktop computer, tablet, or mobile phone app. Whenever talking with the patient occurred at the same time as another activity (such as reviewing notes or examining the patient), the nontalking activity was recorded. Time spent performing procedures, talking with other trainees or staff, talking with attending ophthalmologists, and waiting for patients to be ready were also recorded.

Descriptive statistics for ophthalmology residents and fellows, patients, and timing outcomes were generated in aggregate and also by subspecialty. To examine the effects of factors related to ophthalmology trainees, patients, and encounters on timing requirements, linear mixed effects models were used with ophthalmology trainees and patients as random effects. A separate linear mixed effects model was created for the subset of EHR encounter data alone to determine whether or not prior months of EHR experience or postgraduate year (PGY) training level had a significant impact on EHR use time in clinic. Random intercept and random slope models were evaluated. Covariates included patient's age, gender, ethnicity, language, visit type, dilation status, clinic volume, and number of available technicians and exam rooms. Statistical significance was defined as p < 0.05. Analyses were conducted in R[20] using the lme4 [21] and lmerTest [19] packages.

Results

Seven ophthalmology residents and four ophthalmology fellows were observed, with mean (standard deviation [SD]) PGY level of training of 3.7 (1.2). Trainee demographics are depicted in [Table 1]. The same trainees were not necessarily observed during both the preimplementation phase and the postimplementation phase due to changes in rotation schedules between the two phases of observation. All 11 (100%) trainees had prior experience with an EHR, with a mean (SD) of 50.1 (22.7) months. The average length of prior EHR experience among the preimplementation trainee cohort was 51.75 (23.56) months, whereas average prior EHR experience among the postimplementation trainee cohort was 44.75 (19.97) months (p = 0.61), which was not a statistically significant difference. There were also no significant differences in trainee age (33.1 vs. 33.0 years, p = 0.97) or PGY training level (3.75 vs. 3.25, p = 0.65) between the two study phases.

Table 1 Characteristics of ophthalmology trainees and patients included in time-motion analyses of outpatient encounters in 2018

Characteristic	Ophthalmology trainees (n = 11)	Ophthalmology patients (n = 156)
Mean age[a] (SD, range)	32.8 (4.6, 27–42)	61.7 (20.6, 1–96)
Sex
Female	4 (36.4%)	88 (56.4%)[e]
Male	7 (63.6%)	66 (42.3%)
Race/ethnicity[b]
White	4 (36.3%)	101 (64.7%)
Black or African American	0 (0%)	5 (3.2%)
Hispanic	0 (0%)	16 (10.3%)
Asian	5 (45.5%)	23 (14.7%)
Other race or mixed race	2 (18.2%)	8 (5.1%)
Primary language[c]
English	8 (72.7%)	141 (90.4%)
Spanish	0 (0%)	5 (3.2%)
Other	3 (27.3%)	8 (5.1%)
Subspecialty[d]
Comprehensive	2 (16.7%)	23 (14.7%)
Cornea	2 (16.7%)	14 (9.0%)
Glaucoma	2 (16.7%)	24 (15.4%)
Oculoplastics	3 (25%)	56 (35.9%)
Pediatrics	1 (8.3%)	10 (6.4%)
Retina	2 (16.7%)	29 (18.6%)

Abbreviation: SD, standard deviation.

^a Age in years at the time of the observed clinical encounter.

^b Race based on self-report for ophthalmology trainees, and for patients based on self-reported identification in the electronic registration system.

^c Primary language for patients based on language patient used during the clinical encounter.

^d Subspecialty for patients indicates the subspecialty of the patient's attending ophthalmologist in the observed clinical encounter.

^e Percentages may not add up to 100% due to missing data.

The demographic information of the 156 patients whose encounters were observed are depicted in [Table 1]. The time requirements for different clinical activities before and after EHR implementation are depicted in [Table 2]. Tasks considered as “documentation” were similar on paper charting and EHR charting, including reviewing and writing progress notes and procedure notes, performing medication reconciliation, acquiring and reviewing imaging results, and ordering prescriptions. Time spent by the trainee that is not reflected in [Table 2] represents time that was spent on other miscellaneous activities such as talking with the attending ophthalmologist or staff members about the patient, assisting with procedures, or completing administrative tasks.

Table 2 Time spent on patient encounter by ophthalmology trainees ∼2 weeks before and ∼6 weeks after EHR implementation

	Before EHR implementation		After EHR implementation
	Minutes, mean (SD)	Percentage of total time[c]	Minutes, mean (SD)	Percentage of total time[c]
Documentation time[a]	5.4 (3.5)	48%	6.8 (4.7)	57%[*]
Examination time[a]	3.4 (3.3)	28%	2.9 (2.5)	24%
Talking time[a]	2.8 (2.8)	24%	2.0 (1.6)[*]	18%[*]
Total time[b]	11.6 (6.5)		11.8 (6.9)

Abbreviations: EHR, electronic health record; SD, standard deviation.

^* Indicates p < 0.05.

^a For activities occurring simultaneously (such as examining while talking to the patient), the nontalking activity was recorded.

^b Total time spent by the trainee on the patient encounter during the clinic session. This includes all time spent on the patient's care during the observed clinic session and was not limited to time in the clinic room with the patient.

^c Percentages do not add up to 100%, as other activities may have been performed for patient care, including performing procedures and talking with attendings or with staff about the patient. Percentage time for each activity was calculated per patient and then averaged, but since not all activities were performed for each patient, percentages do not add to 100%.

Total time expenditure per patient was not significantly changed after EHR implementation (+0.17 minutes, 95% confidence interval [CI] for the difference in means: –2.78, 2.45; p = 0.90). Similarly, documentation time did not change significantly after EHR implementation in absolute terms (+1.42 minutes, 95% CI: –3.13, 0.29; p = 0.10). However, the proportion of time spent on documentation was significantly increased on the EHR (48% on paper to 57% on EHR; 95% CI: 2.17, 15.83; p = 0.011). Examination time was not significantly changed after EHR implementation in absolute terms (–0.49 minutes, 95% CI: –0.55, 1.52; p = 0.35) or in proportional terms (–4%, 95% CI: –8.86, 2.65; p = 0.28). Time spent exclusively talking with each patient was significantly less after EHR implementation in both absolute terms (–0.77 minutes, 95% CI: –0.04, –1.50; p = 0.04) and proportional terms (–6%, 95% CI: –0.87, –10.92; p = 0.022).

After EHR implementation, trainees spent a significantly larger proportion of the total time per patient inside the clinic room (84 vs. 70%, 95% CI: –22.40, –6.40, p < 0.01), demonstrating a trend of performing more clinical documentation in the clinic room with the patient present rather than documenting in a paper chart in the hallway between encounters, as average time spent documenting in the hallway decreased from 3.32 minutes prior to EHR implementation to 2.42 minutes after EHR implementation. Across all observed patient encounters, increasing PGY training level was associated with significantly less total time expenditure per patient (1.3 minutes less per additional year of training, p = 0.004), and dilation was associated with significantly greater total time expenditure per patient (increase of 3.29 minutes with dilation, p = 0.002). However, neither PGY training level, patient dilation status, nor other patient demographics (age, gender, ethnicity) significantly influenced time required for documentation across all observed encounters.

Encounters observed after EHR implementation were evaluated separately to assess whether prior EHR experience in other settings (e.g., medical school) and PGY training level influenced total time or documentation time per patient. Neither prior months of EHR experience nor PGY level had a significant influence on total time or documentation time required per patient.

Discussion

Understanding trainee time expenditures during patient encounters is critical for evaluating how EHR implementation affects clinical workflows in academic centers. Additionally, because trainees shoulder a substantial burden of clinical care documentation,[3] understanding these time requirements can also shed light on the trainee experience. This study analyzed the time requirements of various clinical activities during outpatient clinical encounters conducted by ophthalmology trainees before and after EHR implementation. Key findings from this study were that overall time spent per patient encounter did not change at 5 to 6 weeks post-EHR implementation, the proportion of time spent documenting with the EHR increased, and time spent exclusively talking with the patient during the encounter decreased after the EHR was implemented.

While EHR use is sometimes thought to lengthen clinical encounters among ophthalmologists and the field of medicine as a whole, studies examining the impact of EHR implementation on ophthalmology practices have demonstrated a mixed impact on patient volume and the time physicians spend in clinic.[5] [22] [23] [24] [25] [26] [27] [28] In our study, the total time that residents and fellows spent per patient encounter did not change with EHR implementation. Average time spent on a patient encounter during a paper chart-based encounter was 11.6 minutes, compared with 11.8 minutes spent on a patient encounter when using the EHR. Furthermore, this equivalent total time expenditure on patients between paper and electronic documentation occurred only 5 to 6 weeks after EHR implementation, when presumably trainees are still learning how to use the EHR. This lack of change in total time expenditure is similar to that found by Victores et al when examining the workflow of otolaryngology residents on clinic days.[12] In their study, overall efficiency was not affected by the implementation of an EHR, but time spent in clinic was shifted from direct patient care to indirect patient care (defined as documenting and reviewing the medical record and results). This is similar to our study's finding that proportionally more time was spent documenting the clinical encounter when using the EHR versus the paper chart ([Table 2]). This could have also been affected by the fact that the EHR was newly implemented in the observed clinics. Therefore, many of the observed patients were “new to the EHR” even if they were not “new patients” to the clinic per se, because their clinical data had not yet been migrated to the EHR. It is possible that this initial data migration would have taken more time compared with subsequent visits. This represents an area of future study.

Data collection revealed that trainees spent statistically significantly less time talking exclusively with the patient during an encounter that was charted using the EHR compared with using paper documentation (2.0 vs. 2.8 minutes, p = 0.04). Consistent with previously published methods for conducting time-motion studies in ophthalmology,[19] [29] we only recorded the trainee as “talking” if no other activity was being performed simultaneously (such as documenting, examining, or performing procedures). If the trainee was engaged in multitasking, only the nontalking activity was recorded. For instance, if the trainee was talking to the patient while also typing in the EHR, “documenting” was the recorded activity. The decrease in time spent talking exclusively with the patient but the proportional increase in time spent documenting could suggest that trainees were engaged in more frequent multitasking (e.g., talking while documenting simultaneously) after EHR implementation, but further studies recording multitasking data may shed light on the distribution of time during an encounter.

Of note, the proportion of total time spent with the patient in the clinic room itself increased from 70 to 84% after EHR implementation. This may be accounted for by the fact that paper chart review was previously performed outside of the clinic room in the hallway before EHR implementation, whereas after EHR implementation chart review was conducted on a desktop computer inside the clinic room. For years, studies have shown that longer ambulatory visits with time spent with the provider are associated with increased patient satisfaction.[30] [31] In addition, the patient's estimation of the visit length also plays a role in satisfaction with their provider.[30] Taking these two factors into account, it is interesting to consider the impact of the EHR increasing trainee time spent with the patient while also possibly influencing the patient's and trainee's perceived time spent with each other. However, while there is an objective increase in face-to-face time with the patient if the trainee is in the exam room for a longer period of time, studies show that what occurs during that time is also important. Marmor et al demonstrate that EHR usage may negatively influence a patient's perception of a physician's communication skills and overall satisfaction of care.[32] Additionally, the perceived increase in time with a patient while performing data collection through the EHR may also influence trainees, leading them to believe they require less history from the patient themself.[32]

Another important consideration when examining increased proportional documentation time is the impact that this increase has on trainees' perceptions of how they spend their time. Using the EHR, it may be that trainees had access to a larger amount of existing progress notes, laboratory results, and operative notes when using an EHR spanning the whole health system and encompassing multiple specialties than they had when they were reviewing patients' medical information in the paper chart, which only included information from ophthalmology encounters. It is possible that the increase in observed documentation time could be increasing trainee time spent multitasking and thus increasing cognitive burden on trainees who routinely experience significant task interruptions and task switching.[33] [34] EHR use time has been identified as the strongest predictor of burnout among residents, above factors such as sleep and exercise,[3] and the Massachusetts Medical Society recently named EHR reform as one of the three main ways to address physician burnout.[35] Despite our study and others finding equivalent encounter time expenditure while using the EHR as compared with paper-based documentation, it could be that the reported impact of the EHR on trainee burnout rates can be explained by the cognitive burden of task fragmentation created by interacting with the EHR environment.[36] [37] While there may be an educational benefit to having increased information available to the trainee in the form of the EHR, this benefit needs to be weighed against factors that increase the cognitive burden of the EHR.

The number of months of prior EHR experience and PGY level of training did not significantly affect total time or documentation time per patient during the post-EHR implementation phase of this study. All trainees observed had extensive prior EHR experience, as the least experienced EHR user had previously used an EHR for 12 months. Therefore, we were not able to evaluate how time demands of an EHR would affect novice EHR users with less than 12 months of experience. Rodriguez Torres et al also examined the relationship between a trainee's level of experience and their EHR use, demonstrating that the type and format of an EHR play a more significant role in an ophthalmology trainee following institutional documentation guidelines than does level of training.[17] While compliance with documentation was the measured metric in their study rather than timing, it is interesting to note that the design of the EHR played a more important role in documentation compliance than did years of training. Trainees in our study had previously used a variety of EHR platforms, and future studies may explore the role of different EHR platforms on trainee education.

An increasing emphasis has been placed on standardizing EHR use and exposure from the start of medical education.[38] [39] [40] Most medical schools and academic medical centers provide medical students and residents with EHR access,[41] but training in these settings may be varied and can result in a wide range of documentation styles and outcomes.[42] How different methods of EHR training impact educational outcomes and efficiency (whether for trainees or for practicing physicians) may be worthwhile future directions of study.

The UCSD ophthalmology department's paper-to-EHR transition may be a rare occurrence as EHRs become universally adopted, but EHR-to-EHR transitions and updates to existing systems may mimic some of the resulting outcomes observed in our study. While potentially less dramatic than paper-to-EHR transitions, EHR-to-EHR transitions may still entail major changes in clinical information systems and clinical workflows, with potential impacts on trainee experience and education. Our results may also be used to start establishing benchmarks to study other problems. As previously discussed, EHR use time has been identified as an important factor contributing to trainee burnout.[3] [35] Perhaps observed impacts on clinic flow and resident burnout rates would be minimized with more frequent training and optimization sessions that would reduce documentation burden for trainees. It is thus important to continue examining how ophthalmology trainees utilize the EHR, how the EHR affects residency education, and how EHR-specific training may optimize EHR use for trainees.

In summary, this study evaluated the time demands of outpatient clinical encounters of ophthalmology trainees at an academic ophthalmology clinic that recently underwent EHR implementation. Unlike other studies that followed trainees throughout their day during various inpatient, outpatient, and didactic settings and collected broad aggregated timing data on different clinical activities, our study collected detailed timing data on activities performed within individual patient encounters on a second-to-second basis, allowing us to understand how ophthalmology trainees spend their time in clinic with individual patients before and after EHR implementation. We observed that the EHR had little effect on overall time spent with patients but that it did change how trainees divided that time between various clinical activities. As EHRs have been widely adopted among academic centers, it is important to understand how health information technology may influence trainee education so as to improve patient care, learning opportunities, and documentation optimization for ophthalmology trainees.

Conflict of Interest

Dr. Chao reports other from Recens Medical, other from Visgenx, other from Dtx Pharma, other from Zilia Health, grants from National Eye Institute, grants from Brightfocus Foundation, outside the submitted work. Dr. Longhurst reports other from null, outside the submitted work. Dr. Chiang reports grants from National Institutes of Health, grants from National Science Foundation, other from Clarity Medical Systems, personal fees from Novartis, other from Inteleretina, LLC, outside the submitted work; In addition, Dr. Chiang has a patent provisional patent for artificial intelligence diagnostic system pending. Dr. Hribar reports grants from National Institutes of Health (Bethesda, MD), during the conduct of the study; grants from National Institutes of Health (Bethesda, MD), outside the submitted work. Dr. Nudleman reports grants from National Institute of Health, outside the submitted work. Dr. Robbins reports grants and personal fees from American Academy of Pediatrics Royalites, Springer Publishing Royalties, Elsevier Publishing Royalties, Department of Health and Human Services Consultant, Prometic Consultant, The Hartwell Foundation Grant, Retrophin Industry sponsored Grant, outside the submitted work. Dr. Baxter reports grants from National Library of Medicine, grants from Heed Ophthalmic Foundation, during the conduct of the study.

Acknowledgments

Financial Support: This study was supported by the National Institutes of Health (Bethesda, MD; grant numbers T15LM011271, R00LM12238, P30EY10572, P30EY022589, UL RR031980), by the Heed Ophthalmic Foundation Fellowship (San Francisco, CA), and by unrestricted departmental grants from Research to Prevent Blindness (New York, NY).

Recognition of Assistance: UCSD student observers who assisted with time-motion data collection were Jennie Xu, BS, Citra Khalil, BS, Ala'a Erra, Dennis Tran, Nguyen Nguyen, Neil E. Shah, Vivian Do, Alyssa Cadiz, Alyssa Haack, Sonali Bhanvadia, and Manreet Brar.

• References

• 1 Tierney MJ, Pageler NM, Kahana M, Pantaleoni JL, Longhurst CA. Medical education in the electronic medical record (EMR) era: benefits, challenges, and future directions. Acad Med 2013; 88 (06) 748-752
• 2 Peled JU, Sagher O, Morrow JB, Dobbie AE. Do electronic health records help or hinder medical education?. PLoS Med 2009; 6 (05) e1000069
• 3 Domaney NM, Torous J, Greenberg WE. Exploring the association between electronic health record use and burnout among psychiatry residents and faculty: a pilot survey study. Acad Psychiatry 2018; 42 (05) 648-652
• 4 Holmes EG, Connolly A, Putnam KT. , et al. Taking care of our own: a multispecialty study of resident and program director perspectives on contributors to burnout and potential interventions. Acad Psychiatry 2017; 41 (02) 159-166
• 5 Lim MC, Boland MV, McCannel CA. , et al. Adoption of electronic health records and perceptions of financial and clinical outcomes among ophthalmologists in the United States. JAMA Ophthalmol 2018; 136 (02) 164-170
• 6 Ehrlich JR, Michelotti M, Blachley TS. , et al. A two-year longitudinal assessment of ophthalmologists' perceptions after implementing an electronic health record system. Appl Clin Inform 2016; 7 (04) 930-945
• 7 Sharma A, Lo V, Lapointe-Shaw L, Soong C, Wu PE, Wu RC. A time-motion study of residents and medical students performing patient discharges from general internal medicine wards: a disjointed, interrupted process. Intern Emerg Med 2017; 12 (06) 789-798
• 8 Wormer BA, Colavita PD, Yokeley WT. , et al. Impact of implementing an electronic health record on surgical resident work flow, duty hours, and operative experience. Am Surg 2015; 81 (02) 172-177
• 9 Blair T, Wiener Z, Seroussi A, Tang L, O'Hora J, Cheung E. Resident workflow and psychiatric emergency consultation: identifying factors for quality improvement in a training environment. Acad Psychiatry 2017; 41 (03) 377-380
• 10 Urchek RJ, Morscher MA, Steiner RP, Adamczyk MJ. Orthopaedic resident use of an electronic medical record template does not improve documentation for pediatric supracondylar humerus fractures. J Am Acad Orthop Surg 2018; ••• DOI: 10.5435/JAAOS-D-17-00818.
• 11 Victores A, Roberts J, Sturm-O'Brien A. , et al. Otolaryngology resident workflow: a time-motion and efficiency study. Otolaryngol Head Neck Surg 2011; 144 (05) 708-713
• 12 Victores AJ, Coggins K, Takashima M. Electronic health records and resident workflow: a time-motion study of otolaryngology residents. Laryngoscope 2014; ••• DOI: 10.1002/lary.24848.
• 13 Neri PM, Redden L, Poole S. , et al. Emergency medicine resident physicians' perceptions of electronic documentation and workflow: a mixed methods study. Appl Clin Inform 2015; 6 (01) 27-41
• 14 Starmer AJ, Destino L, Yoon CS, Landrigan CP. Research letter: intern and resident workflow patterns on pediatric inpatient units: a multicenter time-motion study. JAMA Pediatr 2015; 169 (12) 1175-1177
• 15 Aylor M, Campbell EM, Winter C, Phillipi CA. Resident notes in an electronic health record. Clin Pediatr (Phila) 2017; 56 (03) 257-262
• 16 Goldstein IH, Hribar MR, Read-Brown S, Chiang MF. Association of the presence of trainees with outpatient appointment times in an ophthalmology clinic. JAMA Ophthalmol 2018; 136 (01) 20-26
• 17 Rodriguez Torres Y, Huang J, Mihlstin M, Juzych MS, Kromrei H, Hwang FS. The effect of electronic health record software design on resident documentation and compliance with evidence-based medicine. PLoS One 2017; 12 (09) e0185052 . Doi: 10.1371/journal.pone.0185052
• 18 Revelle WR. . psych: Procedures for Personality and Psychological Research. 2017
• 19 Read-Brown S, Hribar MR, Reznick LG. , et al. Time requirements for electronic health record use in an academic ophthalmology center. JAMA Ophthalmol 2017; 135 (11) 1250-1257
• 20 R: The R Project for Statistical Computing
• 21 Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4 . J Stat Softw 2015; 67 (01) DOI: 10.18637/jss.v067.i01.
• 22 Chiang MF, Read-Brown S, Tu DC. , et al. Evaluation of electronic health record implementation in ophthalmology at an academic medical center (an American Ophthalmological Society thesis). Trans Am Ophthalmol Soc 2013; 111: 70-92
• 23 Boland MV, Chiang MF, Lim MC. , et al; American Academy of Ophthalmology Medical Information Technology Committee. Adoption of electronic health records and preparations for demonstrating meaningful use: an American Academy of Ophthalmology survey. Ophthalmology 2013; 120 (08) 1702-1710
• 24 Jamoom EW, Heisey-Grove D, Yang N, Scanlon P. Physician opinions about EHR use by EHR experience and by whether the practice had optimized its EHR use. J Health Med Inform 2016; 7 (04) 1000240
• 25 Singh RP, Bedi R, Li A. , et al. The practice impact of electronic health record system implementation within a large multispecialty ophthalmic practice. JAMA Ophthalmol 2015; 133 (06) 668-674
• 26 Pandit RR, Boland MV. The impact of an electronic health record transition on a glaucoma subspecialty practice. Ophthalmology 2013; 120 (04) 753-760
• 27 Lim MC, Patel RP, Lee VS, Weeks PD, Barber MK, Watnik MR. The long-term financial and clinical impact of an electronic health record on an academic ophthalmology practice. J Ophthalmol 2015; 2015: 329819
• 28 Lam JG, Lee BS, Chen PP. The effect of electronic health records adoption on patient visit volume at an academic ophthalmology department. BMC Health Serv Res 2016; 16: 7 . Doi: 10.1186/s12913-015-1255-8
• 29 Baxter SL, Gali HE, Huang AE. , et al. Time requirements of paper-based clinical workflows and after-hours documentation in a multi-specialty academic ophthalmology practice. Am J Ophthalmol 2019; 206: S0002-9394(19)30120-5
• 30 Lin CT, Albertson GA, Schilling LM. , et al. Is patients' perception of time spent with the physician a determinant of ambulatory patient satisfaction?. Arch Intern Med 2001; 161 (11) 1437-1442
• 31 Gross DA, Zyzanski SJ, Borawski EA, Cebul RD, Stange KC. Patient satisfaction with time spent with their physician. J Fam Pract 1998; 47 (02) 133-137
• 32 Marmor RA, Clay B, Millen M, Savides TJ, Longhurst CA. The impact of physician EHR usage on patient satisfaction. Appl Clin Inform 2018; 9 (01) 11-14
• 33 Ly T, Korb-Wells CS, Sumpton D, Russo RR, Barnsley L. Nature and impact of interruptions on clinical workflow of medical residents in the inpatient setting. J Grad Med Educ 2013; 5 (02) 232-237
• 34 Heng KWJ. Teaching and evaluating multitasking ability in emergency medicine residents - what is the best practice?. Int J Emerg Med 2014; 7: 41 . Doi: 10.1186/s12245-014-0041-4
• 35 Jha A, Iliff A, Chaoui A. , et al. A Crisis in Healthcare: A Call to Action on Physician Burnout. Waltham, MA: Massachusetts Medical Society; 2018
• 36 Zheng K, Haftel HM, Hirschl RB, O'Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc 2010; 17 (04) 454-461
• 37 Mazur LM, Mosaly PR, Moore C, Marks L. Association of the usability of electronic health records with cognitive workload and performance levels among physicians. JAMA Netw Open 2019; 2 (04) e191709
• 38 Gagliardi JP, Turner DA. The electronic health record and education: rethinking optimization. J Grad Med Educ 2016; 8 (03) 325-327
• 39 Niedermier VE. Teaching electronic health record documentation to medical students. J Grad Med Educ 2017; 9 (01) 135
• 40 Hammoud MM, Dalymple JL, Christner JG. , et al. Medical student documentation in electronic health records: a collaborative statement from the Alliance for Clinical Education. Teach Learn Med 2012; 24 (03) 257-266
• 41 Foster LM, Cuddy MM, Swanson DB, Holtzman KZ, Hammoud MM, Wallach PM. Medical student use of electronic and paper health records during inpatient clinical clerkships: results of a National Longitudinal Study. Acad Med 2018; 93 (11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 57th Annual Research in Medical Education Sessions): S14-S20
• 42 March CA, Scholl G, Dversdal RK. , et al. Use of electronic health record simulation to understand the accuracy of intern progress notes. J Grad Med Educ 2016; 8 (02) 237-240