Effect of Telemedicine and the COVID-19 Pandemic on Medical Trainees' Usage of the Electronic Health Record in the Outpatient Setting

Abstract

Objectives This study aimed to (1) determine the impact of COVID-19 (coronavirus disease 2019) and the corresponding increase in use of telemedicine on volume, efficiency, and burden of electronic health record (EHR) usage by residents and fellows; and (2) to compare these metrics with those of attending physicians.

Methods We analyzed 11 metrics from Epic's Signal database of outpatient physician user logs for active residents/fellows at our institution across three 1-month time periods: August 2019 (prepandemic/pre-telehealth), May 2020 (mid-pandemic/post-telehealth implementation), and July 2020 (follow-up period) and compared these metrics between trainees and attending physicians. We also assessed how the metrics varied for medical trainees in primary care as compared with subspecialties.

Results Analysis of 141 residents/fellows and 495 attendings showed that after telehealth implementation, overall patient volume, Time in In Basket per day, Time outside of 7 a.m. to 7 p.m., and Time in notes decreased significantly compared with the pre-telehealth period. Female residents, fellows, and attendings had a lower same day note closure rate before and during the post-telehealth implementation period and spent greater time working outside of 7 a.m. to 7 p.m. compared with male residents, fellows, and attendings (p < 0.01) compared with the pre-telehealth period. Attending physicians had a greater patient volume, spent more time, and were more efficient in the EHR compared with trainees (p < 0.01) in both the post-telehealth and follow-up periods as compared with the pre-telehealth period.

Conclusion The dramatic change in clinical operations during the pandemic serves as an inflection point to study changes in physician practice patterns in the EHR. We observed that (1) female physicians closed fewer notes the same day and spent more time in the EHR outside of normal working hours compared with male physicians, and (2) attending physicians had higher patient volumes and also higher efficiency in the EHR compared with resident physicians.

Keywords

EHR usage - telemedicine - COVID-19 - residents - fellows - attendings - physician health

Background and Significance

In March 2020, the first wave of the coronavirus disease 2019 (COVID-19) pandemic hit New York and the impacts included an overflow of inpatient medicine services within the Montefiore Health System (MHS), with many health care physicians, especially residents and fellows, being re-deployed to various roles in the health system to manage the surge in COVID-19 cases. Additionally, many patients experienced increased fear and reluctance to risk exposure for traditional models of outpatient or even emergency care.[1] As such, rates of telemedicine usage increased dramatically during this period.[2] [3] [4] [5] While telemedicine had been discussed as an alternative option prior to the pandemic, the increase in telemedicine usage[6] [7] in March 2020[8] [9] [10] has led to an opportunity to study its impact on physician practice patterns[11] and electheric health record (EHR) documentation, and the possible implications for physician well-being[12] herd burnout.

While the EHR has many benefits to physicians, including standardizing documentation, reducing errors, and enabling better data sharing between physicians, it is also often cited in physician surveys as a common source of burden and even burnout in modern medicine. Studies have noted that physicians, both residents and attendings, spend significant time after work hours in the medical record[13] [14] and that extensive use of the EHR is linked to decreased physician satisfaction and increased rate of burnout.[15] [16] [17] [18] [19] Further, EHR usage metrics can help to audit and assess how physicians spend their time in a clinical setting.[20] Previous studies have also found that both self-reported and objective measures of time spent in the EHR are associated with physician burnout, as validated by the Maslach Burnout Inventory and American Medical Association mini-Z meherrement surveys.[15] [18] [21] [22]

While EHR usage metrics have been studied and validated for attending physicians, data on medical trainees, specifically medical residents and fellows, have not been evaluated. Furthermore, there is limited research comparing resident to attending level data to determine how changes affect each group in tandem, and whether changes in practice patterns vary between these different groups. Our research aims to fill this gap by highlighting not only how the onset of the COVID-19 pandemic and the rise in telehealth impacted trainees at MHS, but also whether the changes are consistent with or different from findings in prior research on attending physicians. Specifically, we aim to (1) determine the impact of COVID-19 and the corresponding increase in use of telemedicine on volume, efficiency, and burden of EHR usage by residents and fellows, stratified by sex and specialty, and (2) compare these metrics with those of attending physicians.

Methods

The study was performed at MHS in the Bronx, New York City and was approved by the Albert Einstein School of Medicine Institutional Review Board. The transition to telemedicine at MHS occurred as follows: in the first week of March 2020, clinics were operating at their typical fully in-person schedules. However, by mid-month, clinic show rates began to decrease,[10] likely due to a fear of the COVID-19 virus. In fact, on March 18, 2020, the Accreditation Council for Graduate Medical Education advised the Graduate Medical Education community to encourage residents/fellows to take part in various telemedicine modalities of patient care during the COVID-19 pandemic.[8] Specifically, MHS incorporated telemedicine modalities, such as increasing phone and video visits by mid-April.[8] The month of May was the first full month of this implementation and represents a key inflection point for studying the impact of telemedicine at MHS.[8] [10] Based on this timeline, we selected three 1-month periods for data extraction. The first period was August 2019, which served as a prepandemic/pre-telemedicine period. The second period was May 2020, when telemedicine uptake increased drastically, and the third period was July 2020 to evaluate persistence of changes, as telemedicine visits began to decrease as in-person clinical visits were taking place again.

We analyzed data from EPIC Signal, a web-based platform to assess EHR usage patterns for providers, of targeted calculated documentation performance metrics collected on all physicians in the outpatient setting. We selected 11 metrics of interest from the Montefiore Health Epic Signal database, consistent with prior research performed on patient volume, physician efficiency, and physician burnout.[18] [23] [24] [25] Using the Epic Signal web-based dashboard, we selected performance metrics displayed under “Overview,” “In Basket,” “Notes and Letters,” and “Workload” metrics as we hypothesized that these categories were most likely to be affected by the COVID-19 pandemic and resulting switch from in-person to telehealth and hybrid models of care. Patient volume was assessed using number of appointments per day and total number of messages received per day. Epic Signal developed and calculated “efficiency” metrics that have been shown to determine changes in physician workflow and overall usage patterns.[26] [27] [28] In this study, physician efficiency was assessed by multiple Epic metrics including (1) same day visit closure rate (how often an encounter is closed the same day as the visit), (2) Physician Efficiency Profile (PEP, a score that compares actual time spent in Epic with expected time, with a higher score indicating less time spent compared with expectations and greater “efficiency”),[26] (3) efficiency score (frequency of use of embedded EHR efficiency tools),[27] [28] (4) minutes spent in notes per appointment, (5) turnaround time (days to respond to In Basket message), and (6) time spent in In Basket per day (minutes).

For all time-related metrics, time was measured from when the physician initially logged into the Epic system and was actively using the system, including mouse movements, clicks, and scrolling, as well as keyboard strokes. The time measurements were paused during inactivity for greater than 5 seconds or when the physician logged out of the Epic system.[27] Proficiency is rated as a composite score from 0 to 10 and is measured as how often a physician uses Epic's in-built efficiency tools, such as chart search, preference lists, quick action tools, and speed buttons.[27] [28] Similarly, the PEP score reflects physician efficiency by comparing a physician's time in the system to the calculated expected time based on workload. The Epic workload component is a weighted composite of all Epic physician users of the same medical specialty.[28] Because the PEP score is normalized relative to each organization, we did not assess changes to this metric over time. Instead, we compared PEP scores among different groups of physicians, as per Epic's recommendation.

Physician burden was deduced by multiple metrics that measured time spent working in the EHR. We evaluated physician burden using the metrics (1) time (average minutes) outside of 7 a.m. to 7 p.m. working hours, (2) time (average minutes) spent on unscheduled days, and (3) pajama time (average minutes spent in Epic outside of 7 a.m. to 5:30 p.m. on weekdays and outside scheduled time on weekends).

To determine which physician data to analyze, physicians within the database were filtered according to their status as residents, fellows, and attendings and were further subdivided by the specialty of training and separated into primary care versus specialty care. Physicians were selected if they were active in outpatient clinic during the studied time periods. For the residents and fellows, Family Medicine, Internal Medicine, and Pediatrics were considered primary care, while Gastroenterology, Endocrinology, Cardiology, Neurology, Nephrology, Obstetrics/Gynecology, Psychiatry, Physical Medicine and Rehabilitation, and various pediatric subspecialties were all considered specialty care. For attending physicians, the same categorization was maintained, but also included additional specialty care specialties, including Adolescent Medicine, Allergy and Immunology, Cardiovascular Disease, Dermatology, Gynecologic Oncology, Hematology and Oncology, Infectious Diseases, Maternal Fetal Medicine, Pain Medicine, Sports Medicine, Sleep Medicine, and numerous pediatric subspecialties. Surgical specialties were also considered as specialty care and included General Surgery, Neurosurgery, Ophthalmology, Otolaryngology, Surgical Oncology, Urology, Transplant Hepatology, Vascular Surgery, Colorectal Surgery, and Interventional Radiology.

Descriptive statistics were calculated for all 11 metrics across the three time periods among all residents and fellows, by gender, and by specialty group. MATLAB's kstest function from the “statistics and machine learning toolbox in hypothesis tests” was used to conduct the one-sample Kolmogorov–Smirnov test for each variable to test normality. The Wilcoxon signed rank test was used to compare the median values in metrics between both postimplementation periods and pre-COVID-19 for the overall analysis group and test for any statistically significant differences. The Mann–Whitney U-test was employed to compare the differences between gender, specialty categories, and attending versus resident/fellow for the descriptive statistics. MATLAB's sign-rank and rank sum functions were used to perform the Wilcoxon signed rank and the Mann–Whitney U-tests, respectively. To assess how residents/fellows and attendings were comparatively impacted by COVID-19, the difference (delta) in physician's objective metrics were compared from August 2019 to May 2020 and from August 2019 to July 2020. The resulting “delta vectors,” describing the quantitative change in performance metrics between the time periods for resident/fellow physicians and attending physician's, separately, were then compared to determine if there were any statistically significant differences among both groups of physicians using the Mann–Whitney U-test. All p-values were considered statistically significant at a value of 0.05.

Results

We analyzed data from 636 physicians across 55 specialties and subspecialties. In total, 22.2% (n = 141) of physicians were either residents (n = 123, 87.2%) or fellows (n = 18, 12.8%); 52% (n = 333) of physicians were female; 41% (n = 260) of all physicians were in primary care, 50% (n = 320) were in specialty care, and 9% (n = 56) were specifically in surgical subspecialties.

Overall Variation in Trainee Outcome Metrics by Time Point

There were significant variations in the descriptive statistics of outcome metrics pre- and during COVID-19 across all trainees ([Table 1]). Patient volume decreased significantly from August 2019 to May 2020, as measured by both appointments per day (4.76 vs. 3.00, p < 0.001) and aggregate messages (4.98 vs. 4.12, p = 0.02). When compared with August 2019, the number of appointments per day was significantly decreased in July 2020 (4.76 vs. 3.73, p = 0.004), but the aggregate messages showed no significant change (4.98 vs. 4.47, p = 0.74).

Table 1 Descriptive statistics of outcome metrics for pre- and post-COVID-19 across all trainees (N = 140)

	Time period			p-Value (May vs. August)	p-Value (July vs. August)
	August 2019 (95% CI)	May 2020 (95% CI)	July 2020 (95% CI)
Appointments per day	4.76 (3.50–6.50)	3.00 (2.00–4.48)	3.73 (2.64–5.00)	<0.001	0.004
Aggregate messages per day	4.98 (2.68–8.31)	4.12 (1.99–6.29)	4.47 (2.28–7.09)	0.015	0.742
Same day visit closure rate	0.72 (0.38–1.00)	0.95 (0.71–1.00)	0.90 (0.59–1.00)	0.972	0.133
Proficiency score	3.36 (3.11–3.81)	3.27 (2.09–3.80)	3.38 (3.15–3.70)	0.001	0.011
Time in notes (minutes per appointment)	13.55 (10.05–17.90)	9.58 (6.60–12.16)	10.97 (9.21–14.90)	<0.001	<0.001
Turnaround time (min per appointment)	2.67 (0.89–9.81)	2.71 (0.48–10.34)	3.51 (0.82–15.09)	0.972	0.133
Time in In Basket (min per appointment)	3.47 (0.93–6.92)	1.40 (0.42–4.08)	2.46 (0.73–6.00)	<0.001	0.144
Time outside of 7 a.m.–7 p.m. (min)	14.23 (3.70–31.15)	12.02 (3.29–23.79)	8.39 (2.56–22.09)	0.042	0.002
Time on unscheduled days (min)	36.12 (18.66–47.72)	27.67 (23.61–43.80)	26.69 (20.18–40.31)	0.236	0.611
Pajama time (min)	32.73 (12.32–56.69)	17.80 (9.40–49.61)	24.29 (10.76–40.26)	0.171	0.074

Abbreviation: CI, confidence interval.

Note: Statistically significant p-values are indicated in bold.

The “efficiency” metrics of proficiency score, Time in notes, and Time in In Basket all showed statistically significant differences pre- and during COVID-19. Time in notes decreased significantly in both postimplementation periods, from 13.55 minutes per note in August 2019 to 9.58 minutes per note in May of 2020 and 10.97 minutes per note in July 2020 (p < 0.001). There was a significant decrease in Time outside of 7 a.m. to 7 p.m. from August 2019 to May 2020 (14.23 vs. 12.02 minutes per day, p = 0.04) and to July 2020 (14.23 vs. 8.39 minutes per day, p = 0.002). While Time in In Basket statistically decreased from August 2019 to May 2020 (3.47 vs. 1.40 minutes per day, p < 0.001), there was no statistically significant decrease from August 2019 to July 2020 (3.47 vs. 2.46 minutes per day, p = 0.14).

Trainee Outcome Metrics by Gender

There were few differences between male and female trainees across all three time periods ([Table 2]). In August of 2019 and May of 2020, female trainees had statistically significant lower same day visit closure rate (0.62 and 0.89, respectively) compared with male trainees (0.84 and 1.00, p < 0.05). Female trainees also had statistically significant higher Time outside of 7 a.m. to 7 p.m. (9.24 vs. 18.08 total minutes per day, p = 0.03) before COVID-19. In July of 2020, female trainees had significantly more pajama time compared with male physicians (40.90 vs. 21.12 minutes per day, p = 0.02).

Table 2 Descriptive statistics of outcome metrics (by gender)

	August 2019
	Male (n = 71; 95% CI)	Female (n = 70; 95% CI)	p-Value
Appointments per day	5.00 (3.72–6.50)	4.67 (3.42–6.86)	1.00
Aggregate messages per day	0.84 (0.48–1.00)	0.62 (0.33–0.96)	0.048
Same day visit closure rate	29.17 (16.92–41.56)	38.12 (27.52–50.91)	0.097
PEP score	20.28 (11.28–49.46)	37.48 (15.80–69.14)	0.073
Proficiency score	9.24 (2.97–26.91)	18.08 (7.83–31.94)	0.026
Time in notes (min per appointment)	13.55 (9.86–17.18)	13.82 (10.10–20.52)	0.548
Turnaround time (min per appointment)	2.62 (0.90–6.21)	4.52 (1.15–7.39)	0.277
Time in In Basket (min per appointment)	4.41 (2.78–8.14)	5.47 (2.73–8.33)	0.642
Time outside of 7 a.m.–7 p.m. (min)	2.87 (0.74–10.27)	2.62 (1.66–9.24)	0.694
Time on unscheduled days (min)	5.25 (4.32–5.70)	4.84 (4.00–5.60)	0.259
Pajama time (min)	3.42 (3.18–3.74)	3.30 (3.10–3.82)	0.401
	May 2020
	Male (n = 71 95% CI)	Female (n = 70 95% CI)	p-Value
Appointments per day	3.00 (2.00–4.13)	3.33 (2.00–4.88)	0.618
Aggregate messages per day	1.00 (0.88–1.00)	0.89 (0.49–1.00)	0.020
Same day visit closure rate	25.85 (17.62–35.8)	34.03 (25.65–56.55)	0.119
PEP score	17.75 (6.58–42.4)	19.24 (9.40–49.61)	0.598
Proficiency score	9.38 (2.20–21.76)	15.69 (4.16–25.90)	0.335
Time in notes (min per appointment)	8.50 (5.36–11.25)	9.66 (7.00–14.22)	0.453
Turnaround time (min per appointment)	1.36 (0.28–3.60)	1.43 (0.59–5.17)	0.249
Time in In Basket (min per appointment)	3.46 (1.71–5.74)	4.53 (2.90–7.73)	0.010
Time outside of 7 a.m.–7 p.m. (min)	2.82 (0.49–11.48)	2.61 (0.48–9.23)	0.621
Time on unscheduled days (min)	5.61 (5.00–5.95)	5.09 (4.78–5.50)	0.003
Pajama time (min)	3.27 (2.17–3.75)	3.31 (2.07–3.83)	0.858
	July 2020
	Male (n = 71 95% CI)	Female (n = 70 95% CI)	p-Value
Appointments per day	4.13 (2.94–5.86)	3.43 (2.53–4.75)	0.222
Aggregate messages per day	0.96 (0.73–1.00)	0.85 (0.50–1.00)	0.243
Same day visit closure rate	26.69 (17.21–40.00)	28.07 (21.53–47.42)	0.903
PEP score	21.12 (10.50–25.54)	40.90 (22.16–60.92)	0.024
Proficiency score	6.11 (1.92–17.25)	10.75 (3.29–29.20)	0.267
Time in notes (min per appointment)	11.45 (9.64–14.16)	10.81 (8.90–20.88)	0.713
Turnaround time (min per appointment)	3.16 (1.12–6.79)	1.74 (0.60–4.38)	0.214
Time in In Basket (min per appointment)	4.54 (2.54–7.12)	4.36 (2.02–7.09)	0.778
Time outside of 7 a.m.–7 p.m. (min)	6.89 (2.06–33.17)	1.86 (0.33–9.30)	0.006
Time on unscheduled days (min)	5.20 (4.78–5.73)	5.17 (4.42–5.70)	0.536
Pajama time (min)	3.36 (3.13–3.75)	3.40 (3.20–3.68)	0.964

Abbreviations: CI, confidence interval; PEP, Physician Efficiency Profile.

Note: Statistically significant p-values are provided in bold.

Trainee Outcome Metrics for Primary Care versus Subspecialty

In all three time periods, the “efficiency” metrics of Time in In Basket, aggregate messages, and turnaround time all were significantly lower among specialty trainees compared with primary care trainees (p < 0.01; [Table 3]). In August of 2019, specialty care physicians had lower proficiency scores compared with those in primary care (3.20 vs. 3.44, p < 0.01); however, during May of 2020, specialty care physicians had no statistically significant difference in proficiency scores compared with primary care physicians (3.37 vs. 3.09, p = 0.12). By July 2020, specialty care physicians again had lower proficiency scores (3.24 vs. 3.46, p < 0.01). Appointments per day were lower for specialty care physicians compared with primary care physicians in both August 2019 and July 2020 (5.00 vs. 3.91 appointments per day and 4.25 vs. 3.22 appointments per day, respectively, p < 0.01) but were not statistically different in May 2020 (3.00 vs. 3.02 appointments per day, p = 0.74).

Table 3 Descriptive statistics of outcome metrics (by specialty category)

	August 2019
	Primary care (95% CI)	Specialty care (95% CI)	p-Value
Appointments per day	5.00 (4.00–7.15)	3.91 (2.80–5.80)	0.002
Aggregate messages per day	0.65 (0.29–0.85)	0.96 (0.57–1.00)	<0.001
Same day visit closure rate	36.45 (23.01–50.24)	28.16 (16.65–40.99)	0.09
PEP score	36.69 (15.86–60.90)	18.91 (8.31–45.14)	0.053
Proficiency score	23.60 (7.40–32.23)	9.16 (3.02–17.54)	0.005
Time in notes (min per appointment)	14.12 (11.47–17.72)	10.84 (8.12–18.20)	0.22
Turnaround time (min per appointment)	5.56 (3.42–9.07)	0.87 (0.27–1.87)	<0.001
Time in In Basket (min per appointment)	7.27 (4.54–9.94)	2.59 (1.15–3.95)	<0.001
Time outside of 7 a.m.–7 p.m. (min)	3.20 (1.73–12.53)	1.13 (0.53–4.66)	<0.001
Time on unscheduled days (min)	5.03 (4.07–5.59)	5.33 (4.48–5.76)	0.12
Pajama time (min)	3.44 (3.21–3.88)	3.20 (2.96–3.50)	0.003
	May 2020
	Primary care (95% CI)	Specialty care (95% CI)	p-Value
Appointments per day	3.00 (2.00–4.31)	3.02 (2.00–4.88)	0.74
Aggregate messages per day	0.94 (0.73–1.00)	0.97 (0.66–1.00)	0.99
Same day visit closure rate	27.65 (23.71–42.30)	31.94 (23.88–45.71)	0.95
PEP score	25.60 (6.71–48.32)	12.47 (9.41–50.15)	0.92
Proficiency score	16.33 (2.55–25.35)	10.68 (3.68–17.10)	0.40
Time in notes (min per appointment)	7.89 (6.01–9.71)	11.74 (7.23–15.09)	0.03
Turnaround time (min per appointment)	2.85 (0.69–5.35)	0.58 (0.09–1.35)	<0.001
Time in In Basket (min per appointment)	4.63 (3.03–7.28)	2.55 (1.13–5.09)	<0.001
Time outside of 7 a.m.–7 p.m. (min)	4.51 (1.05–14.89)	1.21 (0.16–4.18)	<0.001
Time on unscheduled days (min)	5.44 (4.98–5.80)	5.13 (4.83–5.62)	0.11
Pajama time (min)	3.37 (2.23–3.87)	3.09 (2.06–4.34)	0.12
	July 2020
	Primary care (95% CI)	Specialty care (95% CI)	p-Value
Appointments per day	4.25 (3.00–5.73)	3.22 (2.27–4.34)	0.005
Aggregate messages per day	0.88 (0.59–1.00)	1.00 (0.59–1.00)	0.55
Same day visit closure rate	30.54 (22.09–39.98)	21.97 (10.51–70.54)	0.48
PEP score	25.30 (12.61–41.62)	16.31 (8.15–33.06)	0.33
Proficiency score	10.58 (3.92–26.11)	5.74 (1.77–16.97)	0.31
Time in notes (min per appointment)	10.96 (9.65–14.41)	10.97 (5.99–20.58)	0.91
Turnaround time (min per appointment)	3.82 (2.22–6.32)	0.64 (0.16–1.26)	<0.001
Time in In Basket (min per appointment)	5.57 (3.74–8.07)	2.86 (1.55–4.99)	0.004
Time outside of 7 a.m.–7 p.m. (min)	6.89 (2.29–17.80)	0.76 (0.10–3.96)	<0.001
Time on unscheduled days (min)	5.12 (4.66–5.47)	5.53 (4.97–5.80)	0.12
Pajama time (min)	3.46 (3.30–3.97)	3.24 (2.99–3.46)	0.002

Abbreviations: CI, confidence interval; PEP, Physician Efficiency Profile.

Note: Statistically significant p-values are provided in bold.

Trainee Outcome Metrics versus Attending Outcome Metrics

A majority of the outcome metrics differed between trainees and attending physicians ([Table 4]). Patient volume was higher for attending physicians during all three time points, as illustrated by appointments per day and aggregate messages (p < 0.01). Time in In Basket (minutes per day) was greater across all three time periods for attending physicians compared with trainees (p < 0.01). Attending physicians spent significantly more time in the EHR outside of 7 a.m. to 7 p.m. in August of 2019 compared with trainees (19.3 vs. 14.2 total minutes per day, p < 0.01), but showed no statistically significant difference in May of 2020 (12.02 vs. 11.7 minutes per day, p = 0.65). Attending physicians spent significantly more time in the EHR outside of 7 a.m. to 7 p.m. compared with trainees in July of 2020 (8.39 vs. 16.2 total minutes per day, p < 0.01).

Table 4 Descriptive statistics of outcome metrics (trainees vs. attendings)

	August 2019
	Resident/fellow (95% CI)	Attending (95% CI)	p-Value
Appointments per day	4.76 (3.50–6.50)	11.6 (8.0–17.3)	<0.001
Aggregate messages per day	4.98 (2.68–8.31)	14.7 (9.4–23.3)	0.007
Same day visits closure rate	0.72 (0.38–1.00)	0.7 (0.5–0.9)	0.38
PEP score	5.13 (4.28–5.65)	4.8 (4.0–5.6)	<0.001
Proficiency score	3.36 (3.11–3.81)	4.0 (2.8–5.7)	<0.001
Time in notes (min per appointment)	13.55 (10.05–17.90)	6.2 (3.8–9.7)	<0.001
Turnaround time (min per appointment)	2.67 (0.89–9.81)	4.0 (1.9–8.7)	0.20
Time in In Basket (min per appointment)	3.47 (0.93–6.92)	10.3 (5.1–17.5)	<0.001
Time outside of 7 a.m.–7 p.m. (min)	14.23 (3.70–31.15)	19.3 (6.7–39.1)	<0.001
Time on unscheduled days (min)	36.12 (18.66–47.72)	36.5 (20.3–64.4)	0.09
Pajama time (min)	32.73 (12.32–56.69)	28.6 (12.0–60.5)	0.80
	May 2020
	Resident/fellow (95% CI)	Attending (95% CI)	p-Value
Appointments per day	3.00 (2.00–4.48)	7.6 (5.3–10.7)	<0.001
Aggregate messages per day	4.12 (1.99–6.29)	10.0 (5.9–17.8)	<0.001
Same day visit closure rate	0.95 (0.71–1.00)	0.9 (0.7–1.0)	<0.001
PEP score	5.26 (4.85–5.80)	4.9 (3.9–5.7)	0.001
Proficiency score	3.27 (2.09–3.80)	4.0 (2.8–5.9)	<0.001
Time in notes (min per appointment)	9.58 (6.60–12.16)	5.9 (3.8–9.3)	<0.001
Turnaround time (min per appointment)	2.71 (0.48–10.34)	3.0 (1.3–7.6)	0.13
Time in In Basket (min per appointment)	1.40 (0.42–4.08)	7.9 (3.5–15.2)	<0.001
Time outside of 7 a.m.–7 p.m. (min)	12.02 (3.29–23.79)	11.7 (4.7–25.2)	0.65
Time on unscheduled days (min)	27.67 (23.61–43.80)	24.5 (13.3–42.5)	0.08
Pajama time (min)	17.80 (9.40–49.61)	15.1 (5.8–35.9)	0.34
	July 2020
	Resident/fellow (95% CI)	Attending (95% CI)	p-Value
Appointments per day	3.73 (2.64–5.00)	9.6 (6.6–13.1)	<0.001
Aggregate messages per day	4.47 (2.28–7.09)	16.6 (10.1–27.3)	<0.001
Same day visit closure rate	0.90 (0.59–1.00)	0.8 (0.5–1.0)	0.002
PEP score	5.18 (4.72–5.69)	4.8 (3.9–5.6)	0.01
Proficiency score	3.38 (3.15–3.70)	3.9 (2.5–5.9)	<0.001
Time in notes (min per appointment)	10.97 (9.21–14.90)	5.6 (3.5–8.9)	<0.001
Turnaround time (min per appointment)	3.51 (0.82–15.09)	3.2 (1.4–7.3)	0.97
Time in In Basket (min per appointment)	2.46 (0.73–6.00)	9.6 (4.9–18.1)	<0.001
Time outside of 7 a.m.–7 p.m. (min)	8.39 (2.56–22.09)	16.2 (6.0–31.9)	0.003
Time on unscheduled days (min)	26.69 (20.18–40.31)	31.5 (16.5–55.6)	0.42
Pajama time (min)	24.29 (10.76–40.26)	25.0 (7.6–46.8)	0.99

Abbreviations: CI, confidence interval; PEP, Physician Efficiency Profile.

Note: Statistically significant p-values are provided in bold.

Physician's “efficiency” metrics, such as PEP score and proficiency score, were also significantly greater for attending physicians compared with trainees across all three time periods (p < 0.01), while Time in notes (minutes per day) was consistently lower in attending physicians compared with trainees for all three time periods (p < 0.01). After the onset of COVID-19, same day visit closure rate was lower in attending physicians compared with trainees in both May 2020 (0.95 vs. 0.9, p < 0.001) and July 2020 (0.90 vs. 0.80, p = 0.002), but there was no statistically significant difference prepandemic (0.72 vs. 0.7, p = 0.38).

Differences in Outcome Metric Trainees and Attending Physicians from Pre- and during COVID-19

The median change (delta) in each performance metric from August 2019 to May 2020 and from August 2019 to July 2020 was calculated for each performance metric and for all trainees and all attending physicians, separately ([Table 5]). The delta values in performance metric among each group were then compared to determine if any significant changes were present.

Table 5 Difference in outcome metrics from pre- and post-COVID-19 (trainees vs. attendings)

Outcome metrics	Δ Aug 2019 → May 2020			Δ Aug 2019 → July 2020
	Residents and fellows (95% CI)	Attendings (95% CI)	p-Value	Residents and fellows (95% CI)	Attendings (95% CI)	p-Value
Appointments per day	−1.57 (−3.01 to −0.13)	−2.80 (−5.64 to −0.41)	<0.001	−0.47 (−2.24 to 0.66)	−1.63 (−4.25 vs. −0.07)	<0.001
Same day visit closure rate	0.04 (−0.03 to 0.36)	0.05 (−0.019 to 0.21)	0.37	0.04 (−0.01 to 0.27)	0.01 (−0.05 vs. 0.13)	0.03
Time on unscheduled days (min)	−3.36 (−26.25 to 9.18)	−8.11 (−28.53 to 3.78)	0.33	−5.71 (−14.19 to 11.01)	−3.49 (−17.38 vs. 7.10)	0.57
Pajama time (min)	−6.54 (−37.15 to 5.26)	−7.85 (−28.10 to 1.90)	0.58	−3.66 (−20.18 to 3.12)	−3.66 (−19.39 vs. 5.45)	0.68
Time outside of 7 a.m.–7 p.m. (min)	−3.13 (16.27 to 4.59)	−4.51 (−17.59 to 2.86)	0.47	−6.62 (−17.93 to 1.51)	−2.63 (−13.13 vs. 5.38)	0.11
Time in notes (min per appointment)	−2.80 (−6.65 to −1.28)	−0.10 (−1.99 to 1.06)	<0.001	−3.82 (−6.83 to 0.33)	−0.16 (−1.40 vs 0.91)	<0.001
Time in In Basket (min per appointment)	−0.93 (−4.31 to 0.33)	−1.16 (−4.39 to 0.76)	0.81	−0.26 (−1.30 to 0.75)	0.01 (−2.18 vs. 2.18)	0.143
Aggregate messages per day	−0.52 (−3.31 to 1.46)	−3.09 (−7.23 to 0.17)	<0.001	−0.01 (−1.42 to 1.93)	1.13 (−1.69 vs. 4.39)	0.04
Turnaround time (min per appointment)	−0.22 (4.72 to 5.03)	−0.34 (−3.31 to 2.14)	0.58	0.02 (−2.32 to 8.78)	−0.27 (−2.37 vs. 1.68)	0.04
Proficiency score	−0.09 (−1.24 to 0.21)	0.07 (−0.24 to 0.30)	<0.001	0.13 (−0.09 to 0.38)	0.09 (−0.13 vs. 0.28)	0.26

Abbreviations: CI, confidence interval.

Note: Statistically significant p-values are provided in bold.

In terms of median delta across all performance metrics, patient volume for attendings decreased across both time periods, as indicated by negative delta values for appointments per day, while the absolute delta among trainees was significantly smaller across both May (−1.57 vs. −2.80 appointments per day, p < 0.001) and July of 2020 (−0.47 vs. −1.63 appointments per day, p < 0.001), as compared with pre-COVID-19, indicating that attending physicians saw a greater decrease in patient volume. The decrease in aggregate messages among trainees was significantly smaller than attendings between August of 2019 and May of 2020 (−0.52 vs. −3.09 aggregate messages per day, p < 0.001), whereas between August of 2019 and May of 2020, there was a decrease among trainees but an increase among attendings and this difference was significant (−0.01 vs. 1.13 aggregate messages per day, p = 0.04).

Efficiency metrics were also affected as there was a decrease in proficiency score among trainees and an increase among attendings comparing May 2020 to August 2019, which showed a significant difference (p < 0.01), but no significant differences were found in July 2020 as compared with August 2019 (p = 0.26). There were no statistically significant differences in the changes in time on unscheduled days, pajama time, Time outside of 7 a.m. to 7 p.m., and Time in In Basket across both time periods, indicating that both groups were similarly affected during COVID-19 in these metrics. Last, the changes in the majority of studied metrics were negative across both trainees and attending physicians and across all timespans, indicating a decrease in performance metric after the onset of the pandemic.

Discussion

This investigation presents a previously undescribed analysis regarding the impact of the COVID-19 pandemic and telemedicine implementation on EHR usage patterns of medical trainees. The primary strengths of this work are that we evaluate a relatively high number of physicians (n = 636) during a time of rapidly increasing utilization of telemedicine, and we analyze several Epic variables to measure performance and burnout. Using a nonintrusive method of analyzing EHR usage patterns, we can compare trends among trainees and attending physicians, and further stratify by both sex and specialty. Our current study illustrates that (1) during May of 2020 in the pandemic many metrics decreased among trainees and attending physicians, but recovered in July of 2020, although below baseline values; (2) female resident, fellows, and attending physicians spent more time documenting notes in the EHR as compared with male physicians; (3) primary care physicians had a higher patient volume and demonstrated greater efficiency metrics in the EHR as compared with specialty care physicians; and (4) attending physicians had a greater patient volume, spent more time in the EHR, and demonstrated greater efficiency metrics in the EHR compared with trainees.

We observed that many metrics decreased for trainees during the peak phases of the pandemic, including Time in In Basket, Time outside of 7 a.m. to 7 p.m., Time in notes, and appointments per day. Given the declining outpatient volume during this time, many of these outcomes are expected, as several other studies have showed.[29] [30] [31] Several of these metrics were still significantly lower in July 2020 compared with the prepandemic period, though it is reasonable to expect that these may continue to return to the prepandemic baseline with time, as Holmgren et al reported a recovery in patient volume via the EHR by July of 2020.[32] We have also previously shown a similar recovery during the same time span among attending physicians in MHS.[33] Indeed, while patient volume stabilized similarly to prepandemic levels, Phadke et al suggest that the nature of physician health care likely changed, such that e-consult utilization and telehealth modalities of care increased relative to traditional ambulatory care.[34] Similarly, Jelinek et al found that the proportion of telemedicine encounters across races and languages steadily increased between April and November 2020, which further highlights the increasing implementation of telehealth and e-consult utilization in health care settings.[35] Ultimately, the effects of emerging health care modalities on patients, providers, and health care systems should be further studied.

Although our findings would suggest that physicians were less burdened in their outpatient clinic work, as they were spending less time on patient care, especially after hours, many factors were not captured by our study such as the redeployment of physicians to inpatient services. Although less time spent in the EHR has been associated with reduced burnout, the findings from several other studies suggest that there was an increase in burnout during the pandemic.[36] [37] [38] [39] [40] This implies that there are other sources of burnout,[41] [42] despite reduced outpatient volume, or that physicians were picking up additional non-outpatient duties that are not captured by our analysis. Several examples may include physicians being called to work in areas outside of their own specialties, caring for patients without adequate personal protective equipment, and risking their lives and those of their families by increased exposure to SARS-CoV-2. Holzer et al also noted that physician trainees with increased self-reported EHR use since the onset of the pandemic had higher rates of burnout, depression, and posttraumatic stress disorder.[24] This further suggests a multi-modal component of burnout, which incorporates both site of care, metrics included in this study, and other factors, such as autonomy and uncertainty.[38] [41] [43] [44] [45] [46]

Provider-level analyses indicated that female physicians had lower same day closure rate prepandemic, which may have led to their greater time spent outside of hours working and increased pajama time, on average, compared with male physicians. Interestingly, this trend of female physicians spending more time documenting in the EHR after hours is maintained across all training levels in MHS (residents, fellows, attendings).[33] [47] One possible explanation was identified by Gupta et al noted in a related UCSF Health-based case study that found women physicians spent more time in the EHR because they documented longer notes, which possibly contributed to greater burnout.[48] Additionally, differences between male and female physicians may be attributed to gender-specific differences with respect to childcare and familial obligations. Tait et al found that female physicians spent on average 33 more minutes in the EHR system per day, in a study of almost 1,000 physicians, and suggested that interventions reducing time spent in the EHR may further reduce gender-based disparities in the medical field.[49] It is likely then that further research on these gender-based disparities may reduce overall burnout for all physicians.[44]

Notably, we observed several differences between primary care residents and specialty care residents, such as primary care residents had a higher volume of patients than subspecialist residents, similar to Overhage and McCallie,[50] and that primary care residents demonstrated higher proficiency metrics with the EHR both pre- and post-COVID-19. One explanation for these findings is that primary care physicians may have higher patient loads,[51] and are thus required to better learn EHR tools and functions to increase their efficiency. Other studies have noted that family medicine residents spend significant time in the EHR after hours,[13] and that more hours spent in the EHR after hours was associated with burnout and decreased satisfaction among primary care residents.[18] Psychiatry residents were also found to have a high correlation between EHR use and burnout, which further corroborates this finding.[23] Overall, the finding that residents in primary care fields see more patients and are also more proficient in the tools provided by the EHR highlights the unique roles that different physicians play in medical care and suggests that primary care residents may benefit from increased telemedicine adoption to reduce provider burden.

In comparing trainee's performance to that of attending physicians, we observed some consistent trends, including higher overall patient volume seen by attendings and more time spent messaging across all three time points. These findings are expected as attendings are typically scheduled to see more patients and have shorter time windows to see each patient. Another study did note similar results between attending and resident physicians, and specifically found that residents spent more time per note compared with attendings.[9] We also observed that attendings had higher proficiency scores with the EHR, suggesting they are more familiar with the tools offered by the EHR. A study evaluating Internal Medicine usage patterns found that residents focused primarily on information review rather than information entry, which may explain our finding.[52] As residents are in their training phase, they may need more time completing their notes, compared with attending physicians who have greater experience.

Our study also has limitations. While EHR metrics may function as a surrogate for physician productivity, efficiency, and burden, they may not entirely replace qualitative surveys of individual physicians. While the EHR can act as a proxy for doctor–patient encounters, work performed, and physician satisfaction, it certainly cannot capture these measures entirely. In this study, we were unable to survey physicians or use already collected burnout surveys administered as they were not administered uniformly across departments or at uniform time intervals. Additionally, the metrics themselves have limitations, as we are operating based on Epic's definitions of efficiency, proficiency, and normal working hours. Likewise, Epic's “efficiency” performance metrics do not correlate perfectly with actual efficiency. There also may have been limitations with granularity, and details not being entirely captured, such as when patient encounters in MHS are entered in the EHR late after a visit, when providers are only part-time clinical or scheduled for half-day clinic sessions versus full days, or when physicians work different hours, which may affect variables such as time spent outside of 7 a.m. to 7 p.m. Another limitation is that patterns noted here may reflect changes in the group of physicians being evaluated. Due to the inpatient focus of residency, data from each time point are a composite of how residents are performing and may be capturing a different cohort of residents at each time window. Different patient populations or other non-workload factors may also impact differences in metrics across physician groups. Last, the results from this study are from a single institution's health system and may not be generalizable to institutions across the rest of the country.

Conclusion

The dramatic change in clinical operations during the pandemic enables us to use this period as an inflection point to study changes in physician practice patterns via the EHR. We observed that (1) female physicians closed fewer notes the same day and spent more time in the EHR outside of normal working hours compared with male physicians, and (2) attending physicians had higher patient volumes, spent more time messaging in Epic's In Basket communications portal, and had higher efficiency in the EHR compared with resident physicians. Ultimately, the EHR is a powerful clinical tool that can be utilized to evaluate the well-being of trainees from both a burnout perspective and from growth and learning perspectives.

Clinical Relevance Statement

This study provides further evidence that electronic health record metrics may vary significantly in the context of provider-level characteristics. Research aimed at understanding trainees' and attending physician usage patterns can streamline health policy efforts aimed at improving physician efficiency and burnout. Further research should be conducted to test how various changes to hospital operations and system changes (i.e., new legislation) may impact physicians.

Multiple-Choice Questions

1. Which of the following has been found to correlate with increased physician trainees' burnout?

   • Objective and self-reported measures of significant time after work hours in the EHR

   • Being a fellow as compared with a resident

   • Less patient volume and appointments per day

   • Prepandemic time period

   Correct Answer: The correct answer is option a. While the EHR has many benefits to physicians, including standardizing documentation, reducing errors, and enabling better data sharing between physicians, it is also often cited in physician surveys as a common source of burden and even burnout in modern medicine. Previous studies have also found that both self-reported and objective measures of time spent in the EHR are associated with physician burnout, as validated by the Maslach Burnout Inventory and AMA mini-Z measurement surveys.[15] [18] [21] [22]

2. Based on the results of our study, which of the following EHR metrics was greater in trainees as compared with attending physicians during our studied time periods?

   • Patient volume

   • Time in In Basket

   • Physician “efficiency” metrics, such as PEP and proficiency score

   • Time in notes

   Correct Answer: The correct answer is option d. Our results demonstrated that trainees spent more Time in notes compared with attending physicians, whereas attending physicians had a greater patient volume and Time in In Basket, while also having higher PEP and proficiency scores (“efficiency” metrics).

Conflict of Interest

None declared.

Acknowledgments

We sincerely thank Dr. Juan Lin for her guidance with statistical analysis.

Protection of Human and Animal Subjects

This study complied with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Albert Einstein Institutional Review Board. No human subjects were involved in this project.

^# These authors contributed equally to the manuscript.

• References

• 1 Gale R, Eberlein S, Fuller G, Khalil C, Almario CV, Spiegel BMR. Public perspectives on decisions about emergency care seeking for care unrelated to COVID-19 during the COVID-19 pandemic. JAMA Netw Open 2021; 4 (08) e2120940
• 2 Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US. JAMA Intern Med 2021; 181 (03) 388-391
• 3 Chao GF, Li KY, Zhu Z. et al. Use of telehealth by surgical specialties during the COVID-19 pandemic. JAMA Surg 2021; 156 (07) 620-626
• 4 Chu C, Cram P, Pang A, Stamenova V, Tadrous M, Bhatia RS. Rural telemedicine use before and during the COVID-19 pandemic: repeated cross-sectional study. J Med Internet Res 2021; 23 (04) e26960
• 5 Grange ES, Neil EJ, Stoffel M. et al. Responding to COVID-19: the UW Medicine Information Technology Services experience. Appl Clin Inform 2020; 11 (02) 265-275
• 6 Moore C, Valenti A, Robinson E, Perkins R. Using log data to measure provider EHR activity at a cancer center during rapid telemedicine deployment. Appl Clin Inform 2021; 12 (03) 629-636
• 7 Hron JD, Parsons CR, Williams LA, Harper MB, Bourgeois FC. Rapid implementation of an inpatient telehealth program during the COVID-19 pandemic. Appl Clin Inform 2020; 11 (03) 452-459
• 8 Chiu CY, Sarwal A, Jawed M, Chemarthi VS, Shabarek N. Telemedicine experience of NYC Internal Medicine residents during COVID-19 pandemic. PLoS One 2021; 16 (02) e0246762
• 9 Gursky JM, Boro A, Escalante S. et al. Disparities in access to neurologic telemedicine during the COVID-19 pandemic: a Bronx tale. Neurol Clin Pract 2021; 11 (02) e97-e101
• 10 Watts KL, Abraham N. “Virtually Perfect” for some but perhaps not for all: launching telemedicine in the bronx during the COVID-19 pandemic. J Urol 2020; 204 (05) 903-904
• 11 Alipour J, Hayavi-Haghighi MH. Opportunities and challenges of telehealth in disease management during COVID-19 pandemic: a scoping review. Appl Clin Inform 2021; 12 (04) 864-876
• 12 deMayo R, Huang Y, Lin ED. et al. Associations of telehealth care delivery with pediatric health care provider well-being. Appl Clin Inform 2022; 13 (01) 230-241
• 13 Anderson J, Leubner J, Brown SR. EHR overtime: an analysis of time spent after hours by family physicians. Fam Med 2020; 52 (02) 135-137
• 14 Saag HS, Shah K, Jones SA, Testa PA, Horwitz LI. Pajama time: working after work in the electronic health record. J Gen Intern Med 2019; 34 (09) 1695-1696
• 15 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 16 Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause?. Ann Intern Med 2018; 169 (01) 50-51
• 17 Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc 2019; 2019: 136-144
• 18 Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I³ population collaborative. J Grad Med Educ 2017; 9 (04) 479-484
• 19 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 20 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
• 21 Eschenroeder HC, Manzione LC, Adler-Milstein J. et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021; 28 (05) 960-966
• 22 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 23 Domaney NM, Torous J, Greenberg WE. Exploring the association between electronic health record use and burnout among psychiatry residents and faculty: a pilot survey study. Acad Psychiatry 2018; 42 (05) 648-652
• 24 Holzer KJ, Lou SS, Goss CW. et al. Impact of changes in EHR use during COVID-19 on physician trainee mental health. Appl Clin Inform 2021; 12 (03) 507-517
• 25 Mosquera MJ, Ward HB, Holland C, Boland R, Torous J. Using objective clinical metrics to understand the relationship between the electronic health record and physician well-being: observational pilot study. BJPsych Open 2021; 7 (05) e174
• 26 Livingston K, Bovi J. Department-focused electronic health record thrive training. JAMIA Open 2022; 5 (02) ooac025
• 27 Khairat S, Zalla L, Gartland A, Seashore C. Association between proficiency and efficiency in electronic health records among pediatricians at a major academic health system. Front Digit Health 2021; 3: 689646
• 28 Hollister-Meadows L, Richesson RL, De Gagne J, Rawlins N. Association between evidence-based training and clinician proficiency in electronic health record use. J Am Med Inform Assoc 2021; 28 (04) 824-831
• 29 Naidich JJ, Boltyenkov A, Wang JJ, Chusid J, Hughes D, Sanelli PC. Impact of the coronavirus disease 2019 (COVID-19) pandemic on imaging case volumes. J Am Coll Radiol 2020; 17 (07) 865-872
• 30 Berkenstock MK, Liberman P, McDonnell PJ, Chaon BC. Changes in patient visits and diagnoses in a large academic center during the COVID-19 pandemic. BMC Ophthalmol 2021; 21 (01) 139
• 31 Berg GM, Wyse RJ, Morse JL. et al. Decreased adult trauma admission volumes and changing injury patterns during the COVID-19 pandemic at 85 trauma centers in a multistate healthcare system. Trauma Surg Acute Care Open 2021; 6 (01) e000642
• 32 Holmgren AJ, Downing NL, Tang M, Sharp C, Longhurst C, Huckman RS. Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use. J Am Med Inform Assoc 2022; 29 (03) 453-460
• 33 Beiser M, Lu V, Paul S. et al. Electronic health record usage patterns: assessing telemedicine's impact on the provider experience during the COVID-19 pandemic. Telemed J E Health 2021; 27 (08) 934-938
• 34 Phadke NA, Del Carmen MG, Goldstein SA. et al. Trends in ambulatory electronic consultations during the COVID-19 pandemic. J Gen Intern Med 2020; 35 (10) 3117-3119
• 35 Jelinek R, Pandita D, Linzer M, Engoang JBBN, Rodin H. An evidence-based roadmap for the provision of more equitable telemedicine. Appl Clin Inform 2022; 13 (03) 612-620
• 36 Lasalvia A, Amaddeo F, Porru S. et al. Levels of burn-out among healthcare workers during the COVID-19 pandemic and their associated factors: a cross-sectional study in a tertiary hospital of a highly burdened area of north-east Italy. BMJ Open 2021; 11 (01) e045127
• 37 Torrente M, Sousa PA, Sánchez-Ramos A. et al. To burn-out or not to burn-out: a cross-sectional study in healthcare professionals in Spain during COVID-19 pandemic. BMJ Open 2021; 11 (02) e044945
• 38 Amanullah S, Ramesh Shankar R. The impact of COVID-19 on physician burnout globally: a review. Healthcare (Basel) 2020; 8 (04) 421
• 39 Abdulah DM, Musa DH. Insomnia and stress of physicians during COVID-19 outbreak. Sleep Med X 2020; 2: 100017
• 40 Civantos AM, Byrnes Y, Chang C. et al. Mental health among otolaryngology resident and attending physicians during the COVID-19 pandemic: national study. Head Neck 2020; 42 (07) 1597-1609
• 41 Tajirian T, Stergiopoulos V, Strudwick G. et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res 2020; 22 (07) e19274
• 42 Johnson KB, Neuss MJ, Detmer DE. Electronic health records and clinician burnout: a story of three eras. J Am Med Inform Assoc 2021; 28 (05) 967-973
• 43 Baptista S, Teixeira A, Castro L. et al. Physician burnout in primary care during the COVID-19 pandemic: a cross-sectional study in Portugal. J Prim Care Community Health 2021;12:21501327211008437
• 44 McPeek-Hinz E, Boazak M, Sexton JB. et al. Clinician burnout associated with sex, clinician type, work culture, and use of electronic health records. JAMA Netw Open 2021; 4 (04) e215686
• 45 Linzer M, Stillman M, Brown R. et al; American Medical Association–Hennepin Healthcare System Coping With COVID Investigators. Preliminary report: US physician stress during the early days of the COVID-19 pandemic. Mayo Clin Proc Innov Qual Outcomes 2021; 5 (01) 127-136
• 46 Shanafelt TD, West CP, Sinsky C. et al. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2020. Mayo Clin Proc 2022; 97 (03) 491-506
• 47 Ruan E, Beiser M, Lu V. et al. Physician electronic health record usage as affected by the COVID-19 pandemic. Appl Clin Inform 2022; 13 (04) 785-793
• 48 Gupta KMS, Sarkar U, Mourad M, Adler-Milstein J. Differences in ambulatory EHR use patterns for male vs. female physicians. NEJM Catal 2019
• 49 Tait SD, Oshima SM, Ren Y. et al. Electronic health record use by sex among physicians in an academic health care system. JAMA Intern Med 2021; 181 (02) 288-290
• 50 Overhage JM, McCallie Jr D. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med 2020; 172 (03) 169-174
• 51 Young RA, Burge SK, Kumar KA, Wilson JM, Ortiz DF. A time-motion study of primary care physicians' work in the electronic health record era. Fam Med 2018; 50 (02) 91-99
• 52 Wang JK, Ouyang D, Hom J, Chi J, Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One 2019; 14 (02) e0205379

Publikationsverlauf

Eingereicht: 29. Juli 2022

Angenommen: 06. Februar 2023

Accepted Manuscript online:
09. Februar 2023

Artikel online veröffentlicht:
26. April 2023