Physician Electronic Health Record Usage as Affected by the COVID-19 Pandemic

Abstract

Objectives To utilize metrics from physician action logs to analyze volume, physician efficiency and burden as impacted by telemedicine implementation during the COVID-19 (coronavirus disease 2019) pandemic, and physician characteristics such as gender, years since graduation, and specialty category.

Methods We selected 11 metrics from Epic Signal, a functionality of the Epic electronic health record (EHR). Metrics measuring time spent in the EHR outside working hours were used as a correlate for burden. We performed an analysis of these metrics among active physicians at our institution across three time periods—prepandemic and telehealth implementation (August 2019), postimplementation of telehealth (May 2020), and follow-up (July 2020)—and correlated them with physician characteristics.

Results Analysis of 495 physicians showed that after the start of the pandemic, physicians overall had fewer appointments per day, higher same day visit closure rates, and spent less time writing notes in the EHR outside 7 a.m. to 7 p.m. on patient scheduled days. Across all three time periods, male physicians had better EHR-defined “efficiency” measures and spent less time in the EHR outside working hours. Years since graduation only had modest associations with higher same day visit closure rates and appointments per day in May 2020. Specialty category was significantly associated with appointments per day and same day closure visit rates and also was a significant factor in the observed changes seen across the three time periods.

Conclusion Utilizing EHR-generated reports may provide a scalable and nonintrusive way to monitor trends in physician usage and experience to help guide health systems in increasing productivity and reducing burnout.

Keywords

electronic health record - telemedicine - COVID-19 pandemic - burden

Background and Significance

In the spring of 2020, New York City, including the Montefiore Health System (MHS), had its first surge of coronavirus disease 2019 (COVID-19) patients, requiring a rapid and large-scale response from hospital systems. Both inpatient and outpatient operations were restructured to address the increased patient needs, as well as to mitigate the spread of the virus in such a dense population. Although some changes were temporary, such as the cancellation of elective procedures and redeployment of physicians to inpatient services, other changes, such as the widespread implementation of telemedicine, remained in place even after the surge ended.[1] [2]

Physician experience and burden as affected by these unprecedented changes have understandably been a focus of recent studies.[3] [4] [5] [6] [7] Surveys of physicians have sought to provide insight into how these changes may impact work-flow and how that may affect burnout rates.[6] [7] However, with the heterogeneity of the target population, given different specialties, practice locations, and responsibilities, as well as the often rapidly changing clinical environment, surveys may be too burdensome and lack the granularity needed to fully understand how changes may be affecting physicians. A few limited studies have examined the feasibility of electronic health record (EHR)-generated metrics to reflect the changes in physician usage patterns surrounding the time of the pandemic.[3] [8] One study, limited to a cancer center, utilized such metrics and found that during the initial COVID peak, physicians were spending more time in the EHR, on documentation, and in their inbox, although these metrics returned to near baseline in the follow-up period 3 months later.[9] Our prior study compared metrics derived from provider action logs from two time periods (August 2019 and May 2020) to examine differences due to factors relating to the pandemic and the implementation of telemedicine.[3] Our preliminary findings identified not only differences in EHR usage between the two periods, but also other factors such as individual characteristics (gender, years since graduation) and specialty category that may also contribute to these differences. Following these initial observations, this study seeks to further this exploration by expanding the number and types of metrics evaluated and by broadly including all active physicians who practiced in any of the outpatient clinics in MHS, which includes primary care, subspecialty care, and surgical specialty care clinics.

With the growing presence and impact of EHR and other technologies on physicians and burnout, there is a need for a scalable and nonintrusive method to measure physician usage and experience. Understanding the impact of both systemic-level factors and user-level characteristics would be imperative to ongoing efforts to support and optimize physician experience and health care delivery.

Objectives

The objectives of this study were to explore physician EHR usage patterns during the COVID-19 pandemic through (1) examining variation in EHR-generated metrics across three time periods that correlate with preimplementation of telemedicine and pre-COVID (August 2019), the initial telemedicine postimplementation period (May 2020), and a follow-up period where both telemedicine and in-person visits were utilized (July 2020), and (2) determining the association of gender, specialty category, and years since graduation with these variations in user-level outcome metrics among the three time periods.

Methods

We used Epic Signal, a tool within Epic, the EHR used across MHS since 2014, that generates documentation data that have been previously found useful to studying physician usage patterns.[3] [10] We included all attending physicians who were active in the EHR during three 1-month periods. August 2019 was selected as the baseline pretelemedicine implementation period. Telemedicine implementation occurred during the initial surge of patients with COVID-19 in March of 2020. Therefore, May 2020 was selected as the postimplementation period and July 2020 as the follow-up and return-to-normal operation time interval (with telemedicine and in-person clinical visits) to evaluate for persistence of changes.[1]

Physician characteristic data were collected to evaluate for associations between gender, specialty type (primary care vs. subspecialty vs. surgery), and years since graduation of their degree-granting program with our 11 metrics of interest. We selected metrics using the Signal web-based dashboard, focusing on metrics displayed under “Overview,” “In Basket,” “Notes and Letters,” and “Workload” metrics as we hypothesized that these would be impacted by the pandemic and subsequent shift from in-person to telehealth to hybrid. Patient volume was assessed using number of appointments per day (number of appointments/number of scheduled days) and number of aggregate messages received per day. Signal has measures for “efficiency” developed by Epic that have been used by prior studies to study changes in physician workflow and usage patterns.[11] [12] [13] We included these with the understanding that they may not correlate perfectly with actual efficiency: (1) same day visit closure rate; (2) Physician Efficiency Profile (PEP) score, which compares time the user spent in the system and expected time in the system based on workload, calculated using all other users in the system, with a higher score indicating less time spent compared with expected[11]; (3) proficiency score (measures frequency of use of embedded EHR efficiency tools)[12] [13]; (4) time (minutes) spent in notes per appointment; (5) turnaround time (days to respond to in-basket messages); and (6) time (minutes) spent in in-basket per day. When selecting which metrics to use to assess for physician burden, we used metrics that evaluate time spent in the EHR outside working hours. Studies have demonstrated associations between both self-reported[14] [15] [16] and objective[17] measures of time spent in the EHR with physician burnout as measured by validated surveys such as the Maslach Burnout Inventory survey and the American Medical Association mini-Z measurement. We evaluated burden using the following metrics: (1) time (average minutes) spent in EHR outside of 7 a.m. to 7 p.m.; (2) time (average minutes) spent in EHR on unscheduled days (days with no scheduled patients); and (3) pajama time (average minutes spent in the EHR outside of 7 a.m. to 5:30 p.m. on weekdays and outside unscheduled time on weekends).

Descriptive statistics were generated for all 11 metrics across the three time periods among all physicians, by specialty group, and by gender. The Wilcoxon signed rank test[18] was used to compare the differences in metrics between both postimplementation periods and pre-COVID for the overall sample. The Kruskal–Wallis test or ANOVA (analysis of variance) was used to compare the differences among the three specialty types within each time period, depending on the distribution of the data, and the Mann–Whitney U test was used to compare the differences between gender within each time period.[19] The Spearman rank correlation coefficient[20] was used to assess the associations between four selected metrics (appointments per day, same day visit closure rate, time spent outside of 7 a.m.–7 p.m., and PEP score) with years since graduation. p-Values were not adjusted for multiple testing.

Multivariable analyses using linear regression models were performed for the four selected metrics to evaluate the independent effects of physician characteristics including gender, specialty category, and years since graduation in May 2020 and July 2020 after adjusting for prepandemic levels.

Results

We analyzed data from 495 physicians across 55 specialties and subspecialties. In total, 53% (n = 263) of physicians were female; 36% (n = 176) of physicians were in primary care; 53% (n = 263) in subspecialty care; and 11% (n = 56) in surgical specialties. The average year since graduation was 23 years.

Outcome Metrics Across Time for All Physicians

There was significant variation in outcome metrics across the three time periods ([Fig. 1]). Patient volume in the initial postimplementation period of May 2020 was significantly decreased compared with the volume in August 2019 ([Table 1]), as measured by both number of appointments per day (7.6 compared with 11.6, p < 0.01) and aggregate messages (decreased to 10.0 from 14.7, p < 0.01). However, while the median number of appointments per day in July 2020 also remained significantly less than prepandemic numbers (9.6 compared with 11.6, p < 0.01), aggregate messages increased significantly from 14.7 to 16.6 (p < 0.01).

Table 1 Descriptive statistics of outcome metrics for pre- and post-COVID across all physicians (N = 495)

	Time period			p-Value (May vs. August)	p-Value (July vs. August)
	August 2019	May 2020	July 2020
Appointments per day	11.6 (8.0–17.3)	7.6 (5.3–10.7)	9.6 (6.6–13.1)	<0.01	<0.01
Aggregate messages	14.7 (9.4–23.3)	10.0 (5.9–17.8)	16.6 (10.1–27.3)	<0.01	<0.01
Same day visit closure rate	0.7 (0.5–0.9)	0.9 (0.7–1.0)	0.8 (0.5–1.0)	<0.01	<0.01
PEP score	4.8 (4.0–5.6)	4.9 (3.9–5.7)	4.8 (3.9–5.6)	0.92	0.97
Proficiency score	4.0 (2.8–5.7)	4.0 (2.8–5.9)	3.9 (2.5–5.9)	0.017	<0.01
Time in notes	6.2 (3.8–9.7)	5.9 (3.8–9.3)	5.6 (3.5–8.9)	0.22	0.03
Turnaround time	4.0 (1.9–8.7)	3.0 (1.3–7.6)	3.2 (1.4–7.3)	0.11	0.01
Time in in-basket	10.3 (5.1–17.5)	7.9 (3.5–15.2)	9.6 (4.9–18.1)	<0.01	0.49
Time outside of 7 a.m.—7 p.m.	19.3 (6.7–39.1)	11.7 (4.7–25.2)	16.2 (6.0–31.9)	<0.01	<0.01
Time on unscheduled days	36.5 (20.3–64.4)	24.5 (13.3–42.5)	31.5 (16.5–55.6)	<0.01	<0.01
Pajama time	28.6 (12.0–60.5)	15.1 (5.8–35.9)	25.0 (7.6–46.8)	<0.01	<0.01

Abbreviation: PEP, Physician Efficiency Profile.

Note: Analysis done using Wilcoxon signed rank test. Median values reported with interquartile ranges.

Fig. 1 Changes in aggregate messages, appointments per day, same day visit closure rate, and time outside 7 a.m. to 7 p.m. across three time periods.

Regarding “efficiency” metrics, same day visit closure rates increased significantly in both postimplementation periods, from 0.7 in August 2019 to 0.9 in May 2020 and 0.8 in July 2020. There was a significant decrease in time in notes (5.6 from 6.2, p = 0.03) and turnaround time (3.2 from 4.0, p = 0.01) in July 2020 compared with August 2019 although not in May 2020. Time in in-basket was initially decreased in May 2020 (7.9 vs. 10.3, p < 0.01) although this difference was no longer significant by July 2020. The PEP score was not significantly different between all three time periods.

Time spent in the EHR outside working hours all significantly decreased during both postimplementation periods compared with August 2019 (p < 0.01).

Outcome Metrics by Gender

Differences between male and female physicians remained consistent across all three periods ([Table 2]). Male physicians had significantly more appointments per day than female physicians (p < 0.04). They also had higher same day visit closure rates (p < 0.01), higher PEP scores (p < 0.01), and spent less time in in-basket (p < 0.01) or in notes (p < 0.01), but had no significant difference in proficiency score (p = 0.19–0.63) or turnaround time (p = 0.07–0.23). Male physicians spent significantly less time in the EHR during pajama time (p < 0.01), unscheduled days (p < 0.01), and outside 7 a.m. to 7 p.m. (p < 0.01).

Table 2 Descriptive statistics of outcome metrics (by gender)

	August 2019
	Male (n = 232)	Female (n = 263)	p-Value
Appointments per day	12.7 (8.7–19.4)	10.6 (7.4–15.8)	<0.01
Aggregate messages	15.1 (10.0–23.5)	14.6 (8.9–22.6)	0.39
Same day visit closure rate	0.8 (0.5–1.0)	0.7 (0.4–0.9)	<0.01
PEP score	5.0 (4.1–5.9)	4.7 (3.9–5.4)	<0.01
Proficiency score	4.0 (2.8–5.7)	3.9 (2.8–5.6)	0.38
Time in notes	5.1 (2.9–7.9)	7.1 (4.8–11.9)	<0.01
Turnaround time	3.6 (1.8–8.8)	4.3 (2.0–8.7)	0.23
Time in in-basket	9.2 (4.4–16.5)	11.0 (5.9–18.1)	0.01
Time outside of 7 a.m.–7 p.m.	13.2 (6.0–30.6)	25.6 (8.2–42.6)	<0.01
Time on unscheduled days	32.0 (17.7–54.5)	42.6 (24.0–71.1)	<0.01
Pajama time	22.4 (10.0–55.3)	35.2 (14.3–63.4)	<0.01
	May 2020
	Male (n = 232)	Female (n = 263)	p-Value
Appointments per day	8.2 (5.7–10.8)	7.2 (5.0–10.4)	0.04
Aggregate messages	9.5 (5.5–18.2)	10.3 (6.1–17.7)	0.16
Same day visit closure rate	0.9 (0.8–1.0)	0.9 (0.6–1.0)	<0.01
PEP score	5.2 (4.4–6.0)	4.7 (3.5–5.5)	<0.01
Proficiency score	3.9 (2.7–5.9)	4.0 (2.8–5.8)	0.63
Time in notes	4.9 (2.9–7.3)	7.0 (4.8–11.5)	<0.01
Turnaround time	2.5 (1.0–8.5)	3.3 (1.6–7.2)	0.07
Time in in-basket	6.1 (2.7–11.8)	9.8 (4.3–17.2)	<0.01
Time outside of 7 a.m.–7 p.m.	10.0 (3.6–20.9)	13.8 (6.0–27.8)	<0.01
Time on unscheduled days	19.6 (12.0–32.9)	31.3 (16.4–47.9)	<0.01
Pajama time	11.6 (4.7–26.9)	19.7 (7.2–40.2)	<0.01
	July 2020
	Male (n = 232)	Female (n = 263)	p-Value
Appointments per day	10.5 (7.3–14.2)	9.0 (6.0–11.9)	<0.01
Aggregate messages	16.0 (10.6–28.5)	16.6 (9.7–26.7)	0.93
Same day visit closure rate	0.9 (0.6–1.0)	0.7 (0.5–0.9)	<0.01
PEP score	5.1 (4.1–6.1)	4.7 (3.8–5.5)	<0.01
Proficiency score	4.0 (2.6–6.0)	3.9 (2.3–5.6)	0.19
Time in notes	4.6 (2.6–7.4)	6.7 (4.4–10.7)	<0.01
Turnaround time	2.7 (1.2–7.5)	3.4 (1.8–7.2)	0.11
Time in in-basket	7.4 (4.2–16.1)	11.3 (6.2–20.0)	<0.01
Time outside of 7 a.m.–7 p.m.	13.2 (4.6–26.5)	20.7 (7.2–36.7)	<0.01
Time on unscheduled days	26.0 (12.8–43.9)	39.3 (20.9–61.1)	<0.01
Pajama time	20.1 (6.8–39.2)	30.2 (8.8–55.9)	<0.01

Abbreviation: PEP, Physician Efficiency Profile.

Note: Analysis done using the Mann–Whitney U test. Median values reported with interquartile values for variables with skewed distributions. Mean values with standard deviations reported for normally distributed variables.

Outcome Metrics by Specialty Category

In August 2019, physicians in surgical specialties had the most median appointments per day (p < 0.01, [Table 3]); however, during May 2020, they had significantly less appointments than primary care or specialty care (p < 0.01). By July 2020, surgery once again had significantly more appointments per day (p < 0.01). Primary care had the most aggregate messages during all three time periods (p < 0.01).

Table 3 Descriptive statistics of outcome metrics (by specialty category)

	August 2019
	Primary care	Specialty care	Surgery	p-Value
Appointments per day	12.2 (8.3–17.0)	10.1 (7.3–15.2)	24.7 (16.9–32.5)	<0.01
Aggregate messages	19.9 (12.7–36.8)	12.3 (8.3–19.3)	14.7 (9.9–19.6)	<0.01
Same day visit closure rate	0.7 (0.4–0.9)	0.8 (0.5–0.9)	0.9 (0.6–1.0)	0.01
PEP score	4.7 (3.8–5.6)	4.8 (4.0–5.6)	5.2 (4.6–5.7)	0.07
Proficiency score	4.3 (3.6–5.9)	3.7 (2.2–5.4)	3.9 (2.2–6.2)	<0.01
Time in notes	7.0 (5.3–11.9)	6.5 (4.0–9.6)	1.5 (0.4–2.9)	<0.01
Turnaround time	4.1 (2.0–8.4)	3.9 (1.8–8.7)	4.0 (2.1–12.8)	0.68
Time in in-basket	17.3 (10.6–26.8)	8.7 (4.7–13.5)	4.5 (2.4–7.6)	<0.01
Time outside of 7 a.m.–7 p.m.	31.8 (14.3–49.5)	13.8 (5.9–32.3)	6.4 (2.6–17.1)	<0.01
Time on unscheduled days	51.6 (30.2–86.0)	34.9 (20.3–58.4)	20.3 (14.3–33.1)	<0.01
Pajama time	52.1 (21.3–88.1)	23.9 (9.2–53.1)	15.5 (7.3–27.4)	<0.01
	May 2020
	Primary care	Specialty care	Surgery	p-Value
Appointments per day	9.1 (5.6–12.7)	7.1 (5.2–9.4)	6.8 (4.7–10.7)	<0.01
Aggregate messages	15.6 (8.6–30.3)	8.9 (5.8–15.0)	5.0 (3.6–8.1)	<0.01
Same day visit closure rate	0.9 (0.7–1.0)	0.9 (0.7–1.0)	0.9 (0.8–1.0)	0.09
PEP score	4.6 (1.86)[a]	4.8 (1.6)	5.3 (0.99)	0.03
Proficiency score	4.3 (3.3–6.0)	3.7 (2.6–5.7)	4.0 (2.5–6.3)	0.10
Time in notes	6.7 (4.8–10.8)	6.1 (4.0–9.3)	1.8 (0.8–4.3)	<0.01
Turnaround time	3.0 (1.5–7.9)	2.8 (1.1–6.7)	3.5 (1.4–15.0)	0.16
Time in in-basket	14.3 (8.6–23.8)	6.1 (3.1–10.8)	2.7 (1.5–4.8)	<0.01
Time outside of 7 a.m.–7 p.m.	15.4 (6.7–32.2)	11.7 (5.0–23.1)	5.1 (2.0–12.7)	<0.01
Time on unscheduled days	29.7 (14.1–51.5)	26.5 (14.5–40.5)	14.1 (8.8–19.7)	<0.01
Pajama time	17.2 (7.1–40.3)	18.0 (6.2–36.3)	5.5 (3.0–11.9)	<0.01
	July 2020
	Primary care	Specialty care	Surgery	p-Value
Appointments per day	9.6 (6.3–12.7)	9.0 (6.5–11.8)	14.9 (11.6–21.3)	<0.01
Aggregate messages	24.2 (13.0–42.9)	14.0 (8.8–21.3)	13.6 (8.2–17.6)	<0.01
Same day visit closure rate	0.8 (0.5–1.0)	0.8 (0.5–1.0)	0.9 (0.7–1.0)	0.05
PEP score	4.7 (3.6–5.5)	4.8 (3.9–5.6)	5.1 (4.7–5.8)	0.02
Proficiency score	4.2 (3.0–6.0)	3.7 (2.3–5.6)	4.6 (2.1–6.2)	0.08
Time in notes	7.2 (4.7–10.6)	5.8 (3.8–8.9)	1.7 (0.7–3.4)	<0.01
Turnaround time	3.7 (1.7–6.8)	3.0 (1.3–7.0)	3.2 (1.3–9.3)	0.32
Time in in-basket	17.7 (9.7–27.9)	7.8 (4.4–14.3)	3.8 (2.2–6.4)	<0.01
Time outside of 7 a.m.–7 p.m.	27.6 (11.6–41.3)	12.8 (4.9–26.5)	6.9 (2.3–18.4)	<0.01
Time on unscheduled days	40.9 (21.4–77.9)	32.7 (15.9–52.5)	17.2 (11.5–23.8)	<0.01
Pajama time	31.6 (11.1–65.8)	26.0 (7.3–45.0)	11.1 (2.1–25.2)	<0.01

Abbreviation: PEP, Physician Efficiency Profile.

^a Median (IQR) given for continuous outcomes that are skewed among specialty groups. Mean (SD) given for those normally distributed.

Note: Analysis done using Kruskal–Wallis or ANOVA tests. Median values reported with interquartile values for variables with skewed distributions. Mean values with standard deviation reported for normally distributed variables.

The PEP score was not significantly different between specialty categories in the preimplementation period (p = 0.07), but primary care physicians had significantly lower PEP scores compared with the other specialties in May 2020 (p = 0.03) and July 2020 (p = 0.02). The same day visit closure rate was significantly different in August 2019 (p = 0.01) and July 2020 (p = 0.05), but not in May 2020 (p = 0.09).

In all three time periods, primary care physicians spent significantly more time in the EHR during unscheduled days (p < 0.01) and outside 7 a.m. to 7 p.m. ([Fig. 2]). They also spent significantly more time in the EHR during pajama time in August 2019 (p < 0.01) and July 2020; however, in May 2020 (p < 0.01), specialty care spent more time during pajama time (p < 0.01).

Fig. 2 Differences in time spent in EHR outside 7 a.m.to 7 p.m. by specialty category and time period. HER, electronic health record.

Outcome Metrics and Years since Graduation

Physicians with more years since graduation had significantly higher same day visit closure rates in August 2019 ([Table 4], r = 0.09; p = 0.05) and more appointments per day in May 2020 (r = 0.14, p < 0.01).

Table 4 Correlation between outcome metrics and years since graduation

	Years since graduation
	r	p-Value
Appointments per day (Aug.)	0.08	0.06
Appointments per day (May)	0.14	<0.01
Same day visit closure rate (Aug.)	0.09	0.05
Same day visit closure rate (May)	0.06	0.19
PEP score (Aug.)	0.07	0.15
PEP score (May)	−0.03	0.44
Time outside of 7 a.m.–7 p.m. (Aug.)	0.06	0.16
Time outside of 7 a.m.–7 p.m. (May)	0.02	0.68

Abbreviation: PEP, Physician Efficiency Profile.

Note: Analysis done using Spearman rank correlation coefficient.

Multivariable Analyses

Specialty category was independently associated with number of appointments across all three time periods (p < 0.01, [Table 5]). After adjusting for gender and years since graduation, surgical physicians had significantly fewer appointments compared with primary care in May 2020 (p < 0.01) and specialty care in both May 2020 (p < 0.01) and July 2020 (p = 0.03). Gender was not independently associated with appointments per day (p = 0.08–0.30). Years since graduation was predictive of this metric in May 2020 (p = 0.04).

Table 5 Multivariate analyses of provider characteristics on select outcome metrics

	May 2020 vs. Aug. 2019		July 2020 vs. Aug. 2019
Appointments per day	Coeff. [95% interval]	p-Value	Coeff. [95% interval]	p-Value
Female	−0.32 [−0.934, 0.29]	0.30	−0.52 [−1.11, 0.07]	0.08
Primary care[a]	5.69 [4.55, 6.84]	<0.01	0.65 [−0.45, 1.74]	0.25
Specialty care[a]	4.01 [2.90, 5.13]	<0.01	1.20 [0.13, 2.26]	0.03
Years since graduation	0.03 [0.001, 0.05]	0.04	−0.002 [−0.03, 0.02]	0.86
Same day visit closure rate	Coeff. [95% interval]	p-Value	Coeff. [95% interval]	p-Value
Female	−0.03 [−0.7, 0.00]	0.05	−0.02 [−0.05, 0.01]	0.22
Primary care[a]	0.01 [−0.04, 0.07]	0.63	−0.00 [−0.06, 0.05]	0.90
Specialty care[a]	−0.03 [−0.09, 0.02]	0.22	−0.05 [−0.11, −0.003]	0.04
Years since graduation	0.00 [−0.001, 0.001]	0.96	0.00 [−0.0004, 0.002]	0.22
PEP score	Coeff. [95% interval]	p-Value	Coeff. [95% interval]	p-Value
Female	−0.50 [−0.71, −0.28]	<0.01	−0.23 [−0.42, −0.03]	0.02
Primary care[a]	−0.10 [−0.46, 0.25]	0.56	−0.27 [−0.59, 0.05]	0.10
Specialty care[a]	−0.20 [−0.54, 013]	0.23	−0.29 [−0.59, 0.01]	0.06
Years since graduation	−0.01 [−0.02, −0.01]	<0.01	−0.01 [−0.02, −0.0001]	0.05
Time outside of 7 a.m.–7 p.m.	Coeff. [95% interval]	p-Value	Coeff. [95% interval]	p-Value
Female	0.67 [−2.21, 3.54]	0.65	−0.13 [−3.50, 3.23]	0.94
Primary care[a]	2.87 [−2.06, 7.81]	0.25	3.81 [−2.12, 9.74]	0.21
Specialty care[a]	2.85 [−1.68, 7.39]	0.22	0.21 [−5.12, 5.54]	0.94
Years since graduation	0.04 [−0.08, 0.15]	0.53	0.06 [−0.07, 0.19]	0.36

^a Compared with surgery specialties.

For same day visit closure rate, specialty care physicians had a significantly lower rate in July 2020 compared with surgical specialties (p = 0.04), but other variables were not independent predictors. Female physicians and physicians with more years since graduation were significantly associated with lower PEP scores in both May 2020 (p < 0.01) and July 2020 (p = 0.02–0.05).

Gender, specialty category, and years since graduation were not independently predictive of time spent outside of 7 a.m. to 7 p.m.

Discussion

We observed that physician EHR usage patterns varied significantly from August 2019 to May 2020 and July 2020. Patient volume decreased significantly from August 2019 to May 2020, consistent with other studies,[1] [16] although we observed an increase in aggregate messages in July 2020. The same day visit closure rate, time in notes, turnaround time, and time in in-basket all demonstrated improvement in May or July 2020. We also found that during these postimplementation periods, physicians were spending less time in the EHR outside of 7 a.m. to 7 p.m. on scheduled patient days than the preimplementation period in 2019. Qualitative studies utilizing survey data have had variable results regarding changes in burnout rates among physicians during the pandemic.[4] [5] [6] [7] [21] While some have found higher rates of burnout in “frontline” specialties hypothesized to be most affected by COVID-19, such as critical care, emergency medicine, and hospital medicine,[6] [7] one study found within the same specialty (oncology) staff working on a COVID ward had lower rates of burnout compared with staff working on a non-COVID usual ward.[22] However, studies did consistently find that exposure to COVID[5] [7] was associated with increased risk for burnout.

Regarding physician characteristics, we observed significant differences in both “efficiency” and “burden” metrics based on gender, years since graduation, and specialty category. Differences between female and male physicians may be due to gender-specific differences in regard to childcare and other familial obligations. Prior studies have shown that physician characteristics such as gender and years since graduation have an overall impact on efficiency[23] [24] and burnout.[6] [25] [26] [27] However, our multivariate analyses found that gender was only a significant factor in PEP score and years since graduation was only an independent predictor in appointments in May 2020 and PEP scores. Neither was an independent predictor for same day visit closure rate or time spent in EHR outside of 7 a.m. to 7 p.m.. Specialty was a significant factor in appointments per day; understandably, as with the cancellation and delay of surgical procedures, surgical specialties saw a larger change in their volume.

Limitations of our study include the timing of the study. With time periods only being a month long and examining only up to July 2020, the pervasive and long-lasting impact of the many systemic, economic, and public health changes that occurred in mid-2020 may not have been fully captured. For example, recent data suggest a significant decrease in telehealth usage by physicians 1 year after the first COVID-19 surge.[28] There was also only a small percentage of physicians from surgical specialties given the decrease in surgical visits and procedures. There are also limitations regarding the metrics. Using vendor-defined metrics may be biased by desires to minimize the degree of burnout associated with EHR use. We should take care when considering their definitions of “efficiency” and normal working hours, which is suggested to be 7 a.m. to 5:30 p.m. or 7 a.m. to 7 p.m. Logistically, Signal incorporates a 5-second time-out rule where the clock stops measuring if no cursor activity is detected after 5 seconds; therefore, the measured time values may not be reflective of actual time. Appointments per day did not take in account whether physicians had full day versus half day clinic sessions, but just whether they were scheduled for clinic appointments. We also did not control for clinical full time equivalent percentages or scheduled patient hours which would have been a more comprehensive value to use in our calculation of time spent in the EHR outside work hours instead of using standardized time periods. Additionally, although time spent working outside normal working hours is known to correlate with burnout, it is clear that there are other contributors to burnout, such as autonomy and lack of staff, which were cited as leading factors during departmental surveys. The gold standard for burnout is quantitative burnout survey data, which have not been directly measured in our study. Due to the exploratory and extensive nature of our analyses, there likely were statistically significant differences that were due to chance alone and additional focused studies are warranted.

In summary, our study has important implications for using EHR-generated reports to monitor trends in physician experience as impacted by key changes and events, as well as how individual characteristics may impact those trends. Health systems are finding it challenging to simultaneously improve productivity and reduce physician burden. Our data demonstrate that easily accessible EHR data can be used to assess productivity and surrogate markers of burden to improve both. Another important aspect of our study is that our approach is easily scalable and less intrusive (as surveys or other user reporting requirements) on already time-constrained physicians, although utilization of existing peri-pandemic departmental burnout surveys would help further validate its use. Incorporation of operational data such as no-show rates and average wait times can help generate an even more accurate model to be correlated with burnout survey data. Repeated studies across different time periods can further help validate this method to quantitatively and continuously assess physician experience with the EHR to identify areas of interest and improvement.

Conclusion

As health systems aim to improve efficiency without increasing burden on physicians, our study demonstrates that utilizing EHR-generated reports may provide a scalable and unintrusive way to monitor trends in physician usage and experience when impacted by a variety of factors, from individual-level characteristics to large-scale systematic changes such as the pandemic and subsequent widespread implementation of telehealth.

Clinical Relevance Statement

This study provides further support that EHR-generated reports may be used as objective measures that vary significantly based on both external factors and provider-level characteristics. Understanding the factors that impact provider experience and usage patterns can help hospital systems monitor the impact of systemic changes and optimize efficiency and provider burnout. Further efforts should be made to understand the correlation of these metrics and how to best utilize them.

Multiple Choice Questions

1. Which of the following has been found to correlate with increased physician burnout?

   • Work environments focused on quality and safety

   • Increased time spent at home on work-related tasks

   • Increased levels of autonomy

   • Male physicians

   Correct Answer: The correct answer is option b. Studies have found that organizational factors such as increased time spent on work-related tasks at home, lack of autonomy, and increased workloads have correlated with increased reported levels of physician burnout. There has also been survey data that found that female providers have increased rates of burnout.

2. Which of the following significantly increased in the postimplementation follow-up period (July 2020) as compared with the preimplementation period?

   • Number of visits per day

   • Time spent in notes

   • Time spent in EHR outside working hours

   • Number of aggregate messages

   Correct Answer: The correct answer is option d. Our results demonstrated that in the follow-up July 2020 period, only aggregate messages had increased to even more than the preimplementation levels, likely related to the delay in care from the initial surge of the pandemic.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed by the Albert Einstein College of Medicine Institutional Review Board.

• References

• 1 Chatterji P, Li Y. Effects of the COVID-19 pandemic on outpatient providers in the United States. Med Care 2021; 59 (01) 58-61
• 2 Watts KL, Abraham N. “Virtually Perfect” for some but perhaps not for all: launching telemedicine in the bronx during the COVID-19 pandemic. J Urol 2020; 204 (05) 903-904
• 3 Beiser M, Lu V, Paul S. et al. Electronic health record usage patterns: assessing telemedicine's impact on the provider experience during the COVID-19 pandemic. Telemed J E Health 2021; 27 (08) 934-938
• 4 Baptista S, Teixeira A, Castro L. et al. Physician burnout in primary care during the COVID-19 pandemic: a cross-sectional study in Portugal. J Prim Care Community Health 2021;12:21501327211008437
• 5 Amanullah S, Ramesh Shankar R. The impact of COVID-19 on physician burnout globally: a review. Healthcare (Basel) 2020; 8 (04) 421
• 6 Linzer M, Stillman M, Brown R. et al; American Medical Association–Hennepin Healthcare System Coping With COVID Investigators. Preliminary report: US physician stress during the early days of the COVID-19 pandemic. Mayo Clin Proc Innov Qual Outcomes 2021; 5 (01) 127-136
• 7 Shanafelt TD, West CP, Sinsky C. et al. Changes in burnout and satisfaction with work-life integration in physicians and the general us working population between 2011 and 2020. Mayo Clin Proc 2022; 97 (03) 491-506
• 8 Nath B, Williams B, Jeffery MM. et al. Trends in electronic health record inbox messaging during the COVID-19 pandemic in an ambulatory practice network in New England. JAMA Netw Open 2021; 4 (10) e2131490
• 9 Moore C, Valenti A, Robinson E, Perkins R. Using log data to measure provider EHR activity at a cancer center during rapid telemedicine deployment. Appl Clin Inform 2021; 12 (03) 629-636
• 10 Jeppson K. What can Epic's signal data tell us about EHR satisfaction and burnout. KLAS Impact Report; 2020. Accessed July 1, 2022 at: https://klasresearch.com/archcollaborative/report/what-can-epics-signal-data-tell-us-about-ehr-satisfaction-and-burnout/324
• 11 Livingston K, Bovi J. Department-focused electronic health record thrive training. JAMIA Open 2022; 5 (02) c025
• 12 Khairat S, Zalla L, Gartland A, Seashore C. Association between proficiency and efficiency in electronic health records among pediatricians at a major academic health system. Front Digit Health 2021; 3: 689646
• 13 Hollister-Meadows L, Richesson RL, De Gagne J, Rawlins N. Association between evidence-based training and clinician proficiency in electronic health record use. J Am Med Inform Assoc 2021; 28 (04) 824-831
• 14 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 15 Eschenroeder HC, Manzione LC, Adler-Milstein J. et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021; 28 (05) 960-966
• 16 Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I³ Population Collaborative. J Grad Med Educ 2017; 9 (04) 479-484
• 17 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 18 Woolson RF. Wilcoxon signed-rank test. In: D'Agostino RB, Sullivan L, Massaro J. eds. Wiley Encyclopedia of Clinical Trials. Hoboken, NJ: Wiley; 2008. . Accessed July 1, 2022 at: https://doi.org/10.1002/9780471462422.eoct979
• 19 Hazra A, Gogtay N. Biostatistics Series Module 3: comparing groups: numerical variables. Indian J Dermatol 2016; 61 (03) 251-260
• 20 Bishara AJ, Hittner JB. Testing the significance of a correlation with nonnormal data: comparison of Pearson, Spearman, transformation, and resampling approaches. Psychol Methods 2012; 17 (03) 399-417
• 21 Koh D, Lim MK, Chia SE. et al. Risk perception and impact of severe acute respiratory syndrome (SARS) on work and personal lives of healthcare workers in Singapore: what can we learn?. Med Care 2005; 43 (07) 676-682
• 22 Wu Y, Wang J, Luo C. et al. A comparison of burnout frequency among oncology physicians and nurses working on the frontline and usual wards during the COVID-19 epidemic in Wuhan, China. J Pain Symptom Manage 2020; 60 (01) e60-e65
• 23 Giusti EM, Pedroli E, D'Aniello GE. et al. The psychological impact of the COVID-19 outbreak on health professionals: a cross-sectional study. Front Psychol 2020; 11: 1684
• 24 Melnick ER, Ong SY, Fong A. et al. Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis. J Am Med Inform Assoc 2021; 28 (07) 1383-1392
• 25 Patel SY, Mehrotra A, Huskamp HA, Uscher-Pines L, Ganguli I, Barnett ML. Trends in outpatient care delivery and telemedicine during the COVID-19 pandemic in the US. JAMA Intern Med 2021; 181 (03) 388-391
• 26 Williams DC, Warren RW, Ebeling M, Andrews AL, Teufel Ii RJ. Physician use of electronic health records: survey study assessing factors associated with provider reported satisfaction and perceived patient impact. JMIR Med Inform 2019; 7 (02) e10949
• 27 McPeek-Hinz E, Boazak M, Sexton JB. et al. Clinician burnout associated with sex, clinician type, work culture, and use of electronic health records. JAMA Netw Open 2021; 4 (04) e215686
• 28 Kaufman M. Telehealth visits dip as New Yorkers return to doctors' offices. Crain's New York Business. May 11, 2021. Accessed July 9, 2021 at: https://www.crainsnewyork.com/health-care/telehealth-visits-dip-new-yorkers-return-doctors-offices

Publication History

Received: 02 March 2022

Accepted: 12 June 2022

Accepted Manuscript online:
15 June 2022

Article published online:
25 August 2022