Keywords
electronic health records - medical education - resident physicians
Background and Significance
Background and Significance
Following the passage of the Health Information Technology for Economic and Clinical
Health Act of 2009[1] and the widespread proliferation of electronic health records (EHRs),[2]
[3] there has been growing concern over the impact of computers, information technology,
and so-called “desktop medicine” on the health, well-being, and efficiency of physicians.[4]
[5]
[6]
[7]
[8] Recent studies have found strong associations between EHR use and burnout,[9]
[10] especially time spent in the EHR after-hours (i.e., “pajama time”).[11] At the same time there has been growing concern over the number of hours worked
by resident physicians during their training, resulting in the American College of
Graduate Medical Education regulating the number of hours residents can work.[12]
[13] While the impact of high weekly hours on quality of care is still debated, it is
undeniable that heavy workloads take a physical, mental, and emotional toll on resident
physicians.[14]
During their time as residents, physicians are developing clinical and socio-technical
skills simultaneously.[15] They must learn not only the practice of medicine and how they fit within their
organizational culture, but also how to navigate and work with the EHR—complex, specialized
software infamous for poor usability and a steep learning curve.[16] Residents are taxed mentally on constant basis with demands on their time and attention
coming from a variety of disparate sources.[17] Coupled with their high workload, concerns over EHR-driven burnout may be especially
salient for resident physicians as they may be learning a new system. However, little
is known about how resident physicians' EHR usage specifically, or how it changes
as they gain experience.[18]
[19]
[20]
[21] Residents may face a large EHR time burden as they attempt to quickly learn multiple
systems at once, or it may be the case that young, technologically savvy residents
are efficient in the EHR and are able to minimize their after-hours time.[22] While recent studies have shown variation across physician specialty,[23] there is little evidence to date about how physician EHR usage varies by experience
or changes over time as they become more facile with the system. Given the high demands
on residents' time and the increasingly strong base of evidence linking EHR usage
to reduced physician job satisfaction and its concomitant burnout, it is critical
to understand how this population of early-careerists interacts with the EHR.
In this study, we used detailed EHR audit log metadata following resident physicians
in ambulatory settings to address three research questions. First, how much time do
resident physicians spend working in the EHR? Second, what proportion of that time
do resident physicians spend working in the EHR after-hours? And third, how does resident
physician EHR usage change as they gain experience?
Data and Methods
Data
We used de-identified EHR audit log data collected from the Lights On Network reporting
system, which tracks and reports user activity in detail from users of Cerner's Millennium
EHR system (Cerner Corporation, North Kansas City, Missouri, United States). Lights
On records user time spent in all clinical and nonclinical activities through the
EHR system, and reports data aggregated at the encounter level. This system records
all interactions with the EHR software including keystrokes, mouse movement, and clicks,
and where the physician is within the system (such as documentation, orders, or chart
review.) The system measures active time using prespecified, proprietary guidelines,
which are detailed in-depth by Overhage and McCallie (2020).[23] All variables in the study come from the Lights On reporting system. The Lights
On reporting system for this study included only ambulatory care EHR use, and excluded
any inpatient or emergency EHR use.
Our study design was a longitudinal, descriptive study with two-way fixed effects
that included Lights On audit log data for 622 resident physicians practicing in ambulatory
care settings at the University of Alabama at Birmingham Health System, a large academic
medical center located in Birmingham, Alabama, for the 12-month period between July
2017 and June 2018. This includes residents across multiple postgraduate year cohorts.
This Lights On data reports only the amount of time spent using the ambulatory care
portion of the EHR, and excludes any inpatient or emergency EHR time if a resident
also spent time in those service areas during the month. We aggregated the data up
to the physician-month level for our unit of analysis. We limited our sample to physician-month
observations with at least five patients seen that month, as not all resident physicians
practice in ambulatory care settings each month, and we wanted to exclude situations
where a resident on inpatient or emergency service charted one or two follow-up encounters
using the ambulatory EHR—this was our only exclusion criterion, and of the 948 residents
at the institution, 622 (65.6%) met the inclusion criteria for our study. Our final
analytic dataset consisted of 3,703 physician-month observations. Due to the de-identified
nature of our data, we were unable to provide detailed demographic information for
our study sample. The overall population of residents at the University of Alabama
at Birmingham Health System is 34.4% female and 65.6% male, 5.8% Hispanic or Latinx,
18.1% Asian, 4.8% Black or African-American, 0.4% American Indian or Alaskan Native,
and 76.6% White.
Our study was designated exempt by the Institutional Review Board of the University
of Alabama at Birmingham Health System.
Measures
We defined three measures of EHR use and physician productivity as our primary outcome
variables of interest. First was time spent in the EHR per patient encounter, which
is the total amount of time a physician spends working in the EHR for a specific patient
encounter, inclusive of all activities the physician performed. Our second measure
was number of patients seen per day, a count of the number of unique patients seen
each day where the physician signed a note in the EHR (resident physicians often sign
notes, but this task may sometimes fall to the attending physician instead, which
may result in our under-counting patient encounters). Our third measure was the percentage
of EHR time spent after-hours. We defined after-hours time as any time spent working
in the EHR, on any activity, between the hours of 6:00 p.m. and 6:00 a.m. These definitions
of EHR time per patient, patient encounters per day, and after-hours time are consistent
with existing studies on EHR audit log metadata, allowing us to make comparisons to
the literature.[23]
Our primary predictor variable of interest was the number of months of ambulatory
practice experience during the 12-month study period. Each month a physician in the
sample had at least five encounters in an ambulatory setting was counted as a month
of experience. Residents in the data were assigned a dummy identifier number so we
could track them longitudinally in a de-identified setting.
Analysis
We first calculated descriptive statistics for our three outcome variables as well
as our sample characteristics. We then plotted the mean of those three outcome variables
by month of ambulatory care experience as line graphs. Finally, we created three ordinary
least squares (OLS) regression models with each of our outcome variables as the dependent
variable in a model, and number of months of ambulatory care experience as our independent
variable of interest. All models were adjusted for physician and calendar month fixed
effects to account for any time-invariant omitted variable bias from either resident
physician characteristics such as age, gender postgraduate year, prior experience
with the EHR, as well as potential seasonality, with robust standard errors clustered
at the physician level. For a robustness test we also included models using cumulative
patients seen during the study period as our independent variable of interest to measure
ambulatory experience. We also estimate our measure of after-hours EHR time using
raw minutes after-hours rather than percentage of time. Finally, we estimate our two-way
fixed effects model on each individual component of EHR use (documentation, chart
review, orders, and other). Full regression results are available in [Appendix Table A1]. All calculations were done using Stata version 16 (StataCorp, College Station,
Texas, United States).
Appendix Table A1
Full regression results
|
Patients seen per day
|
Time per patient in EHR (hours)
|
% EHR time after-hours
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Months of ambulatory experience
|
0.009
|
0.389
|
−0.012 to 0.030
|
−0.7181524
|
<0.001
|
−0.019 to −1.154578
|
−0.2817266
|
0.592
|
−0.242 to 0.138
|
Note: Based on ordinary least squares regressions with individual and month fixed
effects and robust standard errors clustered at the individual physician level. Includes
controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.
Results
In our sample, the mean number of months of physician ambulatory care experience during
the 12-month study period was 4.8. Residents spent an average of 45.6 minutes in the
EHR per patient. Most time was spent on documentation (20.2 minutes), chart review
(13.8), and entering orders (6.5,) with the remaining time distributed between tasks
such as patient discharge, medication reconciliation, and messaging with patients.
Residents spent 13.55% of their EHR time after-hours. The mean number of patients
seen per day was 2.97 ([Table 1]).
Table 1
Sample descriptive statistics of resident physician EHR use
|
Mean
|
Standard deviation
|
Number of months of ambulatory care experience during the sample
|
4.78
|
3.09
|
Electronic health record time
|
EHR time per patient (min)
|
45.89
|
49.86
|
EHR time: documentation (min)
|
20.20
|
26.57
|
EHR time: chart review (min)
|
13.80
|
15.00
|
EHR time: orders (min)
|
6.50
|
6.69
|
EHR time: other functions (min)
|
5.39
|
8.40
|
Number of patients seen per day
|
2.97
|
1.61
|
Percentage of EHR time spent after hours (%)
|
13.55
|
14.90
|
Abbreviation: EHR, electronic health record.
Notes: Based on residents from July 2017 to June 2018.
When evaluating how residents learn through experience, we found EHR time per patient
fell from 48.2 minutes in their first month to 40.9 minutes in their 12th, patients
seen per day increased from 2.69 to 3.39, and percentage of EHR time after-hours increased
from 15.67 to 16.51% ([Fig. 1]).
Fig. 1 Resident physician EHR usage by months of experience. EHR, electronic health record.
In our multivariate OLS regression models, we found that each month of experience
was correlated with a significant reduction in minutes of EHR time per patient (β = −0.72 minutes, p < 0.001). The relationship between experience and patients seen per day and proportion
of after-hours time was not significant ([Table 2]). Our robustness test using cumulative ambulatory encounters as an alternative measure
of experience found similar results to our main specification ([Appendix Table A2]). Similarly, we find no effect of experience on after-hours time as measured by
number of minutes ([Appendix Table A3]). Finally, in our models examining EHR time per patient across the components of
EHR functions, we found each month of experience during the study period was associated
with decreased time spent in documentation (β = −0.46 minutes, p < 0.001) and other functions such as messaging (β = −0.11 minutes, p < 0.001), but found no effect on chart review or orders. ([Appendix Table A4]).
Table 2
Association between months of experience and EHR use in resident physicians
|
Time per Patient in EHR (min)
|
Patients seen per day
|
% EHR time after hours
|
Coef.
|
p-Value
|
Coef.
|
p-Value
|
Coef.
|
p-Value
|
Months of ambulatory experience
|
−0.72
|
<0.001
|
0.01
|
0.389
|
−0.05
|
0.60
|
Abbreviation: EHR, electronic health record.
Notes: Based on ordinary least squares regressions with individual physician and calendar
month fixed effects with robust standard errors clustered at the individual physician
level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018. Full regression results available
in [Appendix Table A1].
Appendix Table A2
Construction of experience as cumulative ambulatory patients seen during study period
|
Patients seen per day
|
Time per patient in EHR (min)
|
% of EHR time after-hours
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Cumulative ambulatory encounters
|
0.0006
|
0.037
|
0.0000–0.0012
|
−0.017
|
<0.001
|
−0.025 to −0.011
|
−0.001
|
0.657
|
−0.007 to 0.004
|
Note: Based on ordinary least squares regressions with individual and month fixed
effects and robust standard errors clustered at the individual physician level. Includes
controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.
Appendix Table A3
Association between resident physician months of experience during the study and after-hours
EHR time per month
|
After-hours EHR time (min)
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Cumulative ambulatory encounters
|
−1.593
|
0.139
|
−3.70 to 0.518
|
Note: Based on ordinary least squares regressions with individual and month fixed
effects and robust standard errors clustered at the individual physician level. Includes
controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.
Appendix Table A4
Association between resident physician months of experience during the study and EHR
time per patient across component functions
|
Independent variable: additional month of ambulatory experience
|
EHR function
|
Coef.
|
p-Value
|
[95% Confidence interval]
|
Documentation
|
−0.46
|
<0.001
|
−0.69 to −0.22
|
Chart review
|
−0.09
|
0.19
|
−0.22 to 0.04
|
Orders
|
−0.05
|
0.08
|
−0.11 to 0.01
|
Other functions
|
−0.11
|
<0.001
|
−0.18 to −0.04
|
Note: All coefficients represent minutes of EHR time per patient. Based on ordinary
least squares regressions with individual and month fixed effects and robust standard
errors clustered at the individual physician level. Includes controls for resident
physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.
Discussion
Resident physicians clearly demonstrated a pattern of increased efficiency in EHR
usage as they gained work experience. There are two major findings with regards to
how resident physicians learn to use the EHR over time and how that translates into
workloads. As might be expected of professionals working at the highest level, residents
improved over a relatively short timeframe. However, the proportion of EHR time that
occurred after-hours did not decrease, suggesting that residents did not seek to minimize
their “pajama time” as they became more facile with the EHR.
Being able to learn, retain, and act on large amounts of information is a sine qua non of being a successful medical student and resident. Therefore, it is not surprising
that this sample was able to significantly streamline the amount of time they used
the EHR to carry out their clinical rotations. Resident physicians often have academic
backgrounds that lend themselves to critical and analytic thinking. However, this
experience may not be useful in mastering EHR technologies, and our results show that
residents spend more time per patient in the EHR compared with previous studies of
physicians more broadly.[23]
[24] EHRs are notorious for poor usability and lack of intuitive navigability.[16]
[25] The effect size of our results provides some context; however, while we find a statistically
significant impact of months of experience on EHR time per patient, the effect size
is not large—a reduction in 0.72 minutes per patient for each month of experience.
Assuming three patients per day, the result is only 2.16 minutes of time savings per
day. However, if that time savings translate to a larger volume of daily encounters,
it could result in a significant quality-of-life improvement. This is a much smaller
effect compared with the differences between physicians practicing in different national
health systems or for attending physicians in surgical versus medical specialties
or pediatric versus adult care.[23]
[26]
[27]
[28]
Our models adjusting for physician and month fixed effects found no impact of experience
on number of patients seen per day. Residents are strongly incentivized to see as
wide a variety of conditions as possible to gain valuable experiences, and the level
of patient volume they handle may not be fully within their control. While our unadjusted
results show an increase in patients seen per day as residents gain experience, it
may be that they are asked to do more by more senior residents or attending physicians,
and EHR time per patient necessarily decreases as their clinical workload increases.
This could in turn reflect a decrease in the quality of EHR documentation that we
were unable to observe in our data.[29]
Finally, our results make it clear that despite residents' increasing competency with
the EHR, the proportion of that time occurring after-hours remained constant. We also
found no effect of additional months of experience on raw after-hours time. Despite
research indicating that this after-hours work is draining and associated with physician
exhaustion and burnout,[11]
[30] resident physicians do not seem to be translating EHR efficiency gains into less
time working at home. This may reflect a reality that after-hours EHR work is viewed
as a necessary and normal part of the practice of medicine, and early-career physicians
are not encouraged to reduce or avoid it. Given increasing rates of physician burnout
and turnover that is at least partially attributable to the burden of EHR “pajama
time,” this is a concerning result. A culture of medicine that necessitates extensive
after-hours EHR time may have a more difficult time in improving clinician wellness
even if EHR usability improves. However, it may simply be that residents are unable
to reduce their after-hours time despite gaining more EHR proficiency. Future research
should explore how resident physicians use the EHR outside of clinic hours in more
depth.
This article has important limitations. First, data were drawn from a single academic
medical center, and we are unable to guarantee that our data generalize to other sites,
though we do not believe our study setting is meaningfully different from other teaching
hospitals. Additionally, the data are from a single EHR vendor employing proprietary
metrics,[31]
[32] such as defining after-hours time as 6:00 p.m. to 6:00 a.m., which may not be appropriate
for all clinical settings. EHR metadata are a powerful tool, but future studies should
also supplement with methods including direct observation and qualitative data collection
to capture a holistic view of physician EHR work. Second, as stated above, we observe
changes in EHR usage but are unable to determine the causal reason for those changes,
and individuals may differ greatly in the effectiveness of their EHR use. Third, we
only have sufficient data to describe EHR usage patterns in ambulatory settings—it
is possible that EHR use and resident physician learning patterns are different in
inpatient settings. Fourth, due to the de-identified nature of our data, we do not
have data on physician specialty or other characteristics. While our fixed effects
models control for time-invariant omitted variable bias such as specialty, age, gender,
etc., we are unable to examine variation across specialties or physician characteristics,
and it may be possible that the relationship between experience providing ambulatory
care and EHR use is moderated by those variables. It is also possible that other factors
besides experience could have resulted in increases in productivity, such as EHR improvements.
However, no significant changes were made to the University of Alabama at Birmingham
Health System Cerner system during the study period, and we observed no sudden changes
in EHR time or productivity during the study period, and calendar month fixed effects
control for secular changes over time in our regression analyses. Fifth, our data
do not allow us to disaggregate the type of work being done after-hours, though only
time working in clinical activities (chart review, orders, and documentation) is counted
in this metric. Additionally, our measure of ambulatory care experience does not capture
other aspects of experience with both working within the health system and with patient
care, and alternative measures such as time employed by the health system that were
not available in our data may be preferable for future studies. Finally, one of our
measures of physician productivity is patients seen per day, which is measured by
the patient encounters where the physician signs the note. Because residents may not
be the physician signing the note, this metric serves as a proxy measure of productivity
and may not capture some patients who may be seen by resident physicians who do not
sign the chart note.[33] Further, patients seen per day in an ambulatory setting may not be a good measure
of resident productivity for a variety of reasons mentioned above, most specifically
that resident physicians may not be in control of their workloads.
Our results have important implications for policy and practice. Educators and regulators
considering resident physician workload requirements should be aware of the burden
of EHR work, a noninsignificant portion of which is done at home outside of clinic
hours. Policymakers seeking to address the issue of physician burnout and turnover
should also be aware of the impact of EHR work on early career physicians, and should
take note that in the current environment physicians do not appear to be minimizing
their after-hours EHR work. Interventions targeted at reducing the burden of EHRs
should therefore specifically target this after-hours work, as broad attempts to improve
EHR usability in general may not be effective.
Conclusion
There is growing concern over the impact of EHRs on physician wellness, with burdensome
documentation requirements coupled with poor usability culminating in a large amount
of time dedicated to “desktop medicine.” Early-career physicians, such as residents,
may be especially vulnerable to this as they are already subject to long hours and
mentally taxed from learning both clinical and socio-technical skills. Our results
show residents become more efficient in the EHR and reduce time spent per patient
as they gain experience, and they increase the number of patients seen per day and
do not reduce the proportion of EHR time spent after-hours. These data should be concerning
to those seeking to improve the residency experience and physician well-being more
broadly.
Clinical Relevance Statement
Clinical Relevance Statement
Resident physicians spend a significant amount of time working in the EHR. As they
gain experience, they spend less time per patient and are able to see more patients,
but the proportion of time they spend after-hours does not decrease. Practitioners
seeking to address physician well-being should be aware of the burden that after-hours
EHR time represents.
Multiple Choice Questions
Multiple Choice Questions
-
Resident physicians spend most of their EHR time in this function:
Correct Answer: The correct answer is option a.
-
Resident physicians show this type of improvement as they gain experience:
-
Spending less time in the EHR after-hours.
-
Spending less time per patient in the EHR.
-
Spending more time per patient in the EHR.
-
Seeing fewer patients per day.
Correct Answer: The correct answer is option b.