Keywords
physician perception - EHR - meaningful use - clinical performance - emergency department
Background and Significance
Background and Significance
To encourage the adoption of electronic health records (EHRs), the Health Information
Technology for Economic and Clinical Health (HITECH) Act offered financial incentives
for the “meaningful use” (MU) of certified EHRs.[1] This was based on the expectation that MU (i.e., comprehensive documentation and
management of patient care) of an EHR can potentially improve the quality of care
received. In response to HITECH, many health care organizations, including their emergency
departments (EDs), incorporated EHR systems into their practices. Using the EHR, the
MU performance metrics, such as door-to-doctor time and admit decision time, measured
in the ED could be automatically collected and used for internal quality monitoring.
Despite these intentions to improve care, a recent study found that providers who
qualify for MU incentives may not necessarily provide higher quality of care.[2] Furthermore, based on a survey of 2,033 physicians conducted by Emani et al, physicians
expressed negative feelings regarding the MU of EHRs, with only 20% indicating that
the MU of the EHR would improve the quality and efficiency of their care activities.[3] This study also found that physician satisfaction regarding the EHRs used was significantly
associated with their perceptions, where more satisfied physicians had more positive
beliefs about MU.[3] Another study assessing the benefits and additional workload resulting from compliance
with the MU program reported beneficial perceptions of MU requirements related to
basic care; however, MU requirements related to more complex care was believed to
be cumbersome.[4] Moreover, perceptions of ease of EHR use associated with MU-related functions has
been found to be high among features that have been more readily adopted, such as
viewing laboratory reports, and low for features less commonly adopted, such as public
health reporting.[5] Such perceptions, or ways in which information and experiences are processed, understood,
or interpreted, can provide guidance for designing health information technology.
Although EHRs have generally had a positive impact on clinical practice,[6] research has highlighted challenges regarding their use, including usability issues,[7] alteration of shift time distribution,[8]
[9]
[10] and challenges of managing the temporal order of workflow activities.[11] Review of recent literature suggests that EHRs have altered clinical workflow, particularly
in the EDs.[12] That is, the series of steps required to complete a clinical activity, such as documentation
of patient information or management of care with other providers, has been altered
as EHRs often do not account for the dynamic nature of clinical activities in the
clinical environment.
The demands on ED clinicians require technology that supports their task efficiency
and accuracy, especially during peak patient loads.[13]
[14] Studies investigating the impact of EHR use on clinical workflow patterns have suggested
that the time spent on documentation has increased over the last decade,[15] with 50 to 65% of time being spent on EHR-related activities.[13]
[16]
[17]
[18] In the ED, measurement of MU performance metrics can provide insights into these
growing documentation requirements. However, it is important to keep in mind that
reduction to metrics, such as door-to-doctor time and admit decision time, was not
the intention of EHR adoption or the MU program.
In this study, we investigate the relationships between physician perceptions of workflow
as a result of EHR use and MU performance metrics. By investigating the role of two
distinct EHRs in environments with different clinical workflows, levels of awareness
of MU performance metrics, and histories of EHR usage, a foundation for evaluation
of system-specific and system-invariant patterns can potentially be identified.
Materials and Methods
Setting and Participants
This study was conducted at the EDs of Icahn School of Medicine at Mount Sinai in
New York, New York, United States (referred to as Site 1) and Mayo Clinic in Phoenix, Arizona, United States (referred to as Site 2). Site 1 is an urban, academic hospital with a 44-bed adult ED unit that serves approximately
100,000 patients annually. Approximately 27% of ED patients are admitted to the hospital.
Site 2 is also an urban hospital with a 24-bed ED (and 9 additional hallway beds)
serving approximately 33,000 patients a year. About 50% of ED patients are admitted
to the hospital.
To deal with high patient volumes, Site 1 utilizes a split-flow model.[19] This model reduces the time patients wait to see care providers, potentially improving
patient satisfaction and decreasing the likelihood that a patient will leave before
being seen by a clinician. Site 2 utilizes a patient assignment tool to streamline
ED workflow; this system uses a rotational assignment tool to distribute patients
across available physicians.[20] Each physician is assigned four patients at the start of their shift; from the fifth
patient onwards, patients are assigned as per the physician's progress with other
patients and availability. Both sites utilized distinct commercial EHRs and provided
EHR training to physicians.
Participants for this study were board certified attending (n = 14) and resident (n = 6) ED physicians. Physicians were recruited through presentations at biweekly faculty
and resident meetings. The institutional review boards of both hospitals and The New
York Academy of Medicine approved this study. Written consents were obtained from
all participants.
Data Collection
In this prospective observational study, conducted over an 8-month period (February–October
2016), we used face-to-face, semistructured interviews as the primary data collection
method. Interview questions (see Appendix A) were developed based on discussions with ED physicians during a preliminary pilot
study, and with physician and nurse consultants from both study sites.
Each interview had four sections with participant demographics, prior EHR experiences,
ED-specific MU performance metrics, and clinical workflow: (1) physician demographics
including information about their role and experience, (2) implementation and EHR
use questions on physicians' previous EHR experience and their perceptions regarding
their current EHR, and (3) physicians' awareness of ED-specific MU performance metrics
and their impact on clinical workflow. This was followed by questions regarding the
physicians' perception about the EHR's influence on four MU performance metrics they
were exposed to at the time of the interview (see [Table 1] for the ED-specific MU metrics[21]), and (4) perceptions about EHR impact on efficiency of clinical workflow, quality
of care, and patient safety were also collected.
Table 1
ED-specific MU metrics[21]
National Quality Forum no.
|
MU metric
|
Definition
|
ED-005–08
|
Door-to-doctor time
|
Median time from patient's first entry to being seen by an ED physician
|
ED-003–08
|
Admit decision time
|
Median time from patient's first entry to hospital admission decision
|
ED-001–08
|
Boarding time
|
Median time from patient's entry to their ED departure to an inpatient floor
|
ED-006–08
|
Walk out rate
|
Patients who leave the ED without being seen
|
Abbreviations: ED, emergency department; MU, meaningful use.
For this part of the interview, physicians were also asked to rate their EHR's impact
on each of the categories using a 1 to 10 Likert scale (1 representing a low impact,
and 10 a high impact). Finally, physicians were asked to describe any other relevant
topics that were deemed important during the conversation.
Face-to-face interviews were conducted by coauthors (C.A.D. and H.C.S.). Each interview
lasted approximately 45 minutes and were audiorecorded and transcribed for further
analysis.
Data Analysis
Using the grounded theory approach for qualitative data analysis,[22] interview transcripts from both sites were evaluated line-by-line to identify emerging
themes. To achieve thematic saturation, our initial codebook-identified themes related
to perceptions of EHR, usability concerns, and the EHR's influence on workflow efficiency,
quality of care, and patient safety were developed. These codes were reanalyzed to
examine relationships between them to develop higher-level thematic categories, including
perceptions regarding EHR, usability issues, EHR's impact on various factors, and
MU awareness.
Attending and resident physician participants were grouped together for this analysis
based on the similarities in their clinical responsibilities and their use of EHRs.
With assistance from clinician collaborators, three researchers (coauthors C.A.D.,
H.C.S., and A.S.) independently coded the transcripts line-by-line using Microsoft
Excel. After initial coding, using a subset of three transcripts (15% of sample),
an interrater agreement was calculated. There was an 84% overlap across the three
coders, showing high degree of agreement. All coding disagreements were then discussed
and resolved, and agreement reached 100%. The remaining transcripts (i.e., n = 17) were then recoded using the updated codebook agreed upon during the resolution
of disagreements.
Descriptive statistical methods were used to obtain frequencies and means for themes
using Microsoft Excel, SPSS, and SAS 9.4.
Results
Twenty physicians (n = 20) were interviewed (Site 1: attending physicians, n = 4, and residents, n = 6; Site 2: attending physicians, n = 10). The mean physician EHR experience at Site 1 was 3.1 years (standard deviation
[SD] = 2.4), and that of Site 2 was 7.3 years (SD = 2.6).
Seven thematic categories (detailed explanation and examples are provided in [Appendix B]) were identified. These included perceptions of EHR system, perceived time spent
on clinical care, EHR usability issues, influence of EHR use on workflow efficiency,
on quality of care and patient safety, as well as on four MU performance metrics.
Appendix B
Coding categories and subcategories for qualitative analysis
Category
|
Scale used
|
Perception about EHR used
|
Positive, neutral, negative
|
Changes in perception
|
Time spent with patients
|
More time, equal time, less time
|
EHR usability
|
Clicks
|
Frequency of response
|
Multiple windows
|
Difficulty maneuvering the system (navigation)
|
Problems with access to required information
|
Repetition of tasks
|
Screen/Display clutter
|
Useful functions
|
Effect of EHR on
|
Efficiency of work
|
Scale from 1 to 10, with 1 as the lowest and 10 as the highest
|
Effective quality of care
|
Safety compared with paper-based systems
|
MU compliance criteria awareness
|
Door to diagnostic evaluation
|
Positive, neutral, negative
|
Admit decision time
|
Time from arrival to ED departure for admitted patients (boarding)
|
Patients leaving the ED without being seen
|
Impact of EHR on
|
Impact of MU requirements on workflow
|
Positive, neutral, negative
|
Abbreviations: ED, emergency department; EHR, electronic health record; MU, meaningful
use.
Perceptions of EHR and Usability Concerns
There were several EHR usability concerns reported, and the most prominent concern
was the “number of clicks” required to complete tasks (51%), which is consistent with
previous research.[7] One physician (#10; Site 1) noted that their system was “very functional, and yet it [has] too many clicks to get to key functions that we
need in the emergency department, so it's robust, but it's not streamlined.” This physician emphasized a lack of heuristics or shortcuts within the system design,
which may lead to increased number of clicks, higher time consumption, and a need
for assistance to complete the task (e.g., asking a colleague to locate a function).
Abundance of clicks was the prominent usability issue, followed by multiple steps
(and windows) to complete tasks. Another physician noted that “…sometimes there's a lot of stuff that I feel is not necessary, but I'm assuming it's
there because it's necessary for someone else, not me” (#3, Site 1). It was also reported that, “I[‘d] love to just be able to open the EKG in chart completion, but we can’t, we have
to go into the chart and then EKG, and then go into the charting function and then
write it and then sign” (#10, Site 1).
Other usability concerns included navigation (18%), access to information (8%), repetition
of tasks (8%), screen clutter (8%), and multiple windows (6%). One physician explained,
“Going in and out of the patient's chart multiple times” can result in “being lost in it [the patient chart] and having to find things over and over again
rather than having them easily found” (#12, Site 2). Furthermore, the impact of EHR use on workflow and cognitive load
was summarized by a physician as: “due to multiple steps in placing an order, [for example], time adds up and requires
more [physician] attention” (#13, Site 2). See [Table 2] for more illustrative examples of usability concerns at each study site.
Table 2
Illustrative examples of usability concerns
|
Site 1 (n = 10)
|
Site 2 (n = 10)
|
Usability concern
|
Responses
|
Responses
|
Clicks
|
…very functional, and yet it [has] too many clicks to get to key functions that we
need in the emergency department, so it's robust, but it's not streamlined”
|
And there's no real shortcuts other than finding it [the information] with the mouse
and then clicking it. It doesn't make any sense to me
|
Accessing information
|
–
|
…sometimes you just can't find what you're looking for and you've got to get help
to figure out
|
Navigation
|
…sometimes [it is] kind of difficult to find those [patient] orders and you have to
click a couple of different tabs
|
Going in and out of the patient's chart multiple times can result in being lost in
it [the patient chart] and having to find things over and over again rather than having
them easily found
|
Repetitive tasks
|
–
|
When searching [patient notes, the system ends up] repopulating every single time
|
Multiple windows
|
I'd love to just be able to open the EKG in chart completion, but we can't, we have
to go into the chart and then EKG, and then go into the charting function and then
write it and then sign
|
Due to multiple steps in placing an order, time adds up and requires more attention
|
Screen clutter
|
…sometimes there's like a lot of stuff that I feel like is not necessary, but I'm
assuming it's there because it's necessary to someone else, not me
|
The [EHR] templates look like “an order from a restaurant” - abbreviated information
without any flow
|
Abbreviations: EHR, electronic health record; EKG, electrocardiography.
With these usability concerns in mind, physicians reported on a 10-point scale that
their EHRs supported patient safety (9), followed by quality of care (7.3), and, finally,
efficiency of task completion (6.7).
Influence of EHR Usability on Meaningful Use Metrics
Half of the physicians (50%) acknowledged that they were familiar with the ED MU metrics.
When asked about the EHR's role in improving MU metrics, physicians reported a neutral-to-positive
influence on their performance, with a perceived relationship between EHR use and
both door-to-doctor time and in admit decision time (see [Fig. 1]).
Fig. 1 Physician perceptions of electronic health record impact on meaningful use (MU) performance
metrics.
A physician (#9; Site 1) explained that “the process of patient input in the department has probably improved.” However, three additional factors were also identified as improving workflow, including
timeliness in the access of information, the respective workflows of the ED, and patient
load. Physicians reported that the established workflow influenced MU metrics more
than the EHR used. For example, one physician (#14; Site 2) identified the patient
assignment tool's role in improving MU metrics, explaining that the “Patient assignment system [has] decreased LOS [length of stay] as patient ownership
is defined from the start…EHR 2 refinements may or may not help with this.” From this physician's perspective, the algorithm with which patients are assigned
to clinicians is believed to aid in reducing length of stay with improvements to the
EHR not necessarily having a positive influence on reducing this metric. [Table 3] provides some illustrative examples of physician responses.
Table 3
Illustrative examples of EHR's influence on MU metrics
Influence
|
Site 1 (n = 10)
|
Site 2 (n = 10)
|
|
Participant responses
|
Participant responses
|
Door-to-doctor time
|
|
Positive
|
[in text]
|
We have the ability to look up their medical records before we even actually see the
patient. I think that it helps in that regard
|
Neutral
|
I don't think it is changing the time people wait
|
There is influence and decreased time, but because of the patient assignment system,
not the EHR
|
Negative
|
(The EHR) is probably still extending some of our metrics because it takes so much
time rather than helping us
|
If I'm taking more time to document on patients, I'm not seeing the other patients
that come in a timely manner
|
Admit decision time
|
|
Positive
|
Being able to see their (the patient's) whole record and know what's going on has
definitely made decision making go much more quickly
|
Ordering what I feel is necessary, earlier in the process will decrease door to decision
time quite a bit
|
Neutral
|
I don't think the system matters
|
[EHR 2] does not have much impact, but the workflow and getting laboratories and reports
back does
|
Negative
|
–
|
Patient registration time to decision time is worst because [EHR 2] slows down most
workflows
|
Boarding time
|
|
Positive
|
I think the EMR does help with boarding… I no longer take care of them. I talk to
the inpatient floor upstairs, and that care has been transferred, and the inpatient
team is able to actually manage them through the EMR
|
[EHR 2's] a helpful tool once you know how to use it. It does help me decrease the
time the patient would spend in the ED
|
Neutral
|
[in text]
|
[in text]
|
Negative
|
–
|
–
|
Walk out rate
|
|
Positive
|
I think the EHR has probably had some impact on that as well because from the moment
the patient registers, we are able to track their location and progress through the
department a lot more efficiently that we did before
|
I guess it's better than if it was on paper
|
Neutral
|
I'm not sure if you can say that (EHR 1) itself is influencing walk out rate, but
the fact that it is publicly appointed as part of Meaningful Use made us focus on
it
|
I think that's actually probably the same
|
Negative
|
–
|
I think it slows things down (…) because I'm documenting through an EHR and patients
are not getting back as quickly and I'm less efficient
|
Abbreviations: ED, emergency department; EHR, electronic health record; EMR, electronic
medical record; MU, meaningful use.
EHR Use and Time Spent with Patients
Site 1 physicians reported that they spent the same amount of time for direct patient
care activities, utilizing their EHR system to perform indirect patient care, as compared
with their previous experiences, including with paper-based systems. On average, 89%
of the physicians reported that they spent considerably less time with patients, followed
by 13% who spent about equal time in direct patient care and performing indirect patient
care activities using the EHR system.
Discussion
Based on the semistructured interviews with physicians, EHR usability concerns appeared
to be disruptors of physician perception of EHRs, potentially affecting their workflow.
Although both sites used distinct EHR systems, the need for an abundance of clicks
to perform routine tasks was a key issue at both locations. These perceived issues
can potentially lead to more effort and time required to complete patient care tasks,
and reduced direct patient care time. As suggested in recent literature, changes in
the distribution of shift time[8]
[9]
[10] as well as challenges managing the temporal order of workflow activities[11] can decrease levels of user satisfaction and lead to physician burnout.[23] With increased documentation requirements as a result of MU, a change in the physician–patient
relationship has also likely decreased both physician and patient satisfaction.[24] It is also possible that decreased time at the bedside could lead to increased errors
or decreased quality, thus compromising patient safety.
We found that EHR use potentially influenced the efficiency of physicians' workflow,
the perceived quality of care they provided, and, consequently, the safety of patients.
Perception of the EHR's influence can be, in general, traced to the physician perspectives
regarding their EHR: for example, negative perceptions about the EHR were often related
to EHR usability concerns. Furthermore, physician perspectives are indicative of their
beliefs about the quality of the system used and the role of the EHR in supporting
efficiency in the ED.
MU performance metrics are a measure of efficiency, whereas EHR use and workflow elements,
such as split-flow organization and the patient assignment tool, are a means to improve
care delivery. Reduction of MU performance metrics, including door-to-doctor time
and admit decision time, were not intended goals of EHR adoption or the MU program.
However, physicians who had an understanding of these performance metrics, perceived
their EHR use to have been helpful in improving overall ED performance. As is suggested
in the literature, more commonly adopted EHR features, which also may be better understood
by the physician, are viewed in a positive light for clinical practice.[5] These features may also be linked to basic care, contributing to the ease of use
experienced by the physician.[4]
[5]
The complex demands of the ED environment require technology that supports clinician
task efficiency and accuracy. Hence, the need of more standardized and user-centered
designs that can accommodate for physician needs also aligning with the organization's
workflow to provide care with ease. As a part of a larger research program, this study
situates exploration of the MU program and the EHR's influence on clinical workflow
through investigations using EHR data logs,[25] sensor-based technology,[26] ethnographic observations, and data visualizations.[27]
Conclusion
Physicians perceived that the use of EHRs improved clinical workflow, especially activities
related to reducing door-to-doctor time and admit decision time. These perceptions
of the quality of the EHRs used relates to the extent to which they feel the system
changes the way they complete their daily tasks. EHRs should decrease ED throughput
time, increase patient satisfaction, and decrease ED crowding. However, usability
concerns continue to be a contributor to negative perceptions of EHR use, overshadowing
any problems related to MU performance metrics. As efficiency of workflow increases
with the implementation of health information technologies and workflow organizations,
we have to make sure that effectiveness and safety are not compromised.
Limitations
Attending and resident physician participants were grouped together for this analysis
based on the similarities in their clinical responsibilities and their use of EHRs
as our focus was on the general understanding of MU performance metrics. Although
trained researchers collected data at each of the two study sites, it was relatively
challenging to be completely consistent in data collection. We achieved what we consider
thematic saturation; however, additional variance could have been minimized with a
larger sample.
Clinical Relevance Statement
Clinical Relevance Statement
Given the interrelation between perception and cognition, a physicians' perception
of EHRs is likely to influence how they think about their practices. Negative perceptions
of EHR usability problems may override positive aspects of EHR use, including the
influence on MU performance metrics, and contribute to decreased care satisfaction
and physician burnout.
Multiple Choice Questions
Multiple Choice Questions
-
Although EHRs have had positive impacts on clinical practice, research has also highlighted
how EHRs have altered clinical workflow. EHR-related alterations of clinical workflow
can best be described as changes to the:
-
Number of times a task is completed each day.
-
Level of physician and/or patient satisfaction.
-
Location where tasks are completed.
-
Sequential order in which tasks are executed.
Correct Answer: The correct answer is option d. Since clinical workflow is characterized by the series
of required tasks to complete an activity, alterations to the order in which tasks
are completed may impact other tasks (e.g., cause delays). Implementation of EHRs
most dramatically influenced clinicians' documentation and review of patient information.
Carayon and colleagues[11] described the shift between pre-and post EHR implementation, where greater rates
of team communication and order management prefaced clinical documentation and review
by attending physicians than during observations of the paper-based system.
-
Quantitative studies investigating the impact of EHR use on clinical workflow patterns
have suggested that about 50 to 65% of shift time is spent on EHR-related activities.
Qualitative explorations (such as interviews) of workflow can situate these trends
in real context. According to the authors, which of the following best indicates how
physician perceptions inform future design of health information technologies? Perceptions
provide insight about the:
-
EHR's role in supporting efficiency.
-
EHR's role in team interactions.
-
Variations in EHR-related activities.
-
Variations in the quality of the EHR used.
Correct Answer: The correct answer is option a. EHR usability concerns appear to be vital disruptors
of physician perception of EHR, ultimately affecting the workflow. Hence, the need
of more standardized and user-centered designs that can accommodate for physician
needs and align with the organization's workflow to provide efficient care with ease.
As the literature suggests, more commonly adopted EHR features, which also may be
better understood by the physician, are viewed in a positive light for clinical practice.
These features may also be linked to basic care, contributing to the ease of use experienced
by the physician. Tailored training based on an organization's workflow may reduce
the impact usability concerns have on EHR use.
Semistructured Interview Questions: Role of EHR and Meaningful Use Requirements in
Clinical Workflow in the ER
The primary goal of this interview is to understand the nature of your work within
the ED with respect to the EHR.
-
Tell us a little about yourself (when you first started in this ED or elsewhere).
Introduction and Use of EHR
-
Perception: How long have you been using the current EHR? How do you like it? Have
your perceptions changed over time? If they have changed then does this have anything
to do with their familiarity with the system over time? Had it changed the way they
do their daily task, in terms of their workflow (e.g., do they spend more time in
the computer versus with patients)?
-
If they have problems, ask for examples.
-
If they like the current EHR, then also ask for examples.
-
Are there any functions of the EHR that you find problematic?
-
If yes, please explain what barriers you have experienced.
-
If no, what do you like about it?
-
EHRs are supposed to increase efficiency, effectiveness and safety. On a scale of
1 to 10 with 1 being the lowest and 10 being the highest, where would you rate the
system you use on:
-
Efficiency of getting work done in the ED.
-
Effective quality of care in delivering quality care for your patients.
-
Improvement in patient safety in comparison to the paper-based system used earlier.
MU Performance Metrics
5. As you know, we have several metrics to assess what is called “MU” in the ED and
you are required to comply with them. Did you know about them? They are supposed to
make some of the clinical processes more efficient.
6. In your opinion, do you think the use of EHR in the ED influenced any of these
processes I am going to talk about: (You can probe further based on what they say.
If they say yes, then ask how? If no, then ask to elaborate if processed could influence
their work pattern).
-
Door to Doctor Time: Median time from patient's first entry into the ED to being seen
by a physician.
-
Admit Decision Time: Median time from patient's first entry to the decision to admit
from the ED.
-
Boarding Time: Median time for patient's arrival in the ED to their departure after
being admitted (few clinicians provide responses on this metric).
-
Walk Out Rate: Patients who go through registration, but leave before being seen by
a physician.
7. Do any of these indicators alter your work pattern in any way that you are aware
of?
Quality and Safety
8. I'm going to read a statement to you: “One of the most important aspects of using
health information technology such as EHRs, is to provide not only efficient and effective
patient care, but also safe care. To what extent does this resonate with you personally
in your ED practice? (Ask them to elaborate, as needed).