Keywords
problem-oriented medical records - facilitators and barriers - electronic health records
and systems - secondary use - adoption - qualitative
Background and Significance
Background and Significance
One of the core elements in electronic health record (EHR) systems is the problem
list, which is a list of active and inactive problems relevant to the current care
of the patient.[1] Most often, these problems are coded based on an underlying terminology. A problem
list is central in the problem-oriented medical record (POMR), which is defined as
“a structured organization of patient information per presented medical problem.”[1] POMR has proven to be successful in providing a structure that helps physicians
record their notes about patients, which helps them to get a good understanding of
the medical history of the patient.[2]
[3]
[4] Accurate problem lists that provide a concise summary of patient problems can therefore
help physicians to track a patient's status and progress, and organize clinical reasoning
and documentation.[3]
[5]
[6]
Although physicians and patients acknowledge the value of an accurate problem list,[7]
[8] problem lists often remain incomplete, due to lack of responsibility, incomplete
and inaccurate content of the underlying terminology, and because of little technical
and administrative support.[5]
[9]
[10] Subsequently, physicians often modify these coded problems to their preferences,
by adding details for these problems in clinical notes.[11] For instance, if the problem code is glaucoma but the modified free text is suspected glaucoma, the problem list indicates by its code that the patient has glaucoma, despite the
patient not being confirmed to have that diagnosis. These alterations might inhibit
patient care, as they can change the meaning of the code. Consequently, structured
data from problem lists cannot be reused reliably.[12]
[13]
[14]
[15]
To better understand acceptance and use of technologies in health care, such as EHRs,
researchers have frequently studied facilitators and barriers for health care professionals.[16]
[17]
[18]
[19]
[20] To our knowledge, little attention has been given to factors associated with acceptance
and use of problem lists.[5]
[21] Therefore, it is still unclear to what extent health care professionals are adopting
and accepting the problem list for clinical tasks, and potentially for reuse purposes.
According to the unified theory of acceptance and use of technology (UTAUT), a validated
framework that is often applied in health care context,[16]
[22]
[23]
[24] the use of a new technology is influenced by performance expectancy, effort expectancy,
social influence, and facilitating conditions.[16]
[25]
[26] Investigating factors influencing acceptance and actual problem list use using constructs
of UTAUT could help determine what barriers need to be conquered and what facilitators
need to be implemented to increase problem list use.
Objectives
The purpose of this study is to explore factors influencing the acceptance and actual
use of the problem list. Removing existing barriers and implementing facilitators
can help optimize problem list use. That, in turn, will eventually increase the quality
of clinical data, enabling (re)use.
Methods
Study Design
In the Netherlands, health care providers make use of the diagnosis thesaurus (DT)
to select the best-fitting problem code for their patients. The DT is an interface
terminology SNOMED CT used in EHRs of a growing number of Dutch hospitals.[27] We recruited two groups of participants from Amsterdam UMC, that both use the same
EHR system (Epic) since 2016:
-
(1) Physicians and physicians in training who use the EHR system daily.
-
(2) Leaders who play a role in sharing organization's culture and strategic vision,
including heads of medical departments and quality managers.
The participants were contacted via email and snowball sampling was applied. The participants
were asked to read a participation sheet and sign an informed consent form.
Data Collection
Face-to-face or telephone semistructured interviews were conducted with participants
between October and December 2019. The interview guide started with collecting demographics,
including gender, age, and work experience in the field and with the EHR system. Interview
questions were based on UTAUT constructs and are shown in Appendix A. We also included questions related to discrete or structured registration in general,
to better understand what factors cause participants to refrain from coding or modify
diagnosis code descriptions. Answers of the participants were summarized during the
interviews to ensure validation of the answers. Similarity of answers on the same
questions by different participants also strengthens the validity.[28] Interviews were held until no new information was provided, at which point we determined
theoretical saturation had been achieved.
Analysis
E.K. transcribed and coded interviews according to the grounded theory open coding
approach. This is a qualitative research method used to generate a theory from gathered
data.[29] This process started with open coding to create oversight of collected data, then
categorizing the codes into clusters of codes to relate codes to UTAUT constructs
(i.e., axial coding). For example, answers related to EHR training are first coded
as “training” and then reorganized to the UTAUT construct “facilitating conditions.”
The codes were assigned using the Atlas.ti 8.4.4 software. If multiple comments described
similar constructs, one example comment was selected to represent the opinions of
the participants within the UTAUT construct. This approach is comparable to that of
a study of BenMessaoud et al.[30] Coded comments were discussed with other authors (R.C. and N.F.K.). Based on consensus,
each coded comment finally related to at least one UTAUT construct. Next, we aimed
to explain these comments in terms of facilitators and barriers.
In our application of UTAUT, performance expectancy is the degree to which an individual
believes that using the problem list will help him or her to attain gains in their
clinical tasks, also compared with the previous methods (e.g., free text). Effort
expectancy is the degree of effort, such as time and usability, associated with problem
list use. Social influence is the degree to which physicians perceive that peers and
key figures, such as colleagues and heads of medical departments, believe he or she
should use the problem list.[25] The presence of facilitating conditions can be defined as the organizational and
technical infrastructure that exists to support problem list use.[31]
Results
In total, 75 physicians were invited. After 24 (32%) physicians participated in the
interviews, we reached saturation and held no further interviews. Ten interviews were
conducted via telephone, and 14 interviews were conducted face-to-face. The median
duration of an interview was 30.5 (15–70) minutes. The characteristics of the participants
are shown in [Table 1].
Table 1
Demographics of the interview participants (n = 24)
|
Demographic
|
|
N (%)
|
|
Occupation
|
Physician
|
14 (58)
|
|
Physician in training
|
2 (8)
|
|
Head of medical department
|
4 (17)
|
|
Physician and data quality manager
|
4 (17)
|
|
Gender
|
Male
|
18 (75)
|
|
Female
|
6 (25)
|
|
Working years (physician)[a]
|
< 5 y
|
2 (8)
|
|
5 < 10 y
|
5 (21)
|
|
10 < 15 y
|
11 (46)
|
|
> 15 y
|
6 (25)
|
|
Working years with Epic
|
Since 2015
|
15 (63)
|
|
Since 2016
|
9 (37)
|
|
Age group
|
20–30
|
1 (4)
|
|
30–40
|
5 (21)
|
|
40–50
|
11 (46)
|
|
50–60
|
4 (17)
|
|
60–70
|
3 (12)
|
|
Specialty
|
Nonsurgical (including number of participants): anesthesiology (2), ear, nose, throat
(ENT) (3), internal medicine (hematology (1), infection diseases (1) oncology (1)),
neurology (1), obstetrics and gynecology (2), orthopedics (1), pediatrics (6), urology
(1)
|
19 (79)
|
|
Surgical (including number of participants):
dental surgery (1), neurosurgery (1), plastic surgery (1), intensive care (2)
|
5 (21)
|
a The physicians in training were placed in the subgroup < 5 years.
Interview Outcomes
The interview topics were organized using the UTAUT model ([Fig. 1]). Example comments for each construct are included in [Table 2]. The construct Image was omitted, because this was not addressed in any comments. Perceived ease of use and ease of use were combined, because the definitions of these constructs are similar.[25]
Fig. 1 Unified theory of acceptance and use of technology (UTAUT) model adapted from the
UTAUT model of Venkatesh et al.[25]
Table 2
UTAUT constructs with corresponding quotes of the physicians
|
Subthemes
|
Pros
|
Cons
|
|
Performance expectancy
|
|
|
Perceived usefulness
|
P1: “It is just a short time investment, but once it works, your preparation for clinic
is faster, you always have a complete overview, you miss nothing, your letters are
ready in no time (...) you no longer have to search through your old notes. It is
just complete.”
|
C1: “I cannot imagine the profit from the problem list. So I don't need it because
I'm not aware of the potential benefits.”
C2: “Especially because the problem list is not kept up to date, it is not something
I fully trust.”
|
|
Outcome expectancy
|
P2: “For me it is a handy tool to gather the information in 1 place, so it helps me
if a patient of mine is admitted to just call the physician and say, look, the information
is right there.”
|
C3: “In principle the intentions are there, but they (physicians) do not see the added
value of it yet.”
|
|
Relative advantage
|
P3: “If you are going to record things in a discrete manner, that is just not as fast
as writing on a piece of paper with a half-illegible handwriting (...) there must
eventually be some sort of realization that it may take more time at the front gate,
but that it ultimately saves time in the end.”
|
C4: “I think that if you just think and write good notes, then that is actually clear
(...) those automatic click lists cause all kinds of noise.”
C5: “I don't use it if I want to know how my patient is doing. Then I read the notes.
I also think those letters are super ugly. Those automatic letters with hardly any
proper language and also a lot of things that are not useful for you as a physician.”
|
|
Extrinsic motivation
|
P4: “I think everyone is too busy (...) it just has to do with motivation and a kind
of obligation that you feel that you should have your patients records adequately
documented.”
P5: “We cannot get the complications from the central registration, and that is why
we think that this should be done via the problem list.”
P6: “That is of course the only reason why you want to fill it in a uniform manner,
so that you can reproduce it for whatever purpose.”
|
C6: “Everyone knows that you have to use that problem list, and in fact you use it,
but as limited as possible, because it is an obligation. Because you cannot create
a financial code otherwise.”
C7:“You only feel this necessity if you want to do something with the data yourself.”
C8:“It has no added value for me. The diseases that I find interesting, I put them
in a patient list, and I will find those back again very quickly.”
C9:“The core task of the EHR system should be that most professionals can easily see
the status. It should not be the goal to be able to send bills or have some scientific
registration system set up.”
|
|
Job-fit
|
P7: “It appears in my status and it is also quickly generated from the problem list
in my letter.”
|
C10: “The needs of the subspecialties are not the same (…) and as soon as patients
have multiple problems, it is very complicated to get an immediate overview.”
|
|
Effort expectancy
|
|
|
(Perceived) Ease of use
|
P8: “I register it because you can easily create financial codes from the problem
list.”
|
C11:“A lot of terms cannot be found. (…) And suppose it is in there, then you just
have to type just exact the right thing to find that. So you will almost have to know
what is exactly in there, if you want to find it.”
C12: “The terminology of the problems is not something we recognize. That makes it
very difficult to find.”
C13: “These terms do not cover the problem. (...) so I change the description.”
C14: “What we do is always adjust the free text. Therefore it may be that within the
free text it is nicely registered and that it works well for our status, while it
is a completely different problem for the underlying ICD code.”
C15: “It is a phase you are in (...) I write ‘analysis’ in the display field and I
do that consciously because patients get to see this information in their patient
record, in (the application) MyChart.”
|
|
Complexity
|
|
C16: “Someone comes in with a set of symptoms which could fit to four different terms
in the terminology.”
C17: “It is quite complex. It is also very comprehensive and it is very easy to add
something (to the problem list). And for many people it is difficult to throw something
out.”
C18: “I think it is not covering (the diseases), it is not complete, incorrectly described,
and very difficult in use that it takes me too much time. So I just don't use it.”
|
|
Social influence
|
|
|
Subjective norm
|
P9: “Head of medical departments must be asked to ensure that their department propagates
the right policy. I think that is step 1 (...) and then the following step is that
we talk to each other that we are not doing it correctly.”
P10: “They (physicians) know I (head of medical department) am constantly insisting
(to use it). And then it gradually gets better.”
|
C19: “There is currently not much supervision or control.”
C20: “I (head of medical department) can play a role in telling them (physicians)
it is important, but the product must be good then.”
|
|
Social factors
|
P11: “If you can show (to your colleagues) something like ‘this is the problem list
and it is useful, because’ then you can convince people and they will use it.”
P12: “It is a matter of repeating, repeating, repeating, repeating and motivating
and lecturing each other. That is no fun. But it eventually helps.”
P13: “Not everybody has to be happy and interested, you just need one interested (physician)
per specialty and from different layers in hierarchy.”
P14: “Of course you can do it with negative feedback, but in my experience that won't
work. I think you would convince people if you take them with you in your way of thinking.”
P15: “It would not be a solution for us to show a graph every week showing who had
the least problems filled in on the problem list. It is mostly positive learning methods.
You do well if you register it, and you remind each other now and then.”
|
C21: “Physicians can get away with it by just putting a problem on the problem list
that they thought it was at the beginning, while that might not be the case anymore.”
|
|
Facilitating conditions
|
|
|
Compatibility
|
|
C22: “There are two competing systems, the medical history and problem list.”
|
|
Perceived behavioral control
|
P16: “You need to agree upon what you want, measure that, and improve based on that.”
P17: “I think there should be a point of contact if you want to add something, because
it doesn't exist (…).”
|
C23: “The big objection is that the problem list is not updated. And that is also
because no individual physician is responsible for the problem list. (...) I am responsible
for my own notes, so I ensure that they are up-to-date and correct. And I think that
is a central problem with the problem list. There is not one owner and everyone can
mess with it.”
C24: “I do not know if that (a point of contact) is possible or to who I should go
to if I want to add something.”
|
|
Facilitating conditions
|
P18: “I think it would be good if you had some sort of body in the hospital that says:
‘this is the way we want to work with the problem list, and these are the rules and
that is what we stick to.”
P19: “We have a manual for all physician assistants.”
P20: “We have work agreements, and with that we try to limit the number of problems
to two or three, because otherwise you lose the overview.”
P21: “Rules do not lead to better behavior, it is more about internal motivation.”
P22: “We see the recorded problems with patient transfer. So, if something is not
listed, then we will adjust it.”
P23: “If you want to improve problem list use, you have to sit down with the individuals
and do it. But something like the e-mail that was recently sent out, which is completely
free of obligation (...). I think that just doesn't work.”
P24: “There must be a certain profit to be seen. Someone must explain next to us how
we use the problem list and also ensure that we can use the problem list but also
that it helps us. There must be something in it for us.”
P25: “I think training is a form of imposing sanctions (…) I would want to prevent
that everyone at the department has to be (re)trained”
P26: “My suggestion would be to put someone there who can do that (fill in the problem
list), and who will do that for the entire hospital. Then the physicians are also
satisfied.”
|
C25: “With the introduction of Epic, I feel there has been no supervision and training
on how to use the problem list.”
C26: “None of my colleagues, including myself, know what the rules are. It has not
been established.”
C27: “Epic is a system that says that everything is possible, meaning that everyone
will start doing it their own way.”
C28: “When the EHR was introduced, nobody knew what the rules were. So, we actually
had to think for ourselves a little about how we were going to work with the EHRs.
And that particularly applies to the problem list.”
C29: “There are other priorities. That service center is overloaded. They are just
busy with other things.”
|
Abbreviations: EHR, electronic health record; UTAUT, unified theory of acceptance
and use of technology.
Note: “It” refers to the problem list.
Barriers and Facilitators
This section discusses barriers and facilitators that were derived from quotes displayed
in [Table 2]. Examples of barriers have been described under “cons” and examples of facilitators
have been described under “pros.” The quotes are indicated with Cn (cons) and Pn (pros).
Barriers
Performance Expectancy
Physicians reported barriers toward the value of the problem list for doing their
job. First, most physicians were unaware of the added value of problem list compared
with their current registration methods, for example, free-text notes (C1, C3, C4,
C5). Second, physicians reported they do not trust problem list completeness as terms
do not get updated once the diagnosis has been confirmed, for example, by diagnostic
tests (C2). Some physicians mentioned that they only use the problem list to select
the correct reimbursement code (C6) (Dutch: Diagnose Behandeling Combinatie). Furthermore,
not all physicians agreed that reuse of data was a potential motivation to problem
list use. Some physicians mentioned having other, similar, methods to collect data
for research purposes, for example, Excel files. Others reported that the primary
goal of health care is to treat the patient, and that reuse of data is therefore no
incentive to problem list use (C7–C9). Moreover, specialties have different demands
on what should be documented on the problem list (C10). For example, dermatologists
would be interested in all dermatology-related problems, while that would be irrelevant
for cardiologists.
Effort Expectancy
Professionals agreed that the content and presentation of the problem list in the
EHR system itself showed room for improvement to reduce documentation efforts. First,
the functionality to query the underlying terminologies to choose best-fitting terms
was found ineffective (C11). Searching only succeeded if physicians typed in exact
terms. Second, the DT does not include terms that are used in practice (C12). Consequently,
physicians who use the problem list always choose a similar concept, but change the
description (C13). This modified description is displayed to the physician in the
problem list of a patient record. However, few physicians considered this dangerous
for health care while, for example, decision support is triggered by coded problems
and not by modified descriptions (C14). In addition, physicians reported the need
to modify the terms because the diagnosis is not always immediately known from the
beginning and patients have insights in their data too,[32] which thus requires to describe uncertainty about the diagnosis (C15). Physicians
considered the DT too extensive to unequivocally determine the most applicable term
(C16). Furthermore, physicians find it difficult to delete irrelevant or inactivate
passing problems and they find it too time-consuming to code problems on the problem
list (C17, C18).
Social Influence
Several physicians reported the lack of organizational policy including enforcement
mechanisms and the lack of supervision as a barrier for proper implementation of the
problem list (C19). According to a head of department, this is the result of the unawareness
of the relative advantage. Promoting the problem list among colleagues therefore remains
a low priority (C20). As a result, physicians avoid updating the problem list after
they have fulfilled the minimal requirement to put one diagnosis on the problem list
(C21).
Facilitating Conditions
Physicians reported that the EHR system contains a variety of similar registration
options, such as the medical history (C22). Nonetheless, most professionals agreed
that even if the content of the problem list is complete, they would still also describe
clinical information in notes. They reported that clinical information cannot fully
be captured in a predefined set of terms (C23).
Physicians reported uncertainty about responsibility of keeping the problem list up-to-date,
and only a few physicians were aware of the possibility to request new terms to further
complete the DT (C24).
None of the physicians had received specific guidance or training on effective problem
list utilization (C25). Physicians reported that they do not have the knowledge to
use the problem list or know what information is appropriate to document on the problem
list (C26). Consequently, physicians created workaround processes, which resulted
in proliferation of various methods to document findings (C27, C28). In addition,
some physicians reported that the support of the EHR team is limited (C29).
Facilitators
Performance and Effort Expectancy
Some physicians reported that using the problem list supports preparing clinical tasks
by having a complete patient overview (P1–P4). It also has a potential in reusing
data (P6). For instance, for the generation of discharge and referral letters (P7).
Furthermore, it facilitates the capture of complications and of financial codes (P5,
P8).
Social Influence
Physicians reported that heads of medical departments or supervising physicians should
propagate the policy (P9, P10). However, motivation is increased if the benefits relevant
to care are shown, and physicians are reminded repeatedly (P11, P12). To ensure benefits
are shared, physicians reported the need for “champions” within specialties (P13).
Champions are motivated colleagues that have an intrinsic interest and commitment
to implement change.[33]
Physicians also reported they needed to receive positive feedback instead of negative
feedback from their supervisors or colleagues, and reminders on how and why to use
the problem list (P14, P15).
Facilitating Conditions
Some physicians reported that problem list use should not only be motivated through
positive feedback, but also monitored to indicate which specialties perform better
than others (P16). Furthermore, physicians need a point of contact to allow them to
request for new terms in the terminology (P17).
Other reported facilitating conditions were written and enforced policies and guidelines,
which might help professionals in keeping the problem list up-to-date (P18–P20). However,
some physicians disagreed with such central policies, and reported that problem list
use should be mainly stimulated through intrinsic motivation (P21). Some physicians
reported that, within their specialty, supervising staff regularly checks whether
the problem list is up-to-date. These check moments were: during multidisciplinary
consultations, after daily clinic consultations, and whenever a patient needs to be
transferred to another specialty (P22).
Additionally, physicians reported that the best way to promote problem list use is
to show-and-tell in small training sessions, that is, peer-to-peer training by “champions”
(P23). These sessions should focus on benefits relevant to care. For instance, one
physician reported that trainers should exemplify how problem list data can be reused
within specialties (P24). Furthermore, physicians could be retrained during their
career. To determine who should be retrained, problem list use could be monitored.
One head of department specifically mentioned that heads would be motivated to stimulate
physicians to properly use the problem list, if that means the physicians do not need
to be retrained (P25).
Last, several physicians mentioned the need for administrative support, such as scribes,
who maintain the problem list on behalf of the physician (P26).
Discussion
From our study, it became clear that there was a wide variation in attitudes toward
problem list use, as most constructs of UTAUT contained mixed positive and negative
comments. The physicians to whom the relative advantage was known, felt that up-to-date
and accurate problem lists can increase efficiency of consultations, because it supports
preparing clinical tasks by having a complete patient overview. Furthermore, the problem
list has a potential value in reusing clinical data. Contrary, to some physicians
the relative advantage was too low compared with free text because they do not recognize
the benefits or, if they recognize the benefits, they have not experienced the advantage
in their daily practice. Consequently, the use and maintenance of problem lists did
not get priority.
Physicians that recognized the relative advantage also acknowledged some barriers
reported by physicians that did not recognize the relative advantage, namely the lack
of accurate content of the DT and inefficient search functionality of the terms within
the EHR system, leading to modifying diagnosis code descriptions. Lacking content
and poor presentation has also been reported by previous studies that examined factors
influencing problem list use.[5]
[7]
[8]
[10]
[21]
[34]
[35] These studies also support our findings with regard to the lack of policies and
guidelines[5]
[35] and lack of training.[5]
[21]
[34] In our study, we further identified how training can be improved: by involving “champions”
in small peer groups. However, in addition to improved training it is important that
usability issues are solved,[35]
[36]
[37] because it will be difficult to train physicians to use the problem list appropriately
if the problem list is insufficient.[36]
[38] Furthermore, the time available to keep the problem list up-to-date and to promote
problem list use is often a restriction,[5] although this might depend on prioritization of maintaining the problem list, which
we believe will increase once benefits are shared.
Finally, it is remarkable that only a few physicians recognized the risk of modifying
diagnosis code descriptions. One conceivable explanation for this unawareness could
be that the physicians use workarounds and other collection methods, for example,
Excel, for reuse purposes and therefore do not experience the need to keep underlying
diagnosis codes up-to-date.
Recommendations
In [Table 3], we present recommendations to help establish wider agreement on problem list use,
thus reducing inaccuracies and making problem list information accessible for reuse,
including the need to solve usability issues and the need to increase priority through
benefit sharing. During interviews, we noted disagreement on whether problem list
use should be monitored or whether it should only be motivated through peer feedback.
Without monitoring, physicians could fall back to old habits and use the problem list
incorrectly.[1] Additionally, monitoring could help identifying which specialties might need extra
support and training. We also consider the implementation of regular checks a possibility
to ensure that the problem list is kept up-to-date. Some participants suggested medical
scribes, who help maintaining the problem list. However, literature states that to
balance the increased expenses, additional revenue from increased patient visits is
needed for scribes to be financially prudent.[39]
Table 3
Recommendations to increase problem list use
|
Recommendations
|
|
Organizational level, i.e., policy creators, EHR-trainers and hospital board members
|
|
|
Alignment with workflow
|
Create guidelines and policies explaining who is responsible for the maintenance of
the problem list and how to use it; share the guidelines with (supervising) staff;
introduce the guidelines in small training sessions per specialty
|
|
Training and support
|
Initial EHR system training should include problem list aspects and should focus on
the advantages of problem list use in the care process; implement retraining programs;
offer ongoing onsite support; implement new (super-user) training programs by training
tech-savvy and enthusiastic physicians, the “champions,” so they can share their knowledge
with their colleagues
|
|
Monitoring and feedback
|
Monitor problem list use on a specialty level to discover misuse of problem list and
act on it by inviting head of medical departments and/or physicians for retraining
programs
|
|
Increase motivation
|
Motivate physicians and increase prioritization for heads of medical departments through
knowledge and benefit sharing in small training sessions; host meetings per specialty
and give examples of how data can be reused, relevant for those specialties
|
|
Individual level, i.e., physicians and head of medical departments
|
|
|
Peer feedback
|
Monitor problem list use within the specialty to discover misuse of the problem list.
Heads of medical departments and supervisors should notify their colleagues in case
of lack of problem list use.
|
|
Knowledge sharing and demonstration
|
Motivate colleagues through knowledge sharing and demonstration of benefits that save
time
|
|
Stimulate problem list use
|
Implement daily moments to update the problem list, for instance during patient meetings,
on transfer of patients to another specialty and at the end of daily clinic
|
|
System level, i.e., EHR system vendors and terminology holders
|
|
|
Terminology
|
Improve the content of the terminology together with physicians; facilitate a change
proposal process for improving the content of the terminology and keep user up-to-date
whenever new terms are added to the terminology
|
|
EHR functionality
|
Include search functionality of the terminology in EHR systems
|
|
Availability similar methods
|
Reduce the availability of similar registration methods in EHR systems (e.g., medical
history)
|
Abbreviation: EHR, electronic health record.
In parallel with this study, Amsterdam UMC established a committee that aims to improve
data quality in the EHRs to reduce administrative burden and to improve problem list
use. We discussed our recommendations with committee member and all were considered
relevant for future implementation. In fact, some recommendations were already considered
by this committee, such as setting up a key performance indicator to measure if problem
list use increases by excluding the medical history functionality in the EHR system
and the implementation of new training programs for physicians in training who can
share the knowledge with their colleagues. Also, each specialty is required to assign
a “point of contact” in 2020, who is responsible for teaching others how to work with
the EHR system and problem list. This is in agreement with our recommendation of finding
“champions” who should communicate knowledge and benefits of problem list use. Future
research will focus on whether implementation of current methods from the committee
combined with our recommendations indeed improves problem list use.
Strengths and Limitations
Although this research provided some meaningful findings, it has its limitations.
First, although a qualitative approach was considered the best method to achieve our
objectives, such approach has drawbacks. For example, answers of the participants
may have been socially desirable or prone to recollection bias. Furthermore, we did
not present the transcripts to the participants for comments (i.e., member-checking).[40] However, we are confident that we collected accurate results as we used the validated
UTAUT framework, which is commonly used to determine factors influencing acceptance
of (new) technologies in medical context[16]
[22]
[25]
[26] we received similar answers among the participants about problem list use and we
reached saturation. Additionally, literature[5]
[7]
[8]
[10]
[21]
[34] states factors influencing problem list use that correspond to our outcomes. Therefore,
although we examined one organization that uses one software system, other hospitals
that use the same EHR system or the same approach to populate the problem list might
still benefit from the presented results and recommendations.
Conclusion
This study was unique in the application of the UTAUT model to examine factors that
affect acceptance and problem list usage by physicians. The results of this study
will help health care institutions and EHR system and terminology providers to understand
barriers and facilitators to acceptance and use of problem lists. With this understanding,
strategic plans and training programs can be developed to persuade physicians to employ
problem list use in their routine practice. The results show that some physicians
were attracted to the benefits, but to convince all physicians to use the problem
list, usability issues should be solved and facilitators are required. Facilitators
include assigning “champions” who can convince colleagues to use the problem list
through positive experience sharing in small peer groups and top-down control from
supervisors or other directing health care providers, who need to prioritize and follow
the redesigned policies and guidelines. Future research should focus on evaluating
implementation of the proposed recommendations.
Clinical Relevance Statement
Clinical Relevance Statement
Only when data on problem lists are accurate, one can profit from potential EHR benefits
such as automatic generation of discharge and referral letters and reuse of clinical
data for decision support. To establish an accurate problem list, health care management
should redesign training programs, policy, and guidelines, aiming at optimization
of problem list use. Furthermore, EHR and terminology developers should ensure that
presentation of the problem list is efficient and the content of the underlying terminology
is complete.
Multiple Choice Questions
Multiple Choice Questions
-
Which of the statements about problem lists is true, based on this study?
-
I. Problem lists often remain incomplete, due to lack of benefits.
-
II. Problem lists often remain incomplete, due to lacking presentation in EHR systems
and incomplete content.
-
I is true, II is false.
-
I is true, II is true.
-
I is false, II is false.
-
I is false, II is true.
Correct Answer: The correct answer is option d. Benefits of problem list usage are clear, but not
(yet) recognized by physicians.
-
Physicians acknowledge problem list benefits, but what are the best methods to support
these benefits?
-
Through peer-to-peer training and explicit and formal policies describing problem
list use.
-
Through peer-to-peer training and by trusting the physician to decide which problems
belong on the problem list.
-
Through plenary training sessions and by trusting the physician to decide which problems
belong on the problem list.
-
Through plenary training sessions and explicit and formal policies describing problem
list use.
Correct Answer: The correct answer is option a. Peer-to-peer training will stimulate benefit and
knowledge sharing, thus potentially increase prioritization of problem list use. Policies
and guidelines are needed to guide physicians in problem list use and to establish
who is responsible for maintaining the problem list. It is important to trust physicians
to decide which problems belong on the problem list, but, as shown in this study,
without formal policies problem list use is poorly supported.
Semistructured interview questions
General Introduction
(Name, research, interview format, consent and anonymization, audio recording, any
questions before start)
Demographics
(Age (category), gender, work experience in the field as physician, work experience
with EHR system Epic)
Performance/Effort Expectancy
Performance/Effort Expectancy
Social Influence
Facilitating Conditions
-
Usage problem list / structured registration in general
-
○ Why (not)?
-
○ What organizational factors (would) make it (im)possible to use it?
-
○ Any other issues related to (the lack of) problem list use?
-
○ What do your fellow colleagues think?
-
○ Availability of (re-)training with regards to structured registration / problem
list use?
-
Presence of policy with regards to motivation/identifying incorrect modifications;
if not, what should be the design of these policies?
-
Support from others
-
○ Do you know whom to go to?
-
○ Are there super-users?
-
○ How do ‘others’ help?
-
○ Are you satisfied with current support, if not; what should be arranged / changed?
-
Do you plan time to use the problem list / record your findings in Epic?
-
What factors (would) make it (im)possible to allow you to plan time to use the problem
list/ perform structured registration in general?
-
Other barriers or (lack of) facilitating conditions related to the use of the problem
list / structured registration in general?
