The Solution
Vanderbilt Clinical Informatics Center's Approach
Vanderbilt used a self-developed electronic health record (EHR) for several decades,[5] and processes for getting access to the system, getting access to data, and adding
interventions were widely understood. After the transition to Epic in 2017, VUMC personnel
were less certain how to engage with the new systems; pathways for accessing systems
and data were not always documented or formalized. We created VCLIC as a community
of clinical informatics practitioners, researchers, and learners. VCLIC's overarching
goal is to make it easy for VUMC personnel to engage with the EHR, including accessing
data, testing interventions, and evaluating outcomes. Our center manages Vanderbilt's
Physician Builder and distributed data access programs, coordinates clinical informatics
events, and offers various training programs. The training activities are the focus
of this manuscript.
Each year, VCLIC chooses a theme around which to focus its educational and service
activities. The theme is determined through a solicitation process via our center's
Advisory Board and aligned with emerging topics in clinical informatics alongside
VUMC's institutional priorities. VCLIC has had four themes, one for each academic
year since our center's launch: “Making Health Care More Equitable through Technology;”
“Innovative Care Delivery Models for Mental Health;” “Re-envisioning the EHR to Reduce
Burnout and Improve Wellbeing;” and “Health System Efficiency through Clinical Informatics.”
In addition to the theme, after about a year of piloting various events, workshops,
and seminars, we conducted an “educational needs assessment” to help us tailor our
programming further, especially as it related to intended audience ([Supplementary Appendix S1], available in online version). We interviewed eight individuals across the organization
using a convenience sample, expanding it with a snowball method until we reached saturation.
The individuals included 2 HealthIT leaders, one of whom also had a clinical appointment,
1 DBMI faculty member, 3 DBMI students, 1 DBMI trainee/alumnus, and 1 MS-ACI alumnus.
We asked individuals what topics they or their peers would like to learn about; what
effective learning/training experiences they've had; and, if they had been to some
of our events/trainings, what had gone well and what hadn't. This process yielded
a diverse array of information, and the synthesis was a bit open ended, as consensus
on how best to teach or reach the varied audiences across VUMC was low. The biggest
takeaway from the process was that we should try to target our events for different
and specific audiences instead of attempting to reach everyone (students, trainees,
clinicians, IT workers, etc.) all in the same session. In practice, this has proven
difficult, as we advertise our events broadly and don't turn away any potential attendees.
We are currently working to develop new member management strategies that will allow
us to better target offerings to specific groups of people.
Activities
Workshops
We have hosted a series of workshops since Fall of 2021 with topics ranging from where
to find and how to use data resources at VUMC ([Supplementary Appendix S2], available in online version and [Supplementary Appendix S3], available in online version), introduction to Epic for nonclinicians ([Supplementary Appendix], available in online version), FHIR app development ([Supplementary Appendix S5], available in online version), introduction to Tableau ([Supplementary Appendix S6], available in online version), using R for more involved data analysis, building
BestPractice Advisories in Epic ([Supplementary Appendix S4], available in online version), and human-centered design principles ([Supplementary Appendix S7], available in online version). We planned and executed the sessions with the help
of volunteer center members with expertise in each subject area. For each workshop,
the designated instructor met with the center director and project manager 2 to 3
times in the 2 months prior to the session to plan the material and practice the delivery.
When possible, we developed hands-on materials (e.g., worksheets, exercises using
our Epic playground or other systems) to be used during the workshops.
The first year, the training sessions were 3 hours long, but in response to feedback
from attendees that it was difficult to block that amount of time, and from instructors
that it was difficult to plan and host the workshops, we condensed them to 90 minutes
in subsequent years. In 2021 and 2022, we held a mixture of virtual and hybrid events
using Zoom, as some staff were working remotely. In 2023, we transitioned the events
to being fully in person, as we believed that would be the optimal learning environment
for the material we were teaching. Although our attendance dropped precipitously (from
a high of about 140 people in attendance on the Zoom to about 25 in person), we felt
we were reaching a more engaged audience with in-person, hands-on learning events
(during the remote trainings, few students would log into the activities or ask questions).
In fact, after moving back to in-person events, engagement, participation, and follow-ups
increased, even though raw attendance numbers fell.
Most events had a mix of DBMI faculty, staff, and students; HealthIT personnel; and
clinicians from across clinical departments at the medical center. Following each
event, we sent out short feedback surveys to assess session-specific details, as well
as overall satisfaction and ways we could improve for the future.
Vanderbilt Clinical Informatics Practical Short
We typically offered our workshops once a year, and individuals often sought to learn
the topics at other intervals, for example, if they were new to Vanderbilt, had a
new project that required a new skill, had a conflict with the offering of the training,
or recently became interested in a topic. We initially handled these ad hoc by, in
many cases by meeting with the individual or sending written materials. To better
meet these needs, we developed an asynchronous offering: VCLIPS, which are bite-size
instructional videos on practical clinical informatics topics from expert VCLIC faculty
and staff. They include many of the topics we previously covered in workshops, but
instead of long, synchronous sessions, they are broken down into multiple 3- to 5-minute
videos posted on our website. We distributed announcements about VCLIPS via email.
To date we have had three “releases” of these video series, one each in Spring of
2023, Fall of 2023, and Spring of 2024. A complete list of video topics is shown in
[Table 1].
Table 1
Currently Released Vanderbilt Clinical Informatics Center
|
• Overview of VCLIC and clinical informatics at VUMC
• Intro to Epic and build capabilities
• Data resources
• Human-centered design principles and processes
• Evaluation and dissemination of HealthIT interventions
• Exempt IRB research
• Address geocoding
• Using Epic to support clinical research
• Signal data
• Agile methods for leading a clinical or research informatics team
• Medication data tools
• Patient-reported outcome measures
|
Abbreviations: IRB, institutional review board; VCLIC, Vanderbilt Clinical Informatics
Center; VUMC, Vanderbilt University Medical Center.
To produce the videos, we provided the instructors with as many sessions as necessary
with the center director and project manager to develop an outline, create a hands-on
activity, and break the material into short videos ([Supplementary Appendix S8], available in online version). We also provided screen capture and other video recording
software where needed, although most used software they were already familiar with
and had access to. Once the videos were recorded, posted the videos and written materials
to the web.[6]
[Fig. 1] shows the number of views of VCLIPS over time.
Fig. 1 Vanderbilt Clinical Informatics Center views, April 2023 to April 2024.
Special Events
Beyond the workshops and VCLIPS, our center has held several additional events to
develop and engage our community's understanding of clinical informatics. In Spring
2021, we held a “Design Challenge”: The VCLIC Techquity Design Challenge was an 8-week
program that invited teams of students, clinicians, faculty, and staff from different
scientific disciplines and of various backgrounds to propose novel solutions to problems
related to health care equity and technology. Teams worked with each other and assigned
mentors from our center to develop and record a prototype of their solution. The videos
were judged by leaders from our health system and presented in our informatics seminar.
This event engaged 45 participants from a range of disciplines and departments across
the medical center; participating groups brought passion and tenacity as they sought
to make VUMC a more equitable system and place. Although our focus was on design,
many of the projects eventually ended up being implemented at VUMC after the end of
the challenge ([Supplementary Appendix S9] available in online version).
In Spring of 2022, we held a hackathon in Vanderbilt's maker space, the Wond'ry. The
Wond'ry is located at the crossroads of Vanderbilt's undergraduate and medical campuses,
benefitting a goal of our hackathon to engage participants from the medical center
and university. The event was based on our annual theme “technology to improve mental
health care delivery.” Participants included VU undergraduate students, VU medical
students, other medical trainees, graduate students, and staff in the medical center.
Over the course of about 30 hours spanning from early Friday afternoon to late Saturday
afternoon, three teams worked together, assisted by VCLIC mentors, to develop prototypes.
Teams presented wireframes of the planned applications to the audience and three senior
judges from across the medical center and university. The judges deliberated on a
set of criteria we developed to determine a winner ([Supplementary Appendix S10], available in online version).
In Fall of 2022 we held our inaugural InformaticCon event, resulting in our most successful
effort at community engagement. InformaticCon is a local clinical informatics conference,
where we solicit submissions from VUMC and VU faculty, staff, students, and trainees.
After an informal review process, we selected speaker and poster presenters from the
submissions. The presenters were provided with guidance on how to develop their presentations,
whether oral or poster format, as not all of them had prior experience with academic
presentation styles. The VCLIC team organized the on-campus event, providing boards
for the poster session and audiovisual equipment for the speakers. The first year
we hosted an hour of talks (8 seven-minute lighting talks in 1 hour) and then an hour-long
poster session.[7] After the success of the first year, we held a second InformaticCon in Fall 2023,
extending the event to 3 hours, with an hour-long poster session, a 45-minute panel
discussion, “Getting Involved in Clinical informatics at Vanderbilt,” during lunch,
and concluded with the hour of lightning talks.[8] We had approximately 75 attendees the first year and 120 attendees the second year
([Supplementary Appendix S11], available in online version).
Speaker Series
In addition to our annual InformaticCon event, we invite speakers from all over the
country throughout the year. DBMI hosts a weekly biomedical informatics speaker series,
and as part of this series, we partner with DBMI to feature a clinical informatics-focused
speaker approximately once a month. When possible, we also facilitate meals with the
speaker, and meetings between the speaker and students, faculty, and staff in our
center and DBMI. Where possible, we try to select speakers that will be broadly interesting
and relevant to our annual theme.
Clinical Informatics Graduate Course
In the Fall of 2022, we launched a new graduate-level course in clinical informatics
([Supplementary Appendix S12], available in online version). The course covers key topics in clinical informatics,
including an overview of inpatient and outpatient EHR use, clinical decision support,
terminologies, data resources, and HL7 and FHIR standards. To supplement the lectures,
we invite experts on topics like nursing informatics, pharmacy informatics, leadership
in clinical informatics, imaging informatics, and quality and safety related to informatics.
As part of the course, we also organize field trips to various locations at VUMC,
including an ambulatory clinic, intensive care unit, clinical lab, pharmacy, and data
center. These visits allow students to witness the practical application of clinical
informatics systems in real-world settings. Scheduled during the regular 90-minute
class, the trips are coordinated with a knowledgeable host, often a VCLIC member,
who provides a tour and answers student questions. Students are encouraged to reflect
on the types of patients served (if any), the roles of the staff, and how technology
is utilized in each area.
Medical Student Integrated Science Course & Graduate Medical Education Elective
Advanced medical students at Vanderbilt are able to take a range of Integrated Science
Course (ISC) courses. The Clinical Informatics ISC is a 4-week elective offered twice-yearly
to third- and fourth-year medical students.[9] The course was developed in 2020 and first piloted in May 2021 by VCLIC members
who are also senior operational and educational leaders. The course is delivered through
learning modalities analogous to what learners would experience during a traditional
clinical rotation, as shown in [Table 2]. In this course, learners treat the health care system as their patient and look
for opportunities to employ informatics tools to address the organization's needs.
Informatics Clinic offers students an observational experience to learn from clinical informaticists
during their daily practice. Informatics History and Physical requires students to apply the same tools used when caring for a patient to a challenge
identified during workflow observation in various clinical settings. They then present
their work during a weekly H&P Review presentation attended by course directors, leaders from the clinical area of observation,
and other students. Some of the student presentations are chosen for implementation.
During Interactive Learning Activities, learners put into practice new skills while working through real-world cases with
oversight by practicing clinical informaticists. Each week, students synthesize and
summarize what they have learned from one aspect of the course in a required journal
activity. Weekly projects are also assigned to get students hands-on experience learning
to use a particular informatics tool or resource.
Table 2
Analogous concepts for clinical education and treating the health care system as the
patient
|
Clinical setting
|
Clinical informatics
|
|
Patient
|
Health care system
|
|
Clinic
|
Informatics “clinic”
|
|
Taking a history
|
Informatics H&P
|
|
Morning report
|
Interactive learning activities
|
|
Presenting on rounds
|
H&P review
|
|
Notes
|
Journals
|
|
Procedures
|
Projects
|
Abbreviation: H&P, History and Physical.
The VUMC Clinical Informatics Elective for Graduate Medical Education (GME) grew organically
from individual learner requests to do “rotations” in the DBMI.[9] These experiences ranged from research requests to broad survey-styled rotations.
To help standardize the clinical informatics experience, we formed a common entry
point to allow learners to apply for a rotation and to ensure support from the learner's
primary program director. This rotation was officially launched with support from
the Office of GME in 2019.[9]
Initially, the GME rotation focused on the 1-on-1 pairing of GME learners with clinical
informatics faculty. The rotation provided an outline for both the learner and faculty
member and ensured core exposure to fundamental areas of informatics. Based on feedback
from incorporating GME learners into the evolving undergraduate medical education
(UME) ISC course for undergraduate medical students, the GME rotation began to leverage
the core content developed for medical students. Specifically, residents and fellows
appreciated the didactic and group activities created for the medical student curriculum.
As of 2022, the GME rotation is largely unified with the UME course, and learners
are together through all sessions.[9] GME learners have the option of either a 2- or 4-week rotation. These blocks must
align with the course offering for the UME learners.
Clinical Informatics Fellowship
The VUMC Fellowship Program in Clinical Informatics is a 2-year, ACGME-accredited
fellowship with both didactic and rotational training experience. All fellows are
strongly encouraged to also continue clinical practice in their primary specialty.
VUMC Clinical Informatics fellows also complete the VUMC Master of Science in Applied
Clinical Informatics (MS-ACI) for foundational knowledge to support the daily practice
component of the fellowship rotation experiences. VUMC Clinical Informatics Fellowship
graduates are well suited for various roles after graduation. Graduates of this program
have accepted roles in operational leadership, research, and industry. After completing
the program, clinical informatics fellows are eligible to sit for and prepared to
pass the board certification examination for the subspecialty in clinical informatics.
Master of Science in Applied Clinical Informatics Program
In 2016, VUMC launched the MS-ACI, with the first graduating class in 2018. The MS-ACI
program is a fully online program, which offers comprehensive informatics training
for working health professionals. Over 21 months, students complete 36 credits of
clinical informatics coursework, gaining practical insights and hands-on experience.
The program emphasizes real-world application through a mentored capstone project,
and the digital format of the MS-ACI program accommodates distance learning for busy
health care professionals. The curriculum is based on AMIA's core competencies and
foundational domain,[10] and the AMIA delineation of practice.[11] The curriculum prepares physician participants for Board Certification in the Subspecialty
of Clinical Informatics offered by the ABPM and the ABP. Nonphysician participants
are prepared for the AMIA Health Informatics Certification offered by the AMIA Health
Informatics Certification Commission. So far, 36 students have completed the MS-ACI
degree program,[12] several of whom are now in Chief Medical Informatics or Information Officer positions
across the globe.
Support for Epic Certification and Training
VUMC has the largest Epic Physician Builder program, per capita, with 91 builders
across 22 departments. The program was founded by an anesthesiologist and clinical
informatician who is a VCLIC member and is coordinated by our center. While most members
are physicians, our program also includes nurses, pharmacists, and PhD informaticians,
and thus is scoped more broadly than Epic's official Physician Builder training track.
Participants complete EHR build training offered by Epic and, after completing a certification,
are given access to build in our EHR. Crucially, clinical departments are required
to give participants protected nonclinical time to participate in the program, and
our Physician Builders both initiate projects and complete the build themselves, often
supporting efforts led by our HealthIT department. Builders at VUMC have broad access
to build in many different functional areas of Epic and are paired with an “analyst
buddy” in our HealthIT department. In addition to coordinating the program, our center
also gives an annual award to outstanding Physician Builders.
In parallel with the Physician Builder program, VCLIC also initiated and operates
the Vanderbilt Database Access Working Group (VDAWGs) program, which is a pathway
for obtaining access to VUMC clinical data warehouses, including Epic Clarity and
Caboodle. The program currently has 22 participants. These participants complete Epic
training and are given access to the relevant warehouses with approval from their
department chair, institutional review board review, and regular audits. Participants
are asked to predeclare the scope of their work, and audits ensure that participants
are accessing data appropriate for their score. As needs evolve, participants are
able to update their scope.
Clinical Informatics Knowledge Base
Our center developed a wiki-style knowledge repository that houses how-to articles
on a variety of topics, including how to obtain Epic training (with a focus on Physician
Builders and Clarity Database access), how to get access to VUMC's data resources,
and how to evaluate and disseminate clinical informatics projects. Anyone at VUMC
can search for, read, create, and edit articles in the knowledge base, but the majority
have been written by center members. The knowledge base also hosts our VCLIPS videos
and associated text as well as our Clickbusters program,[13] a step-by-step process for improving CDS at VUMC.
Analysis
We surveyed participants on their satisfaction (and regarding how they would improve
the experience) following 20 VCLIC events and aggregated the responses, as shown below.
[Table 3] shows feedback from our center's events. The first eight workshop events we held
were either fully virtual or hybrid, and thus garnered a larger audience (39–137 attendees).
We then switched to fully in-person events, which resulted in lower attendance (16–41
participants) but greater engagement. The FHIR Workshop in March 2023 was an all-day
event and advertised more broadly. InformaticCon conferences garnered greater attendance.
Some of our one-off events (the Design Challenge and the Hackathon) were smaller scale.
On average, around 20% of attendees provided feedback on our events. For most events,
we asked specific feedback questions, but we always included one question asking attendees
to rate satisfaction on a scale of 1 to 5 (5 being the highest). The majority of attendees
were satisfied with our events, with the average satisfaction score being 4.63/5,
and all events averaging a satisfaction score of greater than 4. The response rate
for our evaluation surveys was fairly low, which may limit the generalizability of
our findings. In general, we found the specific comments to be the most valuable part
of the surveys.
Table 3
Number of attendees, number of feedback responses, and satisfaction scores from attendees
of Vanderbilt Clinical Informatics Center events between 2021 and 2023
|
Date
|
Event
|
Approx. no. attendees
|
No. of feedback responses
|
% feedback responses
|
Satisfaction
|
|
February 2021
|
Disparities data lab
|
137
|
30
|
21.9
|
4.57
|
|
March 2021
|
FHIR app building workshop
|
126
|
23
|
18.3
|
4.26
|
|
Spring 2021
|
Design challenge
|
47
|
12
|
25.5
|
4.40
|
|
April 2021
|
Tableau workshop
|
131
|
21
|
16.0
|
4.67
|
|
September 2021
|
Intro to eStar
|
89
|
22
|
24.7
|
4.41
|
|
October 2021
|
Data resources and analytics
|
59
|
8
|
13.6
|
4.88
|
|
November 2021
|
Intermediate analysis
|
39
|
5
|
12.8
|
4.20
|
|
January 2022
|
Design principles in app development
|
81
|
8
|
9.8
|
4.75
|
|
February 2022
|
FHIR app building workshop
|
82
|
4
|
4.9
|
4.25
|
|
February 2022
|
Mental Health hackathon
|
38
|
20
|
52.6
|
4.45
|
|
September 2022
|
InformaticCon
|
119
|
35
|
29.4
|
4.97
|
|
February 2023
|
Intro to Epic and build capabilities
|
41
|
8
|
19.5
|
4.75
|
|
March 2023
|
FHIR workshop
|
64
|
9
|
14.1
|
4.10
|
|
March 2023
|
Data resources
|
22
|
7
|
31.8
|
4.86
|
|
April 2023
|
Human-centered design principles
|
22
|
2
|
9.1
|
5.00
|
|
May 2023
|
Evaluation and dissemination
|
16
|
3
|
18.8
|
5.00
|
|
September 2023
|
InformaticCon
|
127
|
44
|
34.6
|
4.93
|
|
September 2023
|
Intro to Epic for nonclinicians
|
36
|
13
|
31.1
|
4.46
|
|
October 2023
|
Intro to Epic build capabilities
|
29
|
8
|
27.6
|
5.00
|
|
April 2025
|
Patient-reported outcome measures
|
32
|
4
|
12.5
|
4.75
|
|
Average
|
|
66.9
|
14.3
|
21.7
|
4.63
|
We have trained 100s of VUMC and VU faculty, staff, trainees, and students. The most
prevalent group of trainees is from DBMI, but we have had numerous trainees across
our clinical departments and divisions (e.g., General Internal Medicine, Pediatrics,
Anesthesiology, Emergency Medicine, ENT, OB/Gyn, Genetic Medicine, GI, Urology, Psychiatry,
Radiation Oncology, Pharmacy), health policy, staff from our HealthIT operational
group, many medical and nursing students, residents and VU School of Medicine students,
students and faculty from the Nursing School, and faculty and participants from VU-based
departments, including Computer Science, Biomedical Engineering, Electronics and Communication
Engineering, and Anthropology.
