A Mobile, Electronic Health Record-Connected Application for Managing Team Workflows in Inpatient Care

Abstract

Background Clinical workflows require the ability to synthesize and act on existing and emerging patient information. While offering multiple benefits, in many circumstances electronic health records (EHRs) do not adequately support these needs.

Objectives We sought to design, build, and implement an EHR-connected rounding and handoff tool with real-time data that supports care plan organization and team-based care. This article first describes our process, from ideation and development through implementation; and second, the research findings of objective use, efficacy, and efficiency, along with qualitative assessments of user experience.

Methods Guided by user-centered design and Agile development methodologies, our interdisciplinary team designed and built Carelign as a responsive web application, accessible from any mobile or desktop device, that gathers and integrates data from a health care institution's information systems. Implementation and iterative improvements spanned January to July 2016. We assessed acceptance via usage metrics, user observations, time–motion studies, and user surveys.

Results By July 2016, Carelign was implemented on 152 of 169 total inpatient services across three hospitals staffing 1,616 hospital beds. Acceptance was near-immediate: in July 2016, 3,275 average unique weekly users generated 26,981 average weekly access sessions; these metrics remained steady over the following 4 years. In 2016 and 2018 surveys, users positively rated Carelign's workflow integration, support of clinical activities, and overall impact on work life.

Conclusion User-focused design, multidisciplinary development teams, and rapid iteration enabled creation, adoption, and sustained use of a patient-centered digital workflow tool that supports diverse users' and teams' evolving care plan organization needs.

Keywords

mobile application - workflows - user-centered design - handoffs

Background and Significance

Over 96% of U.S. hospitals utilize an electronic health record (EHR).[1] EHRs have undoubtedly improved the accessibility and searchability of patient records.[2] However, most EHRs still replicate the structure of paper charts, with information siloed into hundreds of separate tabs and notes.[2] [3] [4] Despite growing medical complexity, increasing medical error,[5] and distribution of care across multiple teams or clinicians resulting in frequent transitions of care,[5] [6] [7] the structure of EHRs often fails to support clinician needs, particularly when working in teams.

The result is a clunky and often frustrating experience for clinicians, associated with increasing time in the electronic chart, confusion, preventable errors, and clinician burnout.[8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] Clinicians must cope with EHRs that are difficult to navigate and contain large amounts of duplicated, scattered, conflicting, and erroneous information.[15] Further, many EHRs are poorly matched to clinicians' operational models for care delivery.[22] Static documentation within EHRs often lags behind care plans that evolve throughout the day. Despite the collaborative nature of modern inpatient and outpatient medicine, EHRs seldom include a centralized workspace. Instead, documentation and task management workflows are confined to separate note and communication interfaces for each clinician, team, or discipline.[23]

Enterprises with large workforces provide teams with digital project management tools for coordinating processes across individuals.[24] Since EHRs do not provide these collaborative workspaces, clinicians instead rely on paper-based workarounds—index cards, rounding or handoff list printouts, and sticky notes.[3] [25] [26] [27] To better support clinicians, health care similarly needs a workflow-oriented resource for organizing the complexities of patient care—arguably one of the most intricate and high-stakes processes of any industry.[28]

Objective

With improving clinician workflows as our priority, we initially planned to create a digital handoff tool to replace printed documents. However, user inquiries revealed that shift-to-shift handoff was only one of many workflows needing better support. Other identified vulnerabilities included: effective visualizations of laboratory and other structured data, collaborative task management, and transitions of care both within a team (e.g., night coverage) and across teams (e.g., intensive care unit to ward).[29] [30] [31] Learning from prior examples of institutional systems designed to improve clinicians' interactions with EHR functionalities,[32] [33] [34] [35] [36] [37] we recognized opportunities to leverage recent advances—mobile technology, facile web application interfaces, and data interoperability[38]—in the creation of a homegrown EHR-adjunct that provides a modern digital experience.

We therefore broadened our scope, aiming to develop a comprehensive digital workflow application that would provide quick access to real-time clinical data and a team-based care planning workspace that would support handoff, task-management, and other processes. Key requirements included full mobile functionality,[39] collaborative access within and between interdisciplinary teams, and the ability to customize views of data for different user workflows. Rather than the static snapshot captured on list printouts, which become outdated within hours of printing,[40] we wanted our tool to be a dynamic, live view of the patient's state. By designing an application to support evolving team workflows, we hoped to increase transparency between clinicians, improve communication, and reduce errors.[27] [41] This article describes our process, from ideation and development through implementation, and the research findings of objective use, efficacy, and efficiency, along with qualitative assessments of user experiences.

Methods

Setting

We developed our application at Penn Medicine, a quaternary academic health system, as a collaboration between our Office of the Chief Medical Information Officer, Information Systems Department, and Center for Healthcare Innovation. It was implemented at our three downtown Philadelphia hospitals. In fiscal year 2016, clinicians at these three hospitals cared for 71,633 adult admissions across 1,659 licensed beds. When implemented (2016), Sunrise (Allscripts Healthcare Solutions Inc., Chicago, Illinois, United States) was the inpatient EHR, and Epic (Epic Systems Corporation, Verona, Wisconsin, United States) was the outpatient EHR. Within a year of implementation, we transitioned to using Epic as the integrated inpatient and outpatient EHR.

Development Methodology

Our design process involved clinicians from multiple specialties and clinical application engineers from our Information Services Department. We employed user-centered design and Agile development methodologies,[42] eschewing pre-existing design assumptions to work instead from user input to optimize utility, usability, and satisfaction.[43] We incorporated team science and handoff best practices.[44] [45]

We first created mockups of user interfaces with focus groups from different disciplines and training levels (9 physicians, 2 medical students, 1 nurse, 1 social worker, and 1 user experience designer). Initial front-end prototypes were adjusted iteratively using a staging database to input testing data. Once the interface achieved high levels of intuitiveness and usability based on qualitative feedback from further focus group sessions with >60 clinicians, we built the back-end database and the application services layer that connects the two. Because our application would introduce modified or novel workflows, we focused on initially releasing a clinically suitable minimal viable product. This approach maximized our ability to gather early user feedback to guide the evolution of future versions and feature enhancements.[43] Development of these iterative improvements continued well past implementation.

Application Description

We designed our application, Carelign, to have two main feature sets: an integrated clinical data visualization dashboard and a collaborative care planning workspace that supports task management and transitions in care. It was built as a responsive web application with an Angular front-end user interface and a Microsoft SQL server cluster database. The user interface connects to its database and to our organization's many other clinical information systems using application program interfaces, HL7 messages, direct database queries, and web services ([Supplementary Appendix S1], available in the online version).[38] Our decision to build the application alongside (rather than within) the main EHR was largely motivated by our requirement of supporting a mobile, paperless handoff and rounding workflow. This was not possible within our EHR at the time (Allscripts) nor within the EHR our health system planned to implement shortly thereafter (Epic). Moreover, our team needed a platform that enabled rapid development with frequent iterations, which is often not feasible in a large, foundational application such as the EHR of a large health system.

Interoperable Data Visualization

We created Carelign as an interoperable, EHR-agnostic application. Prior to Carelign, our health system's providers used multiple software interfaces, including a homegrown web portal, to access clinical information on a single patient. In Carelign, EHR-generated discrete data along with user-generated, semi-structured textual information are represented in one mobile-optimized display, with elements that can be collapsed, tagged, reorganized, and otherwise manipulated for operational efficiency ([Fig. 1]). By providing clinicians with access to this information in real time at the point of care, we aimed to obviate the need to print static rounding lists and transcribe data onto them. However, given the diverse preferences and workflows of different provider groups, we preserved the ability to generate patient list printouts.

Fig. 1 Optimized for mobile and desktop viewing. Carelign adapts to device form-factor while maintaining intuitive displays of data and care plan information.

Dynamic, Collaborative Care Plan

Carelign addresses the duplicative processes of writing and updating daily progress notes, rounding lists, handoffs, and discharge documentation, by providing a dynamic, problem-based care plan that can be applied to all four activities. Instead of boxes for free-text narrative information, Carelign utilizes semi-structured elements for user-generated problems with their associated plan and task items. These elements can be tagged and reorganized into different views to fit different workflows ([Fig. 2]). Active plan and task elements can be exported from the care plan into the progress note. Older information or completed tasks can be archived for future reference. This enables streamlining the handoff and daily progress note without having to delete and lose this information.

Fig. 2 The Carelign care plan. Carelign utilizes modular, semi-structured elements for user-generated problems and the plan elements and tasks contained within them.

Additionally, plan elements can be tagged in other ways: as anticipatory guidance for covering providers, as related to discharge for interdisciplinary planning, or simply as “important.” They can also be reorganized into different views at the patient level for specific workflows, including rounding, cross-covering, and discharge planning. Carelign provides extensive functionality for list-based management of patient and team tasks with views supporting daily task management of the primary team, the cross-covering team, consulting teams, and discharge planners.

Additional Features and Specifications

Mobile-Friendly Web Application

To best support clinical workflows, users require access to the application from any platform and location—hospital computers, workstations on wheels, personal laptops, smartphones, or tablets. We therefore developed Carelign as a responsive web application (can be accessed via any web browser) rather than a native application (must be downloaded from mobile application stores). The responsive web format allowed us to keep the development process device- and operating system-agnostic. It also enabled us to push feature updates without relying on users to download them. This was important given our frequent version updates incorporating user feedback.

Security and Authentication

Carelign's program architecture complies with health system, state, and national security statutes regardless of platform used or site of access. To access Carelign, authorized users must use existing institutional login credentials and permissions on devices connected to our health system's secure network—on-site via the hospitals' Wi-Fi network or remotely via a virtual private network. Carelign also times out after a period of user inactivity. Finally, no patient data are cached or saved locally on user devices to safeguard against data breaches resulting from lost, stolen, or shared devices.

Implementation

We developed Carelign in two phases. We first built the data integration portion of the application with a read-only view of the EHR's handoff documents in 2014, and later replaced that prior handoff with Carelign's collaborative care plan and task management platform in 2015. After prototype revisions based on initial user feedback, we implemented the full version over 6 months in 2016.

The implementation team consisted of two to three part-time clinical members for training and support and two to four part-time developers for building new features, addressing bugs and issues, and technical support. We identified superusers and clinical users, transferred data from the previous handoff to Carelign, and assisted with workflow optimization. Users were trained via informational flyers and videos sent prior to transitioning as well as shoulder-to-shoulder support from Carelign team members on the day of transition.

To provide ongoing user support, our user-interface design included a prominent in-application feedback button that enabled users to submit issue reports, provide feedback, or request new features. This generated an email with user comments, contact information, and supplemental metadata sent to development and clinical implementation team members. After implementation, we transitioned application support to our health system's general information services help desk, with supplementary email feedback also sent to Carelign clinical champions.

Usage Measurements and User Perceptions

Automated in-application audit logs tracked usage data, generating anonymized weekly and monthly reports on user access and intra-application actions.[46] Using time–motion observations during our initial phase of implementation (2014), we measured the impact of Carelign on team behavior during rounds as compared with teams not yet using the application. We observed how often real-time data were accessed when discussing patients. Additionally, using stopwatches, we measured login times for Carelign as compared with the EHR on different devices including smartphones (Carelign only), desktops, and mobile workstations.

We conducted two user surveys in September 2016 and April 2018 to understand users' perceptions of the application and its impact on clinical care ([Supplementary Appendix S2], available in the online version). These surveys were conducted using Qualtrics software (Qualtrics, Provo, Utah) and were each open to responses for 4 weeks. The first was distributed via institutional email to all hospital clinical staff, while the second was shared with users via a survey link on the application login page. This project was reviewed and categorized as quality improvement by the University of Pennsylvania's Institutional Review Board and was therefore exempt from approval.

Results

Adoption and Usage

By July 2016, Carelign was fully implemented as the health system's standardized handoff and digital workflow tool on 152 of 169 possible primary inpatient services at the three hospitals. Nonadopting services either preferred to use a formatted word document or very rarely had inpatients. By July 2020, seven of the remaining services independently adopted the tool. Since then, clinician adoption has continued to grow, with a 41% increase in average weekly login sessions by 2020 ([Table 1] and [Fig. 3]). Adoption was noted both among users who directly relied on the application for their daily workflows (e.g., physician trainees and advanced practice providers [APPs]) and those for whom use of the application was supplemental (e.g., attending physicians, nurses, pharmacists, etc.). In July 2020, 52% of 3,738 unique users were nurses, 11% were APPs, 20% were physician trainees, 6% were attending physicians, and 11% were other clinical users (e.g., pharmacists, therapists, and social workers). Similar ratios were previously observed in 2016 ([Table 1] and [Fig. 3]). In contrast, mode of access—via the EHR, desktop browser, or mobile device—shifted between 2016 and 2020, with users increasingly accessing the application from within the EHR ([Table 1] and [Fig. 4]).

Table 1 Carelign usage data

	July 2016	July 2020	Percent change	Absolute change
Average weekly usage metrics
Unique users	3,275	3,738	14%	463
Login sessions	26,981	38,039	41%	11,058
Breakdown of unique users, by role (%)
Nurses	44%	52%		8%
Physicians (trainee)	23%	20%		−3%
Physicians (attending)	8%	6%		−2%
Advanced practice providers	7%	11%		4%
Breakdown of weekly sessions, by role (%)
Nurses	13%	25%		12%
Physicians (trainee)	57%	45%		−12%
Physicians (attending)	7%	6%		−1%
Advanced practice providers	8%	14%		6%
Breakdown of weekly sessions, by access method (%, monthly average)
Via EHR	20%	59%		39%
Via desktop	42%	21%		−21%
Via mobile	38%	20%		−18%
Activities performed in application
Task management (i.e., add, edit)	302,678	285,080	−6%	−17,598
Facilitate communication (i.e., view, call team members)	8,648	5,180	−40%	−3,468

Abbreviation: EHR, electronic health record.

Note: Physician trainees had the most average login weekly sessions (16,818), followed by nurses and advanced practice providers (combined totals 14,617). These numbers increased from 2016 to 2020 ([Fig. 3]).

Fig. 3 Usage over time. Panel (A) shows total unique weekly users over time and panel (B) shows total unique weekly sessions over time, both broken down by user discipline. Note that users and sessions decreased each year in December, likely due to reduced elective admissions around United States holidays. Likewise, users and sessions decreased from March to May 2020 given efforts to decrease nonessential inpatient admissions during the COVID-19 pandemic. Note: Due to an error in data collection, login sessions for the following date range were omitted from panel (B): 11/19/2017–1/20/2018.

Fig. 4 Access method over time. Most users access Carelign by launching it from within the electronic health record.

Through the in-application feedback mechanism, users generated over 1,150 feedback communications during the first 10 months after implementation. Users described this feature as an easy avenue for addressing concerns that helped them feel supported during and after implementation.

Functions and Efficiencies

Functions Used and Frequencies

As evidenced by observations of teams rounding and by in-application usage patterns, clinicians incorporated real-time clinical data access via Carelign into their clinical workflows. Users accessed objective clinical data (e.g., vitals, laboratories, medications, imaging studies, and encounters) an average of 116,223 times per week (July 2016). The most frequently viewed data sections were vitals and laboratory results. In-application audit logs demonstrated that users frequently interacted with the problem lists by adding, changing, deleting, reordering, and viewing problems. Task items within problems were viewed, edited, created, completed, or deleted on average 302,678 times per week (July 2016) ([Table 1] and [Fig. 5]). Carelign imports care team assignments and contact information, facilitating communication between clinicians. Weekly average counts of instances of contact information viewed or calls generated from within Carelign were 8,648 in July 2016 ([Table 1] and [Fig. 5]).

Fig. 5 Communication and task management. Panel (A) shows the number of times per week that users used Carelign to view care team information or generate phone calls from within Carelign. This decreased since the implementation of EpicCare Inpatient in 2017 but has remained steady since. Panel (B) shows the number of times per week that users added or edited tasks within Carelign. These numbers have remained steady since implementation (except for the months corresponding to the first Spring 2020 surge of the COVID-19 pandemic). Note: Due to errors in data collection, login sessions for the following date range were omitted from panel (A) (05/24/2016–06/04/2016, 09/10/2017–11/04/2017, 12/17/2017–12/23/2017, and 01/13/2019–04/13/2019) and panel (B) (05/24/2016–06/04/2016, 12/17/2017–12/23/2017, and 01/13/2019–04/13/2019).

Efficiencies

During the 2014 initial implementation, time–motion observations of teams using Carelign (n = 14) versus those not yet using it (n = 10) demonstrated that teams using Carelign on rounds accessed real-time data on average 50% more often (168 vs. 79 times, respectively, with 12.0 vs. 7.9 times per team per rounds, respectively). Although they searched for data more often, Carelign users spent 25% less overall time logging into devices (41.11 total minutes/2.94 minutes per rounds vs. 39.55 total minutes/3.9 minutes per rounds, respectively). This was supported by tests showing that logging into and accessing Carelign on a mobile device was 22 times faster than logging into the EHR on a desktop computer (on average, 2.4 and 52.8 seconds, respectively, across n = 39 observations).

Acceptance and Satisfaction

In September 2016, 760 Carelign users from multiple disciplines and specialties replied to our user experience survey ([Table 2]). Their responses indicated that Carelign was used in various clinical settings and during various clinical activities. Physician trainees, APPs, and nurses all reported using it to support transitions in care more than 50% of the time; notably, 97% of first-year residents used it at these times. When asked how long it took to learn to use the application well enough to accomplish typical daily work, 28% of users took less than 1 day, 41% took 2 to 3 days, 21% took 4 to 7 days, and 10% took longer. Most respondents rated Carelign as having a favorable impact on clinical care ([Fig. 6]).

Table 2 Responses to 2016 user experience survey

	Number of responses[a]	% of total respondents[b]
Q1. What is your patient care role?
Advanced practice provider	63	8.6%
Clinical nutritionist	6	0.8%
Medical student	31	4.2%
Nurse	200	27.2%
Pharmacist	35	4.8%
Physician—attending	99	13.5%
Physician—fellow	34	4.6%
Physician—resident	187	25.4%
Social worker/case management	20	2.7%
Therapist (occupational, physical, respiratory)	28	3.8%
Other	32	4.4%
Total respondents	735
Q2 & Q3. What is your main clinical area of practice?
Family medicine	2	0.4%
Home care	39	7.6%
Medicine	233	45.5%
Multiple specialties	29	5.7%
Neurology	9	1.8%
Obstetrics and gynecology	12	2.3%
Psychiatry	4	0.8%
Radiology	3	0.6%
Surgery	93	18.2%
Other	88	17.2%
Total respondents	512
Q6. In which clinical settings have you used Carelign? (Select all that apply)
Ambulatory	46	7.2%
Diagnostics	14	2.2%
Emergency department, short stay units	68	10.6%
Home care	43	6.7%
Labor and delivery	41	6.4%
Medical floor	401	62.7%
Medical intensive care unit	192	30.0%
Procedural areas	52	8.1%
Surgical floor	172	26.9%
Surgical intensive care unit	153	23.9%
Other	49	7.7%
Total respondents	640
Q9. When do you use Carelign? (Select all that apply)
During rounds	345	54.5%
During transitions in care	390	61.6%
While off campus	232	36.7%
While on call	226	35.7%
While prerounding	325	51.3%
While talking to patients	196	31.0%
While writing notes	350	55.3%
Other	86	13.6%
Total respondents	633
Q10. Do you use Carelign at the patient bedside? (Select all that apply)
Yes, to review data in response to patient questions	280	44.3%
Yes, to review data not prompted by patient	198	31.3%
Yes, to view handoff	216	34.2%
Yes, to update the handoff	99	15.7%
No	280	44.3%
Total respondents	632
Q11. [If yes to Q10] When using Carelign at the patient bedside, do you show/share what you are looking at with the patient (either verbally or visually)?
Always	34	9.7%
Often	73	20.8%
Sometimes	109	31.1%
Seldom	60	17.1%
Never	75	21.4%
Total respondents	351

Note: Out of 760 total survey respondents, the number of respondents to each individual question is listed.

^a Reflects the number of respondents selecting each answer option.

^b Denotes the percent of total respondents selecting each option. In the case of “select all that apply” questions, the aggregate number of responses may exceed the number of respondents.

Fig. 6 User perceptions of Carelign on 2016 Survey. Out of 760 user respondents, 598 responded to statements assessing their opinions on the impact of Carelign on different aspects of their workflow efficiency, effectiveness, and satisfaction. Users rated their agreement with the seven statements on a 5-point Likert scale.

In April 2018, we conducted a second survey of Carelign users with three questions about efficiency and efficacy. Of 226 respondents, 81% said it would take them more time to perform their clinical duties without Carelign, 75% felt that it had helped them prevent a medical error, and 80% felt that they would like to have it available as a tool if they moved to a different institution.

Discussion

In health care's rapidly changing and increasingly complex environment, clinicians need health information technology solutions designed to facilitate their workflows and care-delivery tasks in an accessible, team-based, and intuitive fashion.[36] Such a tool should emulate intuitive dashboards, with displays of current data coexisting with the most up-to-date plans. It should support transitions in care and provide a uniform view for collaboration amongst multidisciplinary providers while remaining flexible enough to support their differing workflows. Finally, it should be easily accessible across platforms and settings: from within any EHR or other clinical information system, from computer workstations, from mobile devices, and from home while on call.[25]

To address this need, we built and implemented a novel clinical workflow application to improve data visualization and facilitate clinical workflow. Following a user-centered design strategy, our data demonstrate rapid adoption and sustained usage following implementation within a large academic hospital system. Users report they were able to quickly learn to use the application, that it increased the safety and quality of their handoffs, and that it improved their work experience. Having mobile access to data increased the frequency with which teams accessed data on rounds, while requiring less login time. Our evaluation finds that intuitively designed mobile tools—built with best principles of Agile methodology and a user-centered, problem-focused approach—can be successfully incorporated into clinicians' daily workflow with rapid adoption and, importantly, user satisfaction.[39]

While our initial aim was to create a tool that supported transitions of care, our contextual inquiry led us to more broadly redesign and rethink communication workflows. We therefore shifted our objective to providing a work- and think-space that enables teams to cohesively create and manage care plans in ways similar to project management software. Carelign now serves as a collaborative and unifying platform with shared information and task lists for all clinicians caring for the patient, beyond just the primary team. Its value is supported by the over 285,000 task management actions performed weekly within Carelign in 2020. Use of Carelign increased over the course of our study, even after the health system's implementation of a system-wide integrated EHR. Clinical groups quickly began to envision new uses and even structure quality improvement projects around the application.[47] [48]

Most modern EHRs fail to provide functionalities for collaborative, continuous plan updates and task management. Consequently, much of clinicians' minute-to-minute notations are performed outside of the EHR, often lost on paper.[3] [25] Rather than mirroring pre-existing static paper workflows in a digital format, our challenge was to create a system that leveraged the full capabilities of digital technology and mobile devices. We attempted to match physicians' mental models more closely by designing Carelign to support updating the care plan more easily and frequently than is currently practical with daily progress notes. Doing so then helps the care plan record keep pace with patients' evolving conditions. This centralized care plan information can then serve multiple purposes across intersecting communication and documentation workflows. It can populate rounding lists and handoffs, generate progress notes, and ultimately become a draft for the discharge summary.

We did encounter challenges to implementing our application, including availability of mobile devices for users and mobile device battery depletion. While Wi-Fi and cellular network coverage within clinical areas was adequate, the use of Carelign on mobile devices while en route to see patients revealed uneven coverage in nonclinical areas (e.g., hallways, staircases, and elevators). This recognition prompted efforts to improve network coverage to support such ambulatory use. Replacing paper lists with a mobile tool in some cases led to a workflow mismatch when users needed to reference their lists on their mobile device during a simultaneous phone call. This was not an issue when using Carelign with a desktop workstation or workstation on wheels. However, when relying only on a mobile device, this occasionally required a workaround such as using a landline or headphones for the phone call, or alternatively keeping a list printout as a backup reference. Indeed, while some services have eliminated printed lists from their workflow (e.g., most medicine resident services), others continue to carry list printouts that they rely on to varying degrees (e.g., attending, APP, and surgical resident services).

Finally, the design and evolution of our application benefited greatly from constant user feedback. During pilot phases, application feedback was provided via an external electronic survey. This was poorly utilized and posed challenges to timely responses for our development team. We therefore transitioned to an in-application feedback mechanism. This proved a more streamlined method for users to contact our team and was highly utilized. Key factors for success of this feedback mechanism were: (1) easy access for users within the application, (2) inclusion of user contact information and application metadata, (3) timely responses to user issues and suggestions (within 24 hours if not sooner for urgent issues), and (4) incorporation of ideas into the application when appropriate.

Conclusion

Employing user-centered design and Agile development methodologies, we created an application to facilitate clinical users' evolving needs and workflows. We demonstrate that an EHR-connected data visualization and handoff tool that supports team-based care planning can be readily adopted and incorporated into clinical processes. Clinicians reported that accessing integrated patient information using Carelign led to improvements in their workflows and experience. An intuitive interface enabled implementation with minimal support and training—a departure from many health information technology implementations. Continued studies of EHR-connected applications are needed to enhance clinician workflow and improve interactions and experience with EHRs.

Clinical Relevance Statement

Clinical teams require adequate support for their care coordination and organization workflows. Focusing on user needs, we designed an EHR-connected data visualization and handoff tool to provide an integrated work- and think-space for providers. Ongoing use and activity in the application highlight how Carelign offers necessary and desired support for clinical workflows in ways otherwise unmet despite a fully integrated EHR.

Multiple Choice Questions

1. Your hospital identifies a safety event in which an immunocompromised patient with recent history of resistant E. coli bacteremia was not started on sufficiently broad antibiotics when she developed a fever overnight. Appropriate antibiotics were not instituted until the error was identified 18 hours later. During root cause analysis, your team identifies that the patient's progress notes made clear that the patient should be started on vancomycin, imipenem, and amikacin in the setting of a new fever; however, the written handoff did not include this information. Which of the following systems would be most likely to prevent this error?

   • Instituting a policy in which all febrile, immunocompromised patients are immediately transferred to the intensive care unit.

   • An easily accessible documentation template for fever in immunocompromised patients.

   • Educating providers about the importance of situational awareness in the written handoff.

   • Populating the progress note and written handoff from a single source of truth.

   Correct Answer: The correct answer is option d. In addition to writing and updating admission notes, progress notes, and discharge summaries, front line clinicians (FLCs) must also update plans of care and problem lists in their handoffs. Often, these documents do not communicate with each other, forcing FLCs to maintain each document independently. As length of stay for an individual patient and team census/workload increases, progress notes and handoffs are known to drift away from each other in content and decrease in overall quality.[49] In the absence of links from information in the EHR to elements of the written handoff, inconsistencies are known to occur for the list of problems, code status, medications, and allergies.[26] [40] [50] In this scenario, the key piece of information that was lost between documents (a contingency plan in case this patient developed a fever) is unlikely to be captured discretely in the EHR. Thus, a unified documentation system that populates both the progress note and the written handoff from a single source of truth would most likely have avoided this error since the FLC would only have had to make an update in one location instead of two.

   Instituting a policy requiring transfer of febrile, immunocompromised patients to the intensive care unit would have introduced another handoff and required substantial time to accomplish. The intensive care unit clinicians would still have needed to have read the most recent progress note in addition to any transfer notes or the handoff to have caught this mistake, which may not be part of their workflow. Thus, this approach is not likely to prevent this error in addition to leading to many potentially unnecessary intensive care unit admissions.

   While a documentation template for fever in immunocompromised patients can provide guidance to FLCs to include explicit fever plans, the problem in this case is not that the patient had no fever plan. Rather, this patient's individualized fever plan was not copied over from the progress note to the handoff. Thus, a documentation template alone would probably not have prevented this error. While it is certainly important to educate providers about the importance of situational awareness in the written handoff, high census, individual patient complexity, and increased length of stay make it increasingly difficult for FLCs to keep all inpatient documents updated and high quality. Thus, education alone is unlikely to prevent the error seen in this patient without technological solutions that streamline workflow and decrease the documentation burden.[50]

2. A health care organization aims to increase the number of problems added to the problem list in the inpatient setting. Which of the following inpatient documentation strategies is most likely to increase frustration with the EHR and decreased clinician satisfaction?

   • Force clinicians to enter each problem on the problem list before being able to document the plan related to that problem.

   • Institute educational sessions for all clinicians emphasizing the importance of the problem list for quality improvement, research, and billing.

   • Audit inpatient progress notes weekly and contact clinicians about possible missed problems that should be added to the problem list.

   • Audit inpatient progress notes and problem lists and let clinicians know where they rank in terms of problem list completeness compared with their peers.

   Correct Answer: The correct answer is option a. In addition to simplifying administrative efforts, quality improvement, and billing, accurate problem list documentation improves patient outcomes.[51] However, because problem list entry is often not a part of the standard workflow of front-line clinicians, problem list completeness and accuracy are often poor. Forcing clinicians to enter a problem on the problem list before being able to document a plan for that problem, mandatory problem-oriented charting may in fact be the most effective method for achieving a more complete problem list[52]; however, this forced departure from workflow to complete a task that does not feedback to an improvement in clinical workflow is certain to lead to increase frustration and burnout. Clinical informaticists should be aware that such rigid structures and forcing functions may aggravate clinicians due to limited expressivity.[53] [54]

   Auditing inpatient progress notes to contact clinicians about possible missed problems and gap reporting improves problem list usage but may have less impact. Auditing inpatient progress notes to notify clinicians where they rank against their peers would be similarly resource-intensive though the benchmarking between colleagues may drive internal desires to improve.

   Educational sessions may help produce an organizational culture that expects problem list completeness and would most certainly be better tolerated than mandatory problem list completion.

3. A research team is working to understand the causes of medical errors overnight and wants to understand the relationship between contingency plans made by the daytime clinical team and overnight clinical decisions. In which inpatient document should overnight contingency plans be found to minimize communication errors?

   • Progress note.

   • Problem list.

   • To-do list.

   • Handoff.

   Correct Answer: Option d is the correct answer. Front-line clinicians in the inpatient setting are responsible for multiple clinical documents for each admitted patient. These documents contain information with different goals and audiences. Of the choices listed, the primary goal of the handoff for each patient is to communicate between daytime and overnight/weekend providers so that the oncoming provider can accomplish the necessary tasks and make appropriate clinical decisions as problems arise. The IPASS method for handoffs has shown decreased medical errors associated with communication between providers and includes an assessment of illness severity, patient summary, action items, situational awareness/contingency planning, and synthesis by the receiver.[55] Thus, the handoff would be the most appropriate place to find information on contingency plans.

   The intended audience for the progress note is wider and includes not only other clinicians caring for the patient but also billers and regulatory agencies, leading to additional included information. Thus, while the progress note often contains much of the same information as a handoff, it is often more difficult to find key information on task management and situational awareness that is of extra importance to a cross-covering provider. The key function of the problem list is to allow any provider to immediately make medical decisions in context for a particular patient.[56] In addition, it provides structured data that can drive clinical decision support, quality improvement initiatives, research, and billing. While it is often incorporated into a handoff and plans about each problem may include contingency plans, the problem list itself generally does not contain a dedicated section for contingency planning where cross-covering providers can look. The to-do list is also often incorporated into a handoff and contains discrete action items that should be completed in a timely manner. It is not intended to help cross-covering providers make medical decisions in case of a change in clinical status.

4. A residency program is trying to increase the workflow of bringing residents “back to the bedside” to improve their teaching environment, their clinical experience, and quality of care. The program leadership has asked medical teams to round in the room with the patient, discuss their plan with participation from the patient, and perform teaching there as well. Despite multiple efforts to implement this new workflow, they have not had good success with adoption.

   Which of the following factors has had the most negative impact on this effort, leading to poor adoption of the initiative?

   • The residents are not interested in teaching; they are too busy because their patient loads are increasing.

   • There are no computers in the hospital rooms, therefore the residents feel they are unable to get enough work done if they spend their whole morning rounding without access to the EHR.

   • The patients have stated they prefer not to be involved in the conversation as it increases their stress levels.

   • Program leadership has not spent enough time teaching the residents how to do bedside rounds.

   Correct Answer: Option b is the correct answer. As we have learned in recent studies,[12] clinicians are spending more and more time in front of a computer to be able to complete the required work to care for patients. Time–motion studies have shown that clinicians are spending approximately 50% of their time in front of the EHR or on other tasks at a desk and approximately 27% on direct patient care. In an environment where clinician work is so heavily dependent on access to the EHR, there is a growing need for mobile access to the EHR or at the very least access to the EHR at the bedside if we want to encourage bringing clinicians back to the bedside.

Conflict of Interest

The application described in this manuscript was designed, developed, and implemented by an internal group of clinicians and clinical application developers at Penn Medicine. There were no outside funds used to design, build, implement, or study the application as described in this manuscript. In 2018—after the design, development, and implementation of this application described in the manuscript—the project leader and the last author of the paper, Dr. Subha Airan-Javia, and the Penn Center for Innovation launched a start-up company to bring the application into other health systems (TrekIT Health Inc. d/b/a CareAlign). Both Dr. Airan-Javia and the Board of Trustees of the University of Pennsylvania own equity in the company and receive royalty payments on an annual basis from sales of the company. Dr. Airan-Javia is a full-time salaried employee and CEO of the company, as well as a member of the Board of Directors. No other authors have any involvement in this company, financial or otherwise. The other authors declare that they have no conflicts of interest related to this work. Dr. Airan-Javia reports no salary from TrekIT Health Inc. during the conduct of the study; but as stated above, has received salary from TrekIT Health Inc. outside the submitted work.

Acknowledgments

This work was supported by and used the facilities and staff of Penn Medicine, located in Philadelphia, Pennsylvania, United States. The authors would like to acknowledge the Penn Medicine Office of the Chief Medical Information Officer, Information Services Department, and Center for Healthcare Innovation for their support of our application. We also received support and collaboration from leadership, residents, and advanced practice providers from clinical departments across our institution. Their input and participation were instrumental to the application's development, design, and implementation. In particular, we appreciate the individual efforts of Carolina Garzón-Mrad, MSE, Eugene Gitelman, MD, and David Do, MD, in initial design and contextual inquiry; of Kinnari Patel and Robert Hussein for initial data extraction from our previous EHR system; of Stephen Kratowicz and Christopher Bellis as key developers from our application development team; and Poppy Bass for her role as initial project manager. Jillian Olsen, MD, studied the impact of Carelign on resident workflows upon initial implementation. Additionally, Shazia Siddique, MD, provided insight into the consultant experience with Carelign and later studied Carelign's impact on consultant service lines. Colleen Mallozzi, MBA, BSN, and Sean Sarles, MSN, were our principal representatives from nursing leadership. The following providers played an invaluable role as clinical ambassadors for their respective service-lines: Brendan DiDonna, PA-C in critical care; David Do, MD in neurology; Eugene Gitelman, MD; Fima Macheret, MD, MS; Mark Yarchoan, MD and Lucas Kimmig, MD in internal medicine; Rebecca Hoffman, MD in general surgery; and Alexander Sackheim, MD in emergency medicine. Finally, we are grateful for the efforts of our many resident moonlighters who assisted in content conversion to facilitate the transition to Carelign.

Author Contributions

Application conception, design, or development: Soegaard, Urbani, Fala, Patel, Leri, Rosin, Hanson, and Airan-Javia. Application implementation: Soegaard, Bass, Urbani, Patel, Leri, and Airan-Javia. Acquisition of data: Urbani, Patel, and Airan-Javia. Analysis or interpretation of data: Soegaard, Urbani, Koppel, and Airan-Javia. Drafting of manuscript: Soegaard, Steinkamp, Denson, Koppel, and Airan-Javia. Critical revision of the manuscript for important intellectual content: Soegaard, Bass, Urbani, Fala, Patel, Leri, Steinkamp, Denson, Rosin, Adusumalli, Hanson, Koppel, and Airan-Javia. Executive, administrative, technical, or material support: Urbani, Fala, Patel, Rosin, and Hanson. Study supervision: Airan-Javia.

Protection of Human and Animal Subjects

No human or animal subjects were directly involved in this project. This project was reviewed and categorized as quality improvement by the University of Pennsylvania Institutional Review Board and was therefore exempt from approval.

• References

• 1 Henry J, Pylypchuk Y, Searcy T, Patel V. ONC Data Brief 35: Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals (2008-2015); 2016. Available at: https://dashboard.healthit.gov/evaluations/data-briefs/non-federal-acute-care-hospital-ehr-adoption-2008-2015.php
• 2 Jamoom EW, Patel V, Furukawa MF, King J. EHR adopters vs. non-adopters: Impacts of, barriers to, and federal initiatives for EHR adoption. Healthc (Amst) 2014; 2 (01) 33-39
• 3 Karsh BT, Weinger MB, Abbott PA, Wears RL. Health information technology: fallacies and sober realities. J Am Med Inform Assoc 2010; 17 (06) 617-623
• 4 Koppel R, Kuziemsky C. Usability across health information technology systems: searching for commonalities and consistency. Stud Health Technol Inform 2019; 264: 649-653
• 5 Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ 2016; 353: i2139
• 6 Abraham J, Kannampallil T, Patel VL. A systematic review of the literature on the evaluation of handoff tools: implications for research and practice. J Am Med Inform Assoc 2014; 21 (01) 154-162
• 7 Denson JL, Jensen A, Saag HS. et al. Association between end-of-rotation resident transition in care and mortality among hospitalized patients. JAMA 2016; 316 (21) 2204-2213
• 8 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
• 9 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 10 Babbott S, Manwell LB, Brown R. et al. Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc 2014; 21 (e1, E2): e100-e106
• 11 Spinelli WM, Fernstrom KM, Britt H, Pratt R. “Seeing the patient is the joy:” a focus group analysis of burnout in outpatient providers. Fam Med 2016; 48 (04) 273-278
• 12 Yan Q, Jiang Z, Harbin Z, Tolbert PH, Davies MG. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc 2021; 28 (05) 1009-1021
• 13 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 14 Overhage JM, McCallie Jr D. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med 2020; 172 (03) 169-174
• 15 Beasley JW, Wetterneck TB, Temte J. et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011; 24 (06) 745-751
• 16 Bowman S. Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag 2013; 10 (Fall): 1c
• 17 Carayon P, Du S, Brown R, Cartmill R, Johnson M, Wetterneck TB. EHR-related medication errors in two ICUs. J Healthc Risk Manag 2017; 36 (03) 6-15
• 18 Leventhal R. Malpractice claims report: EHR documentation errors still far too common. Healthcare Innovation. Published November 4, 2020. Accessed March 20, 2021 at: https://www.hcinnovationgroup.com/clinical-it/electronic-health-record-electronic-medical-record-ehr-emr/news/21161325/malpractice-claims-report-ehr-documentation-errors-still-far-too-common
• 19 Friedberg MW, Chen PG, Van Busum KR. et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Health Q 2014; 3 (04) 1
• 20 Starren JB, Tierney WM, Williams MS. et al. A retrospective look at the predictions and recommendations from the 2009 AMIA policy meeting: did we see EHR-related clinician burnout coming?. J Am Med Inform Assoc 2021; 28 (05) 948-954
• 21 Tajirian T, Stergiopoulos V, Strudwick G. et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res 2020; 22 (07) e19274
• 22 Smith SW, Koppel R. Healthcare information technology's relativity problems: a typology of how patients' physical reality, clinicians' mental models, and healthcare information technology differ. J Am Med Inform Assoc 2014; 21 (01) 117-131
• 23 Abbott PA, Weinger MB. Health information technology: fallacies and sober realities - redux a homage to Bentzi Karsh and Robert Wears. Appl Ergon 2020; 82: 102973
• 24 Miljanic M, Zaric N. Review of collaborative software applications and integration with standard collaboration tools. In: 2020 24th International Conference on Information Technology, IT 2020. Institute of Electrical and Electronics Engineers Inc.; 2020
• 25 Vidyarthi AR, Coffey M. Paperless handover: are we ready?. BMJ Qual Saf 2016; 25 (05) 299-301
• 26 Ham III PB, Anderton T, Gallaher R. et al. Development of electronic medical record-based “Rounds Report” results in improved resident efficiency, more time for direct patient care and education, and less resident duty hour violations. Am Surg 2016; 82 (09) 853-859
• 27 Aylward MJ, Rogers T, Duane PG. Inaccuracy in patient handoffs: discrepancies between resident-generated reports and the medical record. Minn Med 2011; 94 (12) 38-41
• 28 Emanuele J, Koetter LE. Workflow opportunities and challenges in healthcare. BPM & Workflow Handbook 1: 157. 2007;(July). Accessed November 15, 2021 at: https://www.researchgate.net/publication/252065707_Workflow_Opportunities_and_Challenges_in_Healthcare
• 29 Li P, Ali S, Tang C, Ghali WA, Stelfox HT. Review of computerized physician handoff tools for improving the quality of patient care. J Hosp Med 2013; 8 (08) 456-463
• 30 Petersen LA, Orav EJ, Teich JM, O'Neil AC, Brennan TA. Using a computerized sign-out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv 1998; 24 (02) 77-87
• 31 Van Eaton EG, Horvath KD, Lober WB, Rossini AJ, Pellegrini CA. A randomized, controlled trial evaluating the impact of a computerized rounding and sign-out system on continuity of care and resident work hours. J Am Coll Surg 2005; 200 (04) 538-545
• 32 Antoine Geissbuhler RAM. WizOrder, a user-friendly interface for order entry and clinical decision support tools. Proc Annu Symp Comput Appl Med Care 1995; 1002
• 33 Miller RA, Waitman LR, Chen S, Rosenbloom ST. The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform 2005; 38 (06) 469-485
• 34 Duke JD, Morea J, Mamlin B. et al. Regenstrief Institute's Medical Gopher: a next-generation homegrown electronic medical record system. Int J Med Inform 2014; 83 (03) 170-179
• 35 Yount RJ, Vries JK, Councill CD. The medical archival system: an information retrieval system based on distributed parallel processing. Inf Process Manage 1991; 27 (04) 379-389
• 36 Giuse DA. Supporting communication in an integrated patient record system. AMIA Annu Symp Proc 2003; 1065
• 37 Giuse NB, Williams AM, Giuse DA. Integrating best evidence into patient care: a process facilitated by a seamless integration with informatics tools. J Med Libr Assoc 2010; 98 (03) 220-222
• 38 Dullabh P, Hovey L, Heaney-Huls K, Rajendran N, Wright A, Sittig DF. Application programming interfaces in health care: findings from a current-state sociotechnical assessment. Appl Clin Inform 2020; 11 (01) 59-69
• 39 Abbott PA. The effectiveness and clinical usability of a handheld information appliance. Nurs Res Pract 2012; 2012: 307258
• 40 Rosenbluth G, Jacolbia R, Milev D, Auerbach AD. Half-life of a printed handoff document. BMJ Qual Saf 2016; 25 (05) 324-328
• 41 Derienzo C, Lenfestey R, Horvath M, Goldberg R, Ferranti J. Neonatal intensive care unit handoffs: a pilot study on core elements and epidemiology of errors. J Perinatol 2014; 34 (02) 149-152
• 42 Boustani MA, van der Marck MA, Adams N. et al. Developing the agile implementation playbook for integrating evidence-based health care services into clinical practice. Acad Med 2019; 94 (04) 556
• 43 Papoutsi C, Wherton J, Shaw S, Morrison C, Greenhalgh T. Putting the social back into sociotechnical: case studies of co-design in digital health. J Am Med Inform Assoc 2021; 28 (02) 284-293
• 44 Arora V, Johnson J. A model for building a standardized hand-off protocol. Jt Comm J Qual Patient Saf 2006; 32 (11) 646-655
• 45 Starmer AJ, O'Toole JK, Rosenbluth G. et al; I-PASS Study Education Executive Committee. Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs. Acad Med 2014; 89 (06) 876-884
• 46 Melnick ER, Sinsky CA, Krumholz HM. Implementing measurement science for electronic health record use. JAMA 2021; 325 (21) 2149-2150
• 47 Adusumalli S, Siddique S, Airan-Javia S. Carelign: closing the loop on inpatient consultant communication. In: Academy Health Conference Proceedigs. Academy Health; 2017. Accessed November 15, 2021 at: https://academyhealth.confex.com/academyhealth/2017arm/meetingapp.cgi/Paper/17259
• 48 Shafer BM, Gorry T, Tapino P, Airan-Javia S. Carelign: a novel handoff system for medical and surgical consultants. Nepal J Ophthalmol 2020; 12 (01) e63-e66
• 49 Miller DM, Schapira MM, Visotcky AM. et al. Changes in written sign-out composition across hospitalization. J Hosp Med 2015; 10 (08) 534-536
• 50 Airan-Javia SL, Kogan JR, Smith M. et al. Effects of education on interns' verbal and electronic handoff documentation skills. J Grad Med Educ 2012; 4 (02) 209-214
• 51 Hartung DM, Hunt J, Siemienczuk J, Miller H, Touchette DR. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med 2005; 20 (02) 143-147
• 52 Wright A, McCoy AB, Hickman T-TT. et al. Problem list completeness in electronic health records: a multi-site study and assessment of success factors. Int J Med Inform 2015; 84 (10) 784-790
• 53 Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc 2011; 18 (02) 181-186
• 54 Wright A, Maloney FL, Feblowitz JC. Clinician attitudes toward and use of electronic problem lists: a thematic analysis. BMC Med Inform Decis Mak 2011; 11 (01) 36
• 55 Starmer AJ, Spector ND, Srivastava R. et al; I-PASS Study Group. Changes in medical errors after implementation of a handoff program. N Engl J Med 2014; 371 (19) 1803-1812
• 56 Weed LL. Medical records that guide and teach. N Engl J Med 1968; 278 (12) 652-657

Publication History

Received: 27 April 2021

Accepted: 11 October 2021

Article published online:
22 December 2021