An Analysis of Patient Safety Incident Reports Associated with Electronic Health Record Interoperability

Summary

Background: With the widespread use of electronic health records (EHRs) for many clinical tasks, interoperability with other health information technology (health IT) is critical for the effective delivery of care. While it is generally recognized that poor interoperability negatively impacts patient care, little is known about the specific patient safety implications. Understanding the patient safety implications will help prioritize interoperability efforts around architectures and standards.

Objectives: Our objectives were to (1) identify patient safety incident reports that reflect EHR interoperability challenges with other health IT, and (2) perform a detailed analysis of these reports to understand the health IT systems involved, the clinical care processes impacted, whether the incident occurred within or between provider organizations, and the reported severity of the patient safety events.

Methods: From a database of 1.735 million patient safety event (PSE) reports spanning multiple provider organizations, 2625 reports that were indicated as being health IT related by the event reporter were reviewed to identify EHR interoperability related reports. Through a rigorous coding process 209 EHR interoperability related events were identified and coded.

Results: The majority of EHR interoperability PSE reports involved interfacing with pharmacy systems (i.e. medication related), followed by laboratory, and radiology. Most of the interoperability challenges in these clinical areas were associated with the EHR receiving information from other health IT systems as opposed to the EHR sending information to other systems. The majority of EHR interoperability challenges were within a provider organization and while many of the safety events reached the patient, only a few resulted in patient harm.

Conclusions: Interoperability efforts should prioritize systems in pharmacy, laboratory, and radiology. Providers should recognize the need to improve EHRs interfacing with other health IT systems within their own organization.

Citation: Adams KT, Howe JL, Fong A, Puthumana JS, Kellogg KM, Gaunt M, Ratwani RM. An analysis of patient safety incident reports associated with electronic health record interoperability. Appl Clin Inform 2017; 8: 593–602 https://doi.org/10.4338/ACI-2017-01-RA-0014

Keywords

Interoperability - patient safety - electronic health records

1. Background and Significance

Electronic health records (EHRs) have been rapidly adopted by healthcare providers in the United States with over 96% of providers currently using an EHR [[1]]. For many providers, the EHR has become the primary platform for most clinical tasks including documenting patient information and viewing patient history, ordering medications as well as lab and diagnostic tests, viewing results, and communicating with other providers and patients. While EHRs are widely adopted and used extensively for most clinical tasks, EHR technology is at varying levels of interoperability with other health information technology (health IT) within the same provider organization and external to the provider. For example, in a hospital setting the EHR may not be interoperable with radiology, laboratory, or pharmacy information systems. The EHR also may not be interoperable with health IT at external provider organizations including EHRs at other hospitals and outpatient care facilities, and technology in support services like laboratories.

From a technical perspective, semantic interoperability is the ability of different information technology systems and software applications to communicate, exchange data, and use the information that has been exchanged with the same confidence as if that information has been generated in the same system [[2]]. From a frontline clinician perspective, interoperability can be defined as the ability of a system to exchange electronic health information with and use electronic health information from other systems without special effort on the part of the user [[3]]. The criticality of EHR interoperability for improved healthcare delivery, particularly efficiency, has been stressed by several stakeholders [[4]–[7]]. Patient safety implications associated with poor interoperability has received considerably less attention. One study of patient safety hazards associated with computer use by healthcare providers reported that 20% of the safety hazard reports were associated with information transfer and about half of these were related to system integration problems [[8]]. While this study suggests that system integration, and perhaps interoperability impacts patient safety, it is unclear whether these safety hazards are related specifically to the EHR and, importantly, it is unclear which clinical areas are impacted by interoperability challenges (e.g. radiology, laboratory, pharmacy, etc.).

In this paper, we analyzed patient safety event (PSE) reports to better understand safety hazards associated with EHRs and potential interoperability issues with other health IT during the care process. In an effort to improve safety, most healthcare systems have a patient safety reporting system (PSRS) in place [[9],[10]]. These systems provide a method for staff, including physicians, nurses, and technicians, to report on safety events in their environment ranging from near misses, where harm almost reaches a patient, to serious safety events, where a patient is harmed [[11]]. The Institute of Medicine has strongly recommended the use of these systems to identify why patients are harmed by medical errors, and several states require the use of a PSRS [[11]].

Patient safety event reports offer a unique lens into patient safety hazards. Details provided in the PSE reports allow us to identify the flow of information between the EHR and other technology (e.g. sending information or receiving information) and the clinical process that is impacted by the interoperability challenges, such as radiology, laboratory, or pharmacy. Conducting this type of analysis offers key insights into the prevalence of reported interoperability challenges within different clinical areas and the impact of these challenges on patient care. This knowledge can subsequently be used to prioritize interoperability efforts, including identifying key system architectures that need to be addressed and standards that need to be formulated.

2. Objectives

The purpose of this study was to understand the patient safety implications that arise from challenges with the interoperability between EHRs and other health IT. Our objectives were to (1) identify patient safety event reports that reflect EHR interoperability challenges, and (2) perform a detailed analysis of these reports to understand the health IT systems involved, the clinical care process impacted, whether the incident occurred within or between provider organizations, and the reported severity of the patient safety event.

3. Methods

3.1 Data Source

Data were comprised of PSE reports from the Pennsylvania Patient Safety Authority’s Pennsylvania Patient Safety Reporting System, attained through the Institute for Safe Medication Practices (ISMP), and a large healthcare system in the Mid-Atlantic United States area between 2009 and 2016. Combined there were a total of 1.735 million PSE reports included in this analysis and the reports spanned all event type categories (e.g. falls, medication, diagnostic imaging, etc.).

3.2 The Content Structure of Patient Safety Event Reports

Patient safety event reports generally contain structured information such as the time and site of occurrence, role of the participants (e.g. physician, nurse, or technician), patient demographic and clinical attributes, as well as a classification of the severity and type of event. In addition to the structured data elements, the PSE reports also include an unstructured free-text field in which the reporter must provide a narrative describing the event in greater detail [[12]]. Patient safety reporting systems can grow to contain tens of thousands to hundreds of thousands of events. We analyzed thousands of PSE reports, with a focus on the free-text narrative, to identify safety hazards associated with interoperability between the EHR and other health IT.

3.3 Data Selection Criteria and Interoperability Classification

To identify EHR interoperability related reports we first filtered the 1.735 million reports by those that were self-identified (i.e. user selected) as being health IT related. This resulted in 2625 PSE reports. Using the free text narratives as the foundation for coding, the health IT related events were manually coded by three annotators [JP, JH, RR] with expertise in health IT to determine those events explicitly related to interoperability. Ten percent of the data was collectively coded (e.g. each report was coded by all three annotators) to establish inter-rater reliability. Differences were reconciled through group discussion. Inter-rater reliability was calculated using Fleiss’ kappa which resulted in 0.69 for the interoperability coding. The remaining health IT related events were divided among the annotators and were individually coded by the three annotators (e.g. each report was coded by one annotator).

For an event to be identified by the coders as being EHR and interoperability related, the event must have explicitly identified an EHR process and interaction with another health IT system in the narrative of the event report. Computerized provider order entry (CPOE) is considered a component of the EHR. Other health IT systems include any system that is intended to interface with the EHR such as radiology systems (e.g. image viewing and results reporting systems), laboratory systems, pharmacy systems, devices (e.g. glucometer), billing and registration systems, and other EHRs (e.g. from external provider organizations or departments within the provider organization). Examples of EHR interoperability and non-interoperability reports are provided in [Table 1].

Table 1

Examples of interoperability and non-interoperability associated patient safety event reports.

Interoperability Related PSE Report	Non-Interoperability Related PSE Report
Example 1: Recurring problem of pathology reports not interfacing with EHR from lab documentation tool. Biopsy ordered and performed. Report not sent to the patient’s chart.	Example 3: Pt was scheduled for surgery. Pre-op orders were written at 9am but not released in EHR by the ordering prescriber. Delayed releasing of pended orders and administration of the medications led to delay in start of procedure by 1 hour.
Example 2: Physician placed electronic medication orders in EHR for drug. Medication orders were not in pharmacy system. IT notified- interface issue. Medication orders did not reach pharmacy system. Patient missed 1 dose of medication.

Classifying each of the PSE reports necessitated certain assumptions. We assumed clinical competency, meaning providers know the right actions to take unless explicitly stated otherwise (e.g. providers correctly entered orders into the appropriate sections of the EHR), and that electronic systems were being used unless explicitly stated otherwise (e.g. paper charts) and were functioning as intended. The EHR and other health IT systems could be from the same or different vendors.

Reports that described a challenge with two aspects or components of the same EHR product were excluded from analysis. We excluded PSE reports that occurred during EHR downtime, network outages, or connectivity failures. Ambiguously written narratives and those involving patient portals were also excluded. In addition, reports that described events explicitly caused by human error, such as wrong patient selection, and those that involved devices that did not explicitly connect to an EHR system (e.g. bed alarms, medication dispensers) were excluded from analysis.

Ultimately, 209 PSE reports were determined to be related to interoperability. The review process is shown in [Figure 1].

Fig. 1 Overview of the process to identify EHR interoperability related safety events.

3.4 Coding EHR Interoperability Associated Reports

With the 209 interoperability events identified, we coded the reports to identify the clinical areas involved, the aspect of the clinical process in which the issue occurred, whether events occurred within or between provider sites, and the reported harm score.

Clinical Areas of Interoperability Related Events

In our coding, interoperability related PSE reports were analyzed by the following clinical areas:

• Medication events occurred between the EHR (CPOE), the pharmacy health IT systems, and the electronic medication administration record (EMAR).

• Laboratory events occurred between the EHR (CPOE), laboratory systems (e.g. LIS), and other EHR systems or components of the EHR.

• Radiology events occurred between the EHR (CPOE), radiology systems, (e.g. imaging device, PACS, RIS, IDX), and other EHRs or components of the EHR.

• Device events occurred between devices (e.g. glucometers, medication scanners) and the EHR.

• The interoperability events that did not fit into the medication, laboratory, radiology, or device categories were coded as “other”. Examples of “other” interoperability events include the EHR and the billing system or the EHR and the dietary services system.

Sending/Receiving Information and Interoperability Related Events

Each interoperability related event was coded to determine whether the event included the EHR sending information to another health IT system (e.g. medication order; EHR (CPOE) sends information to the pharmacy system) or receiving information from another health IT system (e.g. radiology results; radiology system sends results/images to the EHR).

Provider Sites and Interoperability Related Events

The health IT systems involved in the PSE reports were coded to determine whether the interoperability challenge was within a provider organization, between a provider organization and an external organization, or unclear. If an event report described two systems that were within the same provider organization (e.g. hospital EHR to hospital pharmacy system) then it was coded as within. If an event report described two systems that were at different provider organizations (e.g. outpatient care facility to hospital) then it was coded as external. Events where the provider sites could not be determined were coded as unclear.

Harm Scores of Interoperability Related Events

The person reporting the PSE provides a self-reported severity level or harm score of the safety hazard which is categorized as an A through I level of harm (left side of [Table 2]). We analyzed the harm scores of the events identified as EHR interoperability related and organized the original harm scores into four categories (right side of [Table 2]).

Table 2

Reported severity scores and categories used in analysis.

Original Reported PSE Severity Score	New Categories of Severity Scores
A. Unsafe Condition (Non Event)	Unsafe Condition
B1. Near Miss – No Harm – Didn’t Reach Patient – Caught By Chance	Didn’t Reach Patient – No Harm
B2. Near Miss – No Harm – Didn’t Reach Patient – Active Recovery By Caregivers
C. No Harm – Reached Patient – No Monitoring Required	Reached Patient – No Harm
D. No Harm – Reached Patient – Monitoring Required
E. Harm – Temporary Harm – Intervention Needed	Patient Harm
F. Harm – Temporary Harm – Hospitalization Needed
G. Harm – Permanent Harm
H. Harm – Permanent Harm – Intervention Required to Sustain Life
I. Death

4. Results

Two hundred nine (8%) PSE reports of the 2625 health IT reports were determined to be related to interoperability between the EHR and another health IT system. The interoperability related PSE reports were further analyzed to examine clinical areas, systems sending/receiving information, provider site, and harm score as outlined below.

Clinical Areas of Interoperability Related PSE Reports

Medication events comprised 60 (29%) of the interoperability related reports, followed by laboratory (N = 55, 26%), radiology (N = 43, 21%), other (N = 29, 14%) and device (N = 22, 11%).

Process Areas of Interoperability Related PSE Reports

Of the five categories of EHR interoperability events, medication, radiology, and laboratory involve information being sent from the EHR to systems in these areas (e.g. orders) and information being sent back to the EHR (e.g. results or tasks to complete). These three clinical process areas account for 158 (76%) of the interoperability related reports. From this subset, only 51 (32%) of reports described a hazard that occurred when EHR (CPOE) sent information to another clinical health IT system. The majority of these reports, 107 (68%), involved information being sent from another health IT system into the EHR. [Figure 2] illustrates the directional network relationships between these clinical areas and the EHR.

Fig. 2 EHR and interoperability with other health IT systems.

Provider Sites and Interoperability Related PSE Reports

As seen in [Table 3] below, 167 (80%) reports incorporated sufficient information for the coders to understand the relationship between the two health IT systems involved, either within the same provider organization (N = 130, 62%) or from one provider to an external provider organization (N = 37, 18%). Forty-two reports (20%) did not include enough information for the research team to determine within/external provider organizations.

Table 3

Count and percent of PSE reports describing interoperability within and to external provider organizations.

	Count of Interoperability Related PSE Reports	Percent of Total Interoperability PSE Reports
Within Provider Organization	130	62%
To an External Provider Organization	37	18%
Cannot be Determined	42	20%

Harm Scores of Interoperability Related PSE Reports

We analyzed the patient harm score for the 209 interoperability related events as seen in [Figure 3]. Overall, “unsafe conditions” accounted for 38 (18%) of all interoperability events, “didn’t reach patient – no harm” 55 events (26%), “reached patient – no harm” 111 events (53%), and “patient harm” 5 events (2%).

A majority of medication (N = 42, 70%), laboratory (N = 33, 60%), and radiology (N = 22, 51%) system related events reached the patient. Device and other interoperability related PSE reports primarily described an unsafe hazard that was caught before reaching the patient; 15 (68%) of device events and 17 (59%) of other events never reached the patient

Fig. 3 Harm scores of interoperability related PSE reports identified by clinical areas

5. Discussion

In this study, we examined self-identified health IT PSE reports and coded events that were related to interoperability issues with an EHR and other health IT systems during the care process. Interoperability related events were further analyzed by clinical areas, process areas, within/external provider organizations, and severity level of the reports. Our analysis demonstrates that interoperability failures are involved in patient safety hazards, some of which lead to patient harm.

In our analysis approximately 8% of self-identified health IT PSE reports were related to interoperability issues, consistent with previous literature [[13]]. The results demonstrate that medication, laboratory, and radiology events accounted for over three-quarters of all interoperability related events identified with medication events being the most frequently reported. The majority of interoperability safety events had to do with the EHR receiving information from other systems and not the EHR sending information. Surprisingly, most interoperability challenges were within a provider organization as opposed to challenges sending or receiving information to external provider organizations. Finally, many interoperability related events reached the patient, however, most events did not result in harm to the patient.

Our analysis highlights important areas for focused interoperability groundwork and the knowledge gleaned from the PSE report analysis can be used to prioritize interoperability efforts. This study suggests that medication events should be of central focus, followed by laboratory and radiology. For large provider organizations, efforts should be focused on interoperability between the EHR and other information technology within the organization.

The results highlight important areas of focus for future research. It is important to understand the results presented here in the context of overall EHR activity. Currently, it is unclear as to what proportion of radiology-, medication-, and laboratory-related actions comprise all EHR actions and therefore it is difficult to gauge if one clinical process is particularly over or under-represented in the patient safety event data. In addition, as health IT becomes more pervasive it will be important to develop patient safety event reporting systems that capture health IT interactions to further safety improvements, without burdening the reporter. This might include automated capture of the health IT systems that are active during a safety hazard that is being reported.

There are limitations to our PSE analysis. Patient safety event reports are a self-reported data source and often these data have not been reviewed for accuracy of the event that is being described. We analyzed reports self-identified as health IT related; however, there may be many more health IT related events that were not identified as such, though previous research suggests that the sensitivity of self-identification is high. A 2014 analysis of overall health information technology adverse events in two reporting systems concluded that approximately 98.8% of the event report data was correctly coded as “not health IT related”[[14]]. Corroborating this finding in our data by identifying health IT related events that were not explicitly identified by the reporter is a focus of future work. Our analysis is also limited by the details provided in each event report. In addition, patient safety reports do not capture all of the safety hazards in healthcare and are subject to the biases of the frontline staff entering the reports. These data serve as one source of insight into patient safety hazards.

6. Conclusions

The most common EHR interoperability challenges, as identified through the analysis of patient safety event reports, involved the EHR interfacing with pharmacy, laboratory, and radiology systems, respectively. Interoperability efforts should prioritize these areas, given their clinical importance, and providers should recognize that many interoperability challenges involve the EHR interfacing with other systems within their own organization.

Clinical Relevance Statement

Our research identifies pharmacy, laboratory, and radiology as critical areas to focus EHR interoperability efforts. Health information technology stakeholders should not only focus on interoperability between healthcare provider organizations, but should also recognize the importance of interoperability within a provider organization between the EHR and other health information technology.

Multiple choice question

When examining EHR interoperability with other health information technology what clinical process is associated with the most patient safety event reports?

• Radiology

• Laboratory

• Devices

• Medication

Correct Answer: D (Medication)

Explanation: Our analysis of patient safety event reports describing EHR interoperability with other health information technology systems and was analyzed by clinical process. The results show that EHR interoperability with pharmacy was the most frequently occurring safety reports followed by laboratory and radiology.

Conflict of Interest

The authors declare they have no conflicts of interest related to this study.

Human Subjects Protections

This study was approved by the Institutional Review Board (protocol #2014–101).

• References

• 1 Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals. 2008–2015. ONC Data Brief, no 35. 2016: 35.
• 2 HIMSS. HIMSS Dictionary of Healthcare Technology Terms, Acronyms, and Organizations. 3rd ed. Taylor & Francis; 2015
• 3 ONC. Connecting Health and Care for the Nation: A Shared Nationwide Interoperability Roadmap. 2014;1–77. Available from: http://www.healthit.gov/sites/default/files/nationwide-interoperability-road map-draft-version-1.0.pdf
• 4 Zhou Y, Ancker JS, Upadhye M, McGeorge NM, Guarrera TK, Hegde S, Crane PW, Fairbanks RJ, Bisantz AM, Kaushal R, Lin L. The impact of interoperability of electronic health records on ambulatory physician practices: a discrete-event simulation study. Inform Prim Care 2013; 21 (01) 21-29. Available from: http://dx.doi.org/10.14236/jhi.v21i1.36
• 5 Rosenbaum L. Transitional Chaos or Enduring Harm? The EHR and the Disruption of Medicine. N Engl J Med 2015; 373 (17) 1585-1588.
• 6 Obama B, Romney M. Health Care Reform and the Presidential Candidates. N Engl J Med 2012; 367 (15) 1377-1381. Available from: http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:New+engla+nd+journal#0 PMID: 20573919
• 7 Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The Value of Health Care Information Exchange and Interoperability. Heal Aff. 2005 Jan-Jun(Suppl Web Exclusives): W5–10-W15–18.
• 8 Magrabi F, Ong M-S, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Informatics Assoc 2013; 17 (06) 663-670.
• 9 Aspden P, Corrigan JW, Erickson SM. Patient Safety Reporting Systems and Applications. Patient Saf Achiev a new Stand care. Washington, D.C.: National Academy Press; 2004: 250-278.
• 10 Rosenthal J, Booth M. Maximizing the Use of State Adverse Event Data to Improve Patient Safety. Portlan, ME: 2005
• 11 Clarke JR. How a system for reporting medical errors can and cannot improve patient safety. Am Surg 2006; 72 (11) 1088– 91-48.
• 12 Boxwala AA, Dierks M, Keenan M, Jackson S, Hanscom R, Bates DW, Sato L. Organization and Representation of Patient Safety Data: Current Status and Issues around Generalizability and Scalability. JAMIA 2004; 11 (06) 468-478.
• 13 Magrabi F, Ong M-S, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Informatics Assoc 2012; 19 (01) 45-53.
• 14 Mardon R, Olinger L, Szekendi M, Williams T, Sparnon EM, Zimmer K. Health Information Technology Adverse Event Reporting: Analysis of Two Databases. 2014