CC BY-NC-ND 4.0 · Appl Clin Inform 2018; 09(04): 752-771
DOI: 10.1055/s-0038-1670651
Review Article
Georg Thieme Verlag KG Stuttgart · New York

The Use of Health Information Exchange to Augment Patient Handoff in Long-Term Care: A Systematic Review

Clemens Scott Kruse
1  School of Health Administration, Texas State University, San Marcos, Texas, United States
,
Gabriella Marquez
1  School of Health Administration, Texas State University, San Marcos, Texas, United States
,
Daniel Nelson
1  School of Health Administration, Texas State University, San Marcos, Texas, United States
,
Olivia Palomares
1  School of Health Administration, Texas State University, San Marcos, Texas, United States
› Author Affiliations
Funding None.
Further Information

Address for correspondence

Clemens Scott Kruse, PhD, MHA, MSIT, MBA, FACHE
School of Health Administration, Texas State University
CHP Room 2050A, 601 University Drive, San Marcos, TX 78666
United States   

Publication History

22 April 2018

29 July 2018

Publication Date:
03 October 2018 (online)

 

Abstract

Background Legislation aimed at increasing the use of a health information exchange (HIE) in healthcare has excluded long-term care facilities, resulting in a vulnerable patient population that can benefit from the improvement of communication and reduction of waste.

Objective The purpose of this review is to provide a framework for future research by identifying themes in the long-term care information technology sector that could function to enable the adoption and use of HIE mechanisms for patient handoff between long-term care facilities and other levels of care to increase communication between providers, shorten length of stay, reduce 60-day readmissions, and increase patient safety.

Methods The authors conducted a systematic search of literature through CINAHL, PubMed, and Discovery Services for Texas A&M University Libraries. Search terms used were (“health information exchange” OR “healthcare information exchange” OR “HIE”) AND (“long term care” OR “long-term care” OR “nursing home” OR “nursing facility” OR “skilled nursing facility” OR “SNF” OR “residential care” OR “assisted living”). Articles were eligible for selection if they were published between 2010 and 2017, published in English, and published in academic journals. All articles were reviewed by all reviewers and literature not relevant to the research objective was excluded.

Results Researchers selected and reviewed 22 articles for common themes. Results concluded that the largest facilitator and barrier to the adoption of HIE mechanisms is workflow integration/augmentation and the organizational structure/culture, respectively. Other identified facilitator themes were enhanced communication, increased effectiveness of care, and patient safety. The additional barriers were missing/incomplete data, inefficiency, and market conditions.

Conclusion The long-term care industry has been left out of incentives from which the industry could have benefited tremendously. Organizations that are not utilizing health information technology mechanisms, such as electronic health records and HIEs, are at a disadvantage as insurers switch to capitated forms of payment that rely on reduced waste to generate a profit.


#

Background and Significance

Age is positively associated with chronic conditions and utilization of health resources. As 1.3 million of the population in America age, the chances of them contracting one or more chronic conditions increase as do the chances of some of them entering the 16,100 nursing homes in our nation, and some of these conditions are best managed through an electronic information-based record system.[1] [2] As the complexity of care increases beyond what a residential or community-based care facility can manage, a transition of care must occur to a higher level of care. During these transitions, communication must occur at multiple levels to ensure a smooth transition for both patient and receiving organization, smooth transfer of records between organizations, maximization of outcomes, a minimization of length of stay, and, if possible, a reduction of readmissions to care within 30 or 60 days.[3] [4] Inefficiencies during transfer can cause unnecessary rehospitalization as well as more complicated medical outcomes.[4] Care transitions are often complicated with lack of interoperability of record systems, high utilization at either end, and possibly the patient's reduced ability to participate in the communication portion of the transition of care.

Health information exchange (HIE) is both a noun and a verb: It is both the point of exchange and it also refers to the electronic sharing of health information across organizational boundaries.[5] The use of electronic HIE has modified how healthcare organizations view patient handoff.[1] HIE is a small subset of health information technology (HIT), which has gained large attention over the last decade through various efforts such as in the United States by its Office of the National Coordinator. Long-term care (LTC) organizations such as nursing homes, skilled nursing facilities (SNFs), and assisted living are laggards in the healthcare industry for HIT adoption, and the healthcare industry has been a laggard in IT adoption compared with other industries.[5] As such, there is a gap in the literature about this important topic. Because the U.S. healthcare industry has been a laggard in technological adoption, legislation aimed at increasing the rate at which organizations make use of electronic health records (EHRs) systems was enacted in 2009. The Health Information Technology for Economic and Clinical Health Act of 2009 (HITECH) allocated more than $560 million for states to develop and refine HIE capabilities within their boundaries.[6] None of the incentives outlined in legislation aimed at increasing interoperability were directed toward the approximately 16,100 nursing homes nationwide, and although these organizations face substantial barriers to even initial EHR adoption, states could have included LTC organizations in their HIE efforts, but very few did.[5]

Previous research briefly examined HIE. A recent cross-sectional analysis of secondary data of all U.S. acute-care hospitals reported that of the 1,991 hospitals reporting readmission data, 57.2% (1,139) of these facilities make some level of effort to exchange clinical data with LTC facilities.[3] Those that reported some exchange of data with LTC facilities were more than likely to report qualification for meaningful use (odds ratio [OR], 1.87; p = 0.01 for stage 1 and OR, 2.05; p < 0.01 for stage 2). While this study analyzed 1,991 organizations, it is unclear how many LTC facilities were involved at the other end of the exchange.

HIE can be used for fiscal reasons. Adopting HIT is part of a complicated attempt to manage the present unsustainable cost growth in healthcare.[7] In addition to the cost function, HIT is necessary for HIE which can play a part in patient handoff between levels of care. A lack of clear communication at patient handoff results in increased morbidity, costs, and hospital readmissions.[8] A barrier to HIE adoption is lack of interoperability, lack of funding, and lack of willingness on the part of organizations to change.[1] It is evident that there are multiple factors pushing for an increase in the interoperability of EHR systems: the adoption of HIT and the diffusion of HIE.

HIE can be used for safety reasons. The increase of nosocomial infections and readmissions at patient handoff has plagued organizations as ill, and even immunocompromised individuals are transferred between organizations. Health information exchanges have emerged as an information-based approach that increases the amount of information transferred at patient handoff.[9] Medicare no longer reimburses organizations for unplanned readmissions within 30 days for patients with acute myocardial infarction, congestive heart failure, or pneumonia.[10] This creates an urgent need for organizations to manage information efficiently to reduce the risk of patient readmissions at patient handoff.[9] Regardless, post-acute care providers as well as LTC organizations have the responsibility to ensure that the transfer of a patient is as risk free as possible.

A gap in the literature exists. The increased use of EHR systems serves as an impetus in the use of HIE. However, little is known about hospital-to-LTC utilization of HIE at a nationwide level.[8] Several organizations have taken initiative to increase the interorganizational transfer of information to produce successful patient handoffs. Organizations such as the Continuum of Care Improvement Through Information New York have set out to engage organizations and relay pertinent data to relevant stakeholders.[9] The state of New York has made the largest state-based investment for the improvement of EHR and HIE systems. These improvements serve as a part of the 2005 Healthcare Efficiency and Affordability Law for New Yorkers. The program has invested 440 million dollars in HIT mechanisms to reduce the cost of care for patients.[11] There is currently not a comprehensive record of all mechanisms nationwide which creates a gap in the literature. Further, some extensive research needs to be conducted across the nation to thoroughly explore the topic and publish best practices by those doing it well, particularly for the rest who have not implemented HIE.

The systematic literature review is the appropriate mechanism to fill this gap in the literature because it has been described as “the most reliable source of evidence to guide clinical practice,” it provides a complete overview and analysis of primary research directed toward one research end, and it is often required as the basis of funded research.[12] Systematic reviews often “include a broad range of relevant studies that have been undertaken and provide a detailed critical appraisal and synthesis of the individual studies.”[13] They are “used increasingly to inform medical decision making, plan future research agendas, and establish clinical policy[;] systematic reviews may strengthen the link between best research evidence and optimal health care.”[14] A systematic literature review is appropriate to set a foundation for an extensive study to establish best practices in preparation for the doubling of the population of adults aged 65 years and older will rapidly increase health care costs because they are more likely to use healthcare services including long-term care.

Objective

The purpose of this systematic review is to identify the HIE mechanisms currently in place in LTC institutions as well as recognize barriers to the adoption of a HIE that increases transparency between organizations that transfer patients during care for admission into higher level of care facilities. The LTC organizations under study are nursing homes, SNFs, residential care, and assisted living. The rationale behind this systematic review is to provide a framework for future research by identifying the themes prevalent in the literature. In addition, this research will attempt to define key terms and themes for use in future research.


#
#

Methods

Protocol, Eligibility Criteria, and Information Sources

Specific reporting and execution protocols were chosen for this review. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol[15] (see [Fig. 1]) and it was conducted using techniques from the Assessment for Multiple Systematic Reviews (AMSTAR) standard.[16] Literature gathered for this review was obtained from three separate databases: Cumulative Index of Nursing and Allied Health Administration Literature (CINAHL) Complete, PubMed (which queries MEDLINE), and Discovery Services for Texas A&M University Libraries. The reviewers combined key terms from the U.S. Library of Medicine's Medical Subject Headings (MeSH) with Boolean operators in all three databases to identify articles. Criteria focused on inpatient residential care organizations and the use (or nonuse) of a HIE. Analyzed literature covers a broad array of geographic regions as well as organizations at various levels of meaningful use standards. Search criteria emphasized the transition of care from acute care to a LTC organization.

Zoom Image
Fig. 1 PRISMA checklist.

#

Search and Study Selection

The search terms and applicable Boolean terms were as follows: (“health information exchange” OR “healthcare information exchange” OR “HIE”) AND (“long term care” OR “long-term care” OR “nursing home” OR “nursing facility” OR “skilled nursing facility” OR “SNF” OR “residential care” OR “assisted living”). These terms were deliberately chosen by examining the index list in the MeSH. Search parameters included English articles published from January 2010 to December 2017. This timeframe was selected because it was post-HITECH Act. While there were many HIE efforts prior to HITECH, the EHR adoption rates significantly increased afterward due to the meaningful-use incentives. Organizations with EHRs exchange clinical data through HIE two and a half times more than their non-EHR counterparts.[5] Therefore, it was imperative to observe HIE activity during this catalyst stage of EHR adoption. [Fig. 2] demonstrates the literature review search process that shows the inclusive and exclusion criteria.

Zoom Image
Fig. 2 Search criteria with inclusive and exclusion criteria.

The initial search resulted in 210 articles. We filtered the search for the last 7 years as well as full-text, English-only, and academic journals (to maintain quality of results). This removed all but 39 results. These 39 results were entered on piloted forms for consistency of review. Using a technique from AMSTAR, we assigned abstracts of these 39 articles to all group members in way that ensured each abstract was assessed for its germane nature by at least two reviewers. This process removed an additional 14 articles. Our final group for analysis was 22 articles.


#

Data Collection Process and Data Items

The authors reviewed each article in the same manner as the abstracts and determined the barriers and facilitators for the use of HIE to augment patient handoff in LTC. Piloted forms extracted similar data from each article: authors, year of publication, journal, country of publication, sample size, study design, signs of bias, limitations of study, facilitators for adoption, barriers to adoption, and a column for general comments. Facilitators were defined as characteristics or environmental factors that enabled the adoption and use of HIE between LTC and other levels of care. Barriers were defined as characteristics or environmental factors that serves as an obstacle or impediment to the adoption or use of HIE between LTC and other levels of care. The reviewers also examined potential bias and limitations of each article. Reviewers once again held a consensus meeting to discuss and compile observations as well as resolve any differences in observation.


#

Risk of Bias in Individual Studies and Across Studies

Reviewers recorded observed bias and potential limitations for each article analyzed as well as across all articles. These observations were recorded on the literature matrix along with the rest of the analysis. All results were recorded on a common table and compiled for discussion. [Appendix A] details bias and limitations of studies.

Zoom Image
Appendix A Kappa statistic calculation

#

Synthesis of Results and Additional Analysis

The final consensus meeting consisted of overall discussion of observations and inferences. Once all observations were compiled among all reviewers, narrative analysis and sense making were conducted to combine similar terms;[17] for example, one set of observations was “workflow integration” and another was “workflow augmentation.” Because both had to do with an action on workflows, we combined these into one theme. “Organizational structure” and “organizational culture” were also similar because they were both attributes of organizations. “Missing data” and “incomplete data” both described deficiencies of data, so they were combined. Finally, “privacy” and “security” were combined because these are often combined when discussing patient information. Based on the themes in the narrative analysis, affinity tables were created to synthesize themes for discussion.


#
#

Results

Study Selection

Our literature search narrowed 210 results down to 22 by assessing each for suitability to our research objective.[2] [3] [4] [5] [6] [7] [8] [9] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] We excluded 16 results because they were outside our target date range of 2010 to 2017. Another 157 results were excluded when we used the filters of English-only, full-text, and academic journals. The abstracts of the remaining 40 articles were assessed for suitability by multiple reviewers which reduced the final group to 22. A kappa statistic was calculated to assess the similarity in selection by reviewers.[31] It calculated to 0.99, which is near-perfect agreement.[32] See [Fig. 2] for the selection process. See [Appendix A] for detailed calculations of the kappa statistic.


#

Study Characteristics

From each study, reviewers extracted observations of facilitators, barriers, potential bias, and limitations. These are summarized in [Table 1]. We sorted the group of articles by date, most recent to oldest. There were three from 2017,[2] [3] [19] three from 2016,[4] [20] [21] six from 2015,[5] [18] [22] [23] [24] five from 2014,[5] [9] [26] [27] [28] three from 2013,[7] [8] [29] one from 2012,[30] and one from 2011.[11]

Table 1

Facilitators and barriers

Facilitators

Barriers

Ease of data transfer

Inefficiency

Reduce healthcare costs

Cost

Government funding

Low usage/Adoption

Adoption of EHR is likely to use HIE

Competing organizations

Abbreviations: EHR, electronic health record; HIE, health information exchange.



#

Risk of Bias within Studies

The biases observed within studies were selection bias,[3] [27] workplace bias,[2] self-report bias,[4] [26] [30] data collection bias,[5] [9] cognitive bias,[8] and nonresponse bias.[8] The bias observed did not appear strong enough to discount any of the articles in the group for analysis.


#

Results of Individual Studies

Articles analyzed identified a variety of facilitators and barriers, but common themes could be traced through the group. A detailed list of all observations and how they line up with themes can be seen in [Appendix B]. Facilitators are illustrated in [Table 2]. The asterisk (*) by the reference number indicates that the theme occurred within the article multiple times.

Table 2

Affinity matrix of facilitator themes in the literature

Facilitators

Theme

References

No. of occurrences

Organizational structure/culture

[3] [4] [18] [21] [25] [28] [29]

7

Workflow integration/augmentation

[6] [19] [*],[21] [27] [28]

6

Increase effectiveness of care

[2] [19] [22] [24] [27]

5

Enhance communication

[5] [9] [23] [27]

4

Adoption of EHR

[4] [5] [20] [28]

4

Proper funding

[4] [7] [11]

3

Patient safety

[25] [27]

2

Ease of data transfer

[2] [9]

2

Efficiency

[11] [16]

2

Market conditions

[13] [31]

2

Reduce healthcare cost

[10] [19]

2

39

Abbreviation: EHR, electronic health record.


* The asterisk by the reference number indicates that the theme occurred within the article multiple times.


Appendix B

Table of observations, themes, bias, and limitations

Authors

Facilitator observations

Themes

Barrier observations

Themes

Bias

Limitations

Cross and Adler-Milstein[3]

Policy initiatives have improved transitions between hospitals and LTC providers by incentivizing better coordination and promoting the adoption of IT and information sharing. Analysis of 2014 hospital survey data has indicated that the majority of hospitals routinely engage in electronic exchange

Organizational structure/culture

Self-assessments of HIE implementation may result in inaccurate results. Measures used for HIE implementation vary, so interpretation across boundaries is inaccurate

Missing/incomplete data

Self-report data are subject to bias such as social acceptability

The measure of hospital exchange was limited to SCR with LTC providers, and there may have been other types of exchange. This would tend to understate the applicability of the results of this study

Meehan and Staley[19]

Usage of HIE in LTCs has shown a reduction in readmission rates due to less transcription errors

Increase effectiveness of care

Because LTCs do not qualify for meaningful use incentives, they are less likely to personally invest into an EHR system of their own and hence health information exchange

Cost

None identified

Study limited to ACOs, so generalizability outside of the ACO is greatly limited

Faster and more accurate billing

Workflow integration/augmentation

The ACO already struggles with communication within its complex organizational structure; communicating outside the organization is a large barrier

Inefficiency

Overall improved quality of documentation and reduced costs to the patient associated with duplicate tests/orders the patient has recently had done in a previous office

Workflow integration/augmentation

Reduce healthcare cost

Meehan[2]

HIE can be used to expedite information during the transition of care from acute hospitals to the LTC

Efficiency

At best, acute care staff's inaccurate completion of the EHR

Missing/incomplete data

Workplace bias: staff interviewed had access to the HIE which partially justified their role at the long-term post-acute care facility

Small sample size (n = 20) from one location may limit the external validity of the results

This patient group especially requires management of multiple chronic illnesses that contribute to their overall plan of care

Increase effectiveness in the management of care

Results in an inefficient process (staff making follow-up phone calls to clarify on orders, tests)

Inefficiency

The systems provide a seamless exchange of patient data that ultimately improve quality of care and improve productivity. Many organizations are in the early stages of implementation and are focusing on the benefits of implementation of EHRs

Ease of data transfer

At worst, these gaps can adversely affect the patient and create a readmission

Organizational structure/culture

Workflow integration/augmentation

Jamoon et al[20]

This nationwide study by the U.S. DHHS reports on how many private practices have implemented EHR systems since ONC certified the systems as meeting the meaningful use criteria. The report states 15% of these practices are actively sharing information with LTCs

Adoption of EHR

None identified

None identified

None identified

Based on secondary data (National Ambulatory Medical Care Survey) which may not be the test data to answer the research question

Towne et al[4]

The location of an organization (rural or urban) was not found to be a limitation, with rural organizations actually having higher rates of adoption of EHRs and HIE technology

Adoption of EHR

Increasing number of individuals reaching the age in which LTC is required puts an additional strain on the already strained healthcare system

Organizational structure/culture

Self-report data may be subject to bias such as social responsibility

A cross-sectional study does not provide trends over time. Data were limited to the 2010 NSRCF dataset and compared with the 2012 data. Data were limited to the facility level, and therefore inferences could not be made to any other size organization. No data to enable inferences to rural health

Receptive leadership

Organizational structure (process)/culture

Overburdened staff

Inefficiency

Proper funding

Adequate funding

Cost

Cost

Alexander et al[21]

The research team found that assembling a network of providers that express the unique needs of the organization is critical for proper implementation of EHRs and HIEs in a nursing home or long-term care facility. HIE incorporated into existing workflows.Participation in the HIE both in and out of the facility

Workflow integration/augmentation

The proper implementation of EHRs, and much less HIEs, is contingent on successful staff training

Tech support

None identified

The leadership team for the organization was not present during the interviews.

One of the participating facilities had only two participants

Appropriate training and retraining

Tech support

High turnover in nursing homes and similar organizations negatively affects the implementation of these IT processes

Organizational structure/culture

Getting others to use the HIE

Market conditions

Getting the HIE operational

Tech support

Putting policies for technology in place

Organizational structure/culture

Filipova[6]

Quicker billing was identified as a facilitator to the implementation of EHRs and HIE in long-term care facilities. Improving the speed of tasks and functions is a benefit that may serve as an incentive when no government-provided incentive exists

Workflow integration/augmentation

Financial costs associated with the implementation of HIEs has limited organizations, to the point that some organizations do not even have the technical capacity for public health reporting purposes

Cost

None identified

None identified

Tech support

Nurses do not have the proper training

Lack of training

Yeaman et al[22]

The implementation of an HIE and HIT has resulted in a decrease in the number of readmissions following the study. This sheds light on the true benefits of HIE implementation

Increase effectiveness of care

The importance of an HIE cannot be overstated in LTC organizations because patients have much higher healthcare needs than other populations (comorbid conditions, take multiple medications, etc.). The biggest barrier experienced was a necessary cultural change

Organizational structure/culture

None identified

Small sample (convenience sample, n = 5) size brings into question whether it properly represents the population.Statistical analysis was restricted to descriptive only. As a result, even correlation would be difficult to justify.

Only chief complaints were examined for transfer and did not examine whether the complaints were related to an existing condition or new condition

Peters and Bunkers[23]

The coordination of chronic care has been identified as requiring the assistance of many individuals. Providing accurate information is critical to the implementation of a successful HIE

Enhance communication

Billing and CPT coding procedures must be properly understood by the provider and the patient. If this is not the case, the reported information may be incorrect and not of use to the medical benefit of the patient. Comorbidities exacerbate the possibility of confusion, a commonality among the geriatric population

Missing/incomplete data

Not a study

Hill et al[24]

HIEs have been demonstrated as efficient aids to physicians by allowing a more in-depth, long-term view into the patient's history

Efficiency

Currently, “unique” identifiers may pose issues if organizations overlap the format used as a unique identifier

Lack of data standards

Not a study

Increase effectiveness of care

Formats for storing medical records create silos that resist sharing

Missing/incomplete data

Alexander et al[18]

Researchers implemented process improvement measures in a geographic location with some of the highest rates of 30-d readmissions. The 16 organizations in the study were receptive to phased implementation of a HIE. The results of the implementation can be utilized for benchmarking purposes. This was an implementation study and authors were also part of the implementation team; therefore, there was a potential for cognitive bias

Organizational structure/culture

Network pitfalls can occur as a result of poor network planning or improper alignment with clinical workflow

Tech support

None identified

Additional technological and human resources are difficult to come by for every organization

Use cases were helpful

Market conditions

Every facility needed additional technological and human resources to build the HIE network

Organizational structure/culture

Hassol et al[25]

The introduction of an HIE to facilitate the exchange of information necessary to create continuity of care documents was well received among long-term care providers

Organizational structure/culture

IT constraints in the long-term care setting

Tech support

None identified

Only 29 of the 51 HIEs responded, which may limit generalizability to all HIEs

The proper exchange of information will result in safer transitions of patients into the long-term care setting

Patient safety

LTC organizations receive no incentive for the adoption of HIT initiatives

Cost

Abramson et al[5]

Organizations with an EHR were found to be 2.5 times more likely to participate in some form of HIE, although the most common HIE mechanism in place is in correspondence with pharmacies

Adoption of EHR

Lack of fiscal incentives for HIT adoption

Cost

Data collection bias because not all surveys were complete

Only conducted in New York City, so external validation cannot be assured. Not all surveys were completed, which made conclusions less than robust

Although there is no financial incentive for nursing home adoption, organizations have adopted due to the demonstrable benefits of improved communication

Enhance communication

Lack of interoperability with current systems

Lack of data standards

Initial cost of acquisition

Cost

Competing priorities

Organizational structure/culture

Ongoing cost of maintaining an EMR

Cost

Richardson et al[9]

Data transfer via HIE quickly aggregates patient record data from various HIE sources and distributes patient status info to the entire clinical team (not just one clinician)

Ease of data transfer

Interorganizational HIE data transfer. Lack of competing stakeholder buy-in

Organizational structure/culture

The study was performed within the researchers' own facility, which could bias the data itself

No ED physician interviews to augment data collection

Enhance communication

Misalignment with clinical workflows that inhibited use of HIE-based patient transfer data

Inefficiency

Campion et al[26]

The implementation of data sharing among organizations has been found to reduce healthcare costs

Reduce healthcare cost

Technological maturity

Market conditions

Self-report data may be subject to bias such as social responsibility.Participants were not provided a clear definition of barriers to adoption, so their responses may threaten construct validity

Small sample size brings into question whether it properly represents the population.

The data used in this study do not actually illustrate the sharing of patient data

Vendor participation

Market conditions

Privacy and security

Privacy and/or security

Regulatory requirements

Legal environment

Technical support

Tech support

Organizational structure

Organizational structure/culture

Workflow integration

Inefficiency

Data standards

Lack of data standards

Provider attitudes

Organizational structure/culture

Financial resources

Cost

Health plan participation

Market conditions

Lyngstad and Hellesø[27]

Lack of communication is a leading cause of error in healthcare

Enhance communication

Once implemented, a distinct limitation remains that EHRs contain retrospective patient data and caregivers need systems for means of prospective communication about the patient

Missing/incomplete data

Selection bias leaned toward more experienced workers, so external validity is limited

Only examined Norwegian organizations

Decrease adverse events

Increase effectiveness of care

Save time

Workflow integration/augmentation

Patient safety

Patient safety

Wang et al[28]

Hospital-based LTCs and/or nursing homes have a greater likelihood of adopting EHR systems into their practice (especially if in a rural location)

Adoption of EHR

Organizational barriers to EHR systems in LTCs include number of beds, urban or real location, hospital-based or freestanding, and for-profit or nonprofit

Market conditions

None identified

There is no unanimous agreement on measuring HIT adoption

Ultimately this will increase productivity among staff and providers

Workflow integration/augmentation

IT acceptance is influenced by performance expectancy, effort expectancy, social influence, and voluntariness

Organizational structure/culture

MacTaggart and Thorpe[7]

The report summarizes how LTCs can facilitate patient care using current technologies and provides measures for success utilizing government resources and their employees

Proper funding

Funding continues to serve as the main problem in implementing HIT tools at LTCs

Cost

Not a study

Kessler et al[8]

This very comprehensive review of over 200 articles on the elderly patients' transition from nursing homes to the ED gives a fairly accurate portrayal of current issues and also provides some recommendations as solutions

None identified

The article reinforces the common theme that a primary barrier of treating the elderly is a lack of communication between all facilities

Inefficiency

Cognitive bias and behavioral bias

None identified

Lack of training

Lack of training

Hamann and Bezboruah[29]

Analysis of an organization's tax status provides insight on HIE use rates, as well as rates of implementation of IT mechanisms. Nonprofit organizations are shown to be more likely to utilize EHRs than their for-profit counterparts

Organizational structure/culture

Privacy reasons can function to reduce the likelihood of statistically significant results

Privacy and/or security

None identified

Limited sample also limits the external validity

Regulatory requirements also function differently on a state-to-state basis and can skew the continuity of results

Legal environment

Wolf et al[30]

Organizations left out of adoption incentives require the greatest assistance in the adoption of technology and demonstrate a tremendous need for IT due to the population needs

Tech support

Current survey questions utilized do not properly assess the need of EHRs in LTC facilities. Survey questions should be modified to accurately depict the need

Missing/incomplete data

Nonresponse bias from ineligible hospitals (countered by weighting).

Self-report data may be subject to bias such as social responsibility

Secondary data were used which may not properly address the research question

Kern et al[11]

All facilities the authors reached out to participated in the survey. Also, they all received the same type of state funding toward EHR implementation (HEAL 1 grantees). The study assessed facilities based in New York, where there is the largest state-based investment (nationwide) of EHRs and HIE systems. New York can serve as a model and the study has national implications as the country is moving toward continued investment in HIT infrastructure

Proper funding

The sample size (26) is a major limitation, and the results were of borderline statistical significance. The study also questions only one type of grant recipient, although the state has various HIE grants. The results may vary if the study is readministered with all of the HEAL program grantees

Cost

None identified

Small sample size and results that were borderline significant. The latter was helped by large effect sizes. Only one source of funding was examined.

This study collected observations only after 1 y: Longer times are necessary to identify sustainability

Abbreviations: ACO, accountable care organization; ED, emergency department; EHR, electronic health record; HIE, health information exchange; HIT, health information technology; LTC, long-term care.


About 33% of the facilitators were captured with six themes. The themes organizational structure/culture [3] [4] [18] [21] [25] [28] [29] was observed seven times. These articles expressed issues such as the organization characteristics such as bed size, location, ownership, office or hospital based, system affiliation, or a flexible organization culture that enabled HIE. Accountable care organization incentive structures also enable HIE with LTC. The theme workflow integration/augmentation [6] [19]*, [21] [27] [28] was observed six times out of 39 occurrences. These articles described various aspects of workflow integration and or augmentation. External data are effective only if it can be integrated into the clinical workflow.[33] When an organization designs a workflow, it must take a human-systems approach, which is to say that the human is the embedded component of a system that supports people through a recognition of human capabilities, limitations, and performance needs. When the system is designed this way, outcomes are improved for patient, provider, and organization.[34] Some observed that adoption of HIE for patient transfer integrated with their existing workflows, and they observed that an organization with a supportive culture was conducive to a successful implementation. The asterisk (*) next to the reference number indicates that more than one facilitator was observed for that theme in the same article. For instance, one article pointed out that HIE integrated nicely with their existing workflows, and it improved or augmented their workflows as well. This was seen in both their billing cycle and their documentation.[19] Faster billing was observed multiple times and was captured under the same theme.[6] Articles described policy initiatives that incentivized better coordination of transfer and promoted information sharing.[6] [21] A receptive and supportive leadership that supports IT acceptance is influenced by performance expectancy, effort expectancy, social influence, and voluntariness.[12] [18] LTC facilities pointed out that HIE was necessary to create continuity-of-care documentation that was greatly appreciated by the receiving organizations.[25] It was also noted that nonprofit organizations were more likely to adopt HIE than their for-profit counterparts.[29]

HIE is attributed to improving the effectiveness of care.[2] [19] [22] [24] [27] Five articles discussed this attribute. The use of HIE was shown to decrease readmissions and decrease adverse events.[19] [22] [27] Others intimated that HIE allowed more in-depth, long-term views into a patient's history, particularly those with complex care.[2] [24] These five occurrences represented 13% of the total occurrences observed.

Four articles mentioned that HIE improved their communication both within and external to their own organization.[5] [9] [23] [27] HIE was named responsible for distributing patient status to the entire clinical team, and because the coordination of care involves many people, HIE has been adopted to ease the human resources burden to this task.[5] [23] Enhanced communication helps fill a gap where the lack of communication is a leading cause of error in healthcare.[27] The United States is not the only country to encourage HIE. In the United Kingdom, HIE is used to enhance communication as well.[35] These four occurrences represented 10% of all occurrences.

Adoption of the EHR [2] [9] was also observed four times, representing another 10% of the total observations. This observation noted that the adoption of HIE has been easier along with the adoption of the EHR. HIE was attributed for safer transitions of care, which is where errors most occur. Of private practices that responded to one survey, 15% of them were actively sharing information with LTC.[27] Proper funding was mentioned three times in the literature[4] [7] [11] representing 8% of all occurrences.

The last five themes were each mentioned twice in the literature, accounting for 25% of all occurrences. Patient safety [25] [27] was mentioned because organizations liked the reduction of duplicate testing, because a reduction in duplication increased communication, which may lead to fewer medical errors, and improved patient outcomes. The others were ease of data transfer,[2] [9] efficiency,[11] [16] market conditions,[13] [31] and reduce healthcare cost.[10] [19] The proper funding theme spoke of the cost of both EHR and HIE capabilities; some of these organizations were included in state efforts of HIE, and therefore they were able to afford the capability. Articles that highlighted the theme of ease of data transfer spoke of their perception that exchanging clinical data through HIE is easier than through fax or courier. Issues of efficiency were similar to issues of workflow, but they did not expressly use this term. We could have combined these into the workflow theme. The theme of market conditions spoke of concerns that resources spent on HIE were necessary because other organizations in their market were using HIE, and therefore to compete, they felt obligated to invest in the capability. Finally, the theme of reduce healthcare cost referred to the ongoing efforts to reduce the cost of care. Many organizations feel that a continued investment in IT will only increase the cost of care rather than realizing the cost saving and patient safety effects of HIE by preventing duplicate test, improving communication, saving time, and improving outcomes. The articles highlighted in this group realized the latter.

Five themes represented approximately 81% of all occurrences (39/48) of barriers. These are illustrated in [Table 3]. The asterisk (*) by the reference number indicates that the theme occurred within the article multiple times.

Table 3

Affinity matrix of barrier themes in the literature

Barriers

Theme

References

Occurrences

Cost

[5] [*],[4] [6] [7] [11] [19] [25] [26]

10

Organizational structure/culture

[2] [4] [5] [9] [18] [21] [22] [26] [*]

9

Missing/incomplete data

[2] [3] [23] [24] [27] [30]

6

Inefficiency

[2] [4] [8] [9] [19] [26]

6

Market conditions

[26] [*],[28]

4

Lack of data standards

[5] [24] [26]

3

Privacy and/or security

[26] [29]

2

Lack of training

[6] [8]

2

Legal environment

[26] [29]

2

44

* The asterisk by the reference number indicates that the theme occurred within the article multiple times.


Cost was identified as the second most often mentioned: 10 out of 44 occurrences or 23%.[5]*, [4] [6] [7] [11] [19] [25] [26] These articles pointed out the lack of incentives such as the meaningful use program[5] [19] [25] or state and local funding.[7] [11] Others pointed out that LTC organizations often do not have the budget for either the acquisition or upkeep costs that are required for such an IT implementation.[4] [5]

Organizational structure was observed in 9 of the 44 occurrences (20%).[2] [4] [5] [9] [18] [21] [22] [26]* We did combine organizational culture issues along with organizational structure. The organizations that were the subjects of these articles were not conducive to change or they did not embrace the new technology of HIE, which are cultural issues. Some articles highlighted that their organizations were not suited for even an EHR and the high turnover in LTC organizations is not conducive to an IT implementation like the adoption of HIE.[21] One article stated that the largest barrier to the adoption of HIE was a necessary cultural change.[22] Another mentioned that it did not have sufficient stakeholder buy-in.[9]

The next barrier mentioned most often was the missing, incomplete, or inaccurate data associated with HIE.[2] [3] [23] [24] [27] [30] This was found in 6 of the 44 occurrences, or 14%. These articles pointed out the importance of the correct billing codes for prompt reimbursement, and if the data are not accurate or complete in the transfer, then a delay or penalty will follow, and that comorbidities can complicate the codes.[23] One article mentioned that unique EHR systems that are not interoperable create data silos because they cannot share information.[24]

Six articles pointed out a perception of inefficiencies that HIE would cause their organization.[2] [4] [8] [9] [19] [26] These articles discussed a misalignment of workflows,[9] [26] a communication barrier,[5] [8] [19] and overburdened staff which would serve as barriers to their adoption of HIE.[31] These represented 14% of all occurrences of barriers listed in the literature.

The barrier of market conditions was identified four times in the literature, representing 9% of all occurrences.[26]*,[28] These articles expressed concern about the technological maturity on HIE, and the level of both vendor and health plan participation in HIE. They felt that there was insufficient saturation of HIE in their market to compel them toward the technology.

The other four themes appeared in the literature 20% of what was analyzed. These themes were lack of data standards,[5] [24] [26] which refers to over 300 vendors of EHRs and no one common national solution to sharing data; privacy/security concerns,[26] [29] which highlighted the constant fear that some organizations can operate with in regard to the security of clinical data and potential fines from both state and federal government; lack of training,[6] [8] which refers to constant change of technology and how the training for these changes is often the responsibility of the organization; and legal environment,[26] [29] which parallels the issue of privacy/security concerns because often part of the remedy for data breach is credit monitoring and other monetary damages and time in court.


#

Additional Analysis

[Tables 2] and [3] illustrate the identified themes throughout the literature for both facilitators and barriers to the adoption of HIE to support patient handoff between levels of care. We also evaluated these themes in terms of internal versus external factors, as depicted in [Tables 4] and [5]. These themes are listed in the same order as previously listed: most often mentioned to least often mentioned. As depicted, the facilitators list only 2 of 11 themes (18%) as external to the organization. That is to say, only two items listed are beyond the control of the organization: proper funding, mentioned 7% of all occurrences, and market conditions, mentioned 5% of all occurrences. The barriers, however, show five of nine themes (55%) as external. These themes were cost, market conditions, lack of data standards, privacy/security, and the legal environment, mentioned 21, 8, 6, 4, and 2% of all occurrences, respectively.

Table 4

Facilitator themes with internal/external association

Facilitators

Theme

Workflow integration/augmentation

Internal

Organizational structure/culture

Internal

Enhance communication

Internal

Increase effectiveness of care

Internal

Patient safety

Internal

Adoption of EHR

Internal

Proper funding

External

Ease of data transfer

Internal

Efficiency

Internal

Market conditions

External

Reduce healthcare cost

Internal

Abbreviation: EHR, electronic health record.


Table 5

Barrier themes with internal/external association

Barriers

Theme

Organizational structure/culture

Internal

Cost

External

Missing/incomplete data

Internal

Inefficiency

Internal

Market conditions

External

Lack of data standards

External

Privacy and/or security

External

Lack of training

Internal

Legal environment

External

A brief analysis of the quality of methodology and overall study design can be found in [Appendix C]. This appendix was created as part of the piloted forms, but it did not add significant content for the review.

Appendix C

Detail of study design, quality, care setting, and critique of study

Authors

Study design

0 = Qualitative

1 = Quantitative

2 = Mixed

3 = Review

4 = Other

Quality

Critique of study

Care setting

Cross and Adler-Milstein[3]

Cross-sectional, quantitative study; secondary data analysis

1

High-quality, large sample (n = 1,991)

Confounders not considered for effects attributed to HIE. Number of LTC facilities at the other end of HIE communication not identified. Region of country not identified

Hospitals and long-term care facilities

Meehan and Staley[19]

Review

2

Moderate-quality review of several good-quality studies (N = 11)

Narrow focus of the study makes its conclusions difficult to use outside ACOs. Study limited to ACOs, so generalizability outside of the ACO is greatly limited

Outpatient

Meehan[2]

Qualitative, interview-based study

0

Moderate-quality, small sample size for social sciences (n = 20)

Were staff members interviewed the right people to know? Were confounding factors properly considered for effects queried about? ACOs may not be old enough to really know proper cause and effect relationships

Long-term post-acute care

Jamoon et al[20]

Data brief

4

High-quality data brief summarizing EHR adoption rates nationwide 2013 to 2014

None

office-based care

Towne et al[4]

Cross-sectional, quantitative, using secondary data

1

High-quality, large sample (n = 2,302).

Self-report data were not followed up with telephone calls or emails to question or extend the data

Residential-care facilities

Alexander et al[21]

Qualitative, interview-based study

0

Moderate-quality, small sample for social sciences (n = 15) of nursing home leaders

Limited scope of study prevents applying results external to the small number of organizations interviewed. Level of experience of the leaders interviewed was not normalized for comparison

Nursing homes

Filipova[6]

Qualitative, interview-based study

0

High-quality study, good sample size (n = 156)[a]

Cofounding factors for findings not explored

skilled nursing facilities

Yeaman et al[22]

Retrospective, quantitative design

1

Marginal quality with a very small sample for social sciences (n = 5)

Confounding factor of readmission rates decreasing across the nation was not explored. Was the effect reported attributable to the use of HIE or to natural national trend? The researchers noted that it is doubtful that their findings could be duplicated, which sheds doubt on the scientific nature of this study

Long-term care and acute-care settings

Peters and Bunkers[23]

Editorial

4

N/A

Chronic care management, office-based and home-based care

Hill et al[24]

Editorial

4

N/A

Hospital-based, office-based

Alexander et al[18]

Mixed method

2

Good-quality, moderate-sized sample for social sciences (n = 320), clinical observation, use cases, and semistructured interviews with staff of 16 nursing homes

Researchers did not follow up surveys with calls, nor did they attempt to contact those HIEs that did not respond. Valuable information could have been gathered from these other means of communication

Nursing homes

Hassol et al[25]

Mixed method

2

Good-quality, moderate-sized sample for social sciences (n = 27), survey asked several questions to HIEs across the nation

Researchers did not follow up surveys with calls, nor did they attempt to contact those HIEs that did not respond. Valuable information could have been gathered from these other means of communication

Long-term acute-care settings

Abramson et al[5]

Cross-sectional, mixed methods

2

Good-quality, moderate-sized sample for social sciences (n = 375), clinical observation, use cases, and semistructured interviews with staff of 16 nursing homes

Researchers limited their study to the state of New York, which makes it difficult to apply their results elsewhere. It would have been helpful if the researchers had contacted the nursing homes that did not respond to the survey to promote their participation

Nursing homes

Richardson et al[9]

Semistructured telephone and in-person interviews with informaticians, healthcare administrators, software engineers, and providers

0

Good-quality, moderate-sized sample for social sciences (n = 18), of providers and administrators of a SNF

Research was conducted with inherent bias because it was within the researcher's own facility

Skilled nursing facility

Campion et al[26]

Cross-sectional retrospective study using secondary data

1

Moderate quality with a small sample size for social sciences (n = 8) to evaluate data from HIEs in New York

This is another study that was limited to the state of New York. It would have been better if the study would have extended beyond one state so that results could have been more widely applied

None

Lyngstad and Hellesø[27]

Cross-sectional, mixed methods study

2

Good quality with large sample (n = 1,075) interviewed both home-health care nurses and general practitioners

Factors not considered were personal preferences for communication and generational differences in acceptance of technology

Home health

Wang et al[28]

Cross-sectional, mixed methods study

2

Good quality, good sample (n = 136)

This study design seemed to be driven more by budget than by intention

Long-term care

MacTaggart and Thorpe[7]

Editorial

3

N/A

Long-term care and post-acute care settings

Kessler et al[8]

Review

3

High-quality review (n = 200)

N/A

Geriatric care transitions to ED

Hamann and Bezboruah[29]

Retrospective, mixed design, using secondary data

2

High-quality using good sample size (n = 1,174) using a stratified, multiphase sampling technique for nursing homes across the country

None

Nursing homes

Wolf et al[30]

Retrospective, quantitative design, using secondary data

1

High-quality using large sample (n = 3,653)

The dataset from the AHA could have been augmented with the dataset from HIMSS to corroborate trends and findings

Long-term acute care (n = 144), rehabilitation (n = 108), psychiatric (n = 240), short-term acute care (n = 3,161)

Kern et al[11]

Longitudinal cohort study of community-based organizations fu

2

Moderate-quality, small sample size for social sciences (n = 26)

None

Inpatient and outpatient. Surveys included all participating HIE organizations in New York state

Abbreviations: ACO, accountable care organization; AHA, American Hospital Association; EHR, electronic health record; HIE, health information exchange; HIMSS, Healthcare Information and Management Systems Society; HIT, health information technology; LTC, long-term care.


a A caution should be noted that in qualitative research, more is not necessarily better because a larger sample might indicate a lack of depth of interview.



#
#

Discussion

Summary of Evidence

Our literature search identified 26 articles that were pertinent to our research objective. From these 26 articles, we identified facilitators and barriers identified in the literature. Several of the facilitators and barriers were similar; so, we created themes to capture the essence of the details identified by the literature. We identified 11 themes for facilitators and 9 themes for barriers each listed 39 and 44 times, respectively.

Healthcare leaders should recognize the top facilitators and leverage them for future implementation. These leaders have control over a strong majority of the themes identified. Create an organizational culture that is conducive to the adoption of HIE recognizing that others have easily integrated it into their existing workflows. Leaders often look for ways to increase communication both within and external to the organization, and several articles identified HIE as a way to augment both. Leaders should also recognize that an HIE implementation will take time and should emphasize both process improvement and training of users before undertaking it.

Policy makers should examine the barriers identified in this review. LTC should be included in future financial-incentive programs to help offset the cost of implementation. Policy makers should also help set market conditions that are most conducive to HIE adoption and continue to develop clear data standards that protect privacy and promote security for the industry.

The reviewed articles indicated a primarily optimistic approach to the future implementation of HIEs in patient handoff. A pilot study implementing a form of HIE between five Oklahoma LTC facilities and their corresponding emergency departments indicated a decrease in the number of readmissions throughout the duration of the 20-month study.[22] This decrease correlates with the shift in the U.S. healthcare system to a patient-centered, value-based approach that emphasizes HIT as a tool to improve care.[22]

The overall health of the patient population admitted into LTC facilities must be considered as well. Elderly patients are more likely than the rest of the population to take multiple prescriptions, have comorbidities, and see a multitude of providers.[22] With the multiple opportunities for error that exist because of the health of the geriatric population, organizations and patients alike can benefit from the accurate transfer of information.

Nearly one in four individuals in the United States suffers from multiple conditions, while one-half of the population suffers from a chronic condition. The percentage of the population who suffers from at least one chronic condition increases to 75% among older adults.[5] The successful management of these conditions is contingent on the collaboration of multiple stakeholders throughout the course of patient care.[23] Thus, comprehensive and accurate information is required. A study of 20 clinical staff from a U.S. Midwestern long-term post-acute care facility (LTPAC) found that providing LTPAC staff with a more in-depth view of a patient's condition aids the course of care beyond the acute-care setting. Medication errors such as the erroneous continuation of an antibiotic can be prevented if proper information is available.[5] In many countries, the LTPAC covers services such as LTC hospitals, nursing homes, SNFs, and residential care.[36]

The end goal of the accurate and efficient transfer of information through an HIE must be articulated as a need for LTC facilities. The concept for an HIE comes in many forms and requires the cooperation of organizational stakeholders. Health information exchanges can aid in the treatment and billing of chronic care coordination. The proper course of action for chronic care coordination requires open communication between pertinent providers to ensure that conditions are properly treated.[23] For now, many LTC facilities are focused on the proper implementation of EHRs, a concept that the rest of the healthcare world has long since implemented due to the HITECH Act of 2009. LTC facilities that have implemented EHRs utilize HIE mechanisms at a rate of 2.5 times more than their non-EHR counterparts.[5] Among a cross-sectional study across all New York State nursing homes, most information was exchanged with pharmacies and laboratories. The state of New York has invested a considerable amount of time on both research and implementation of HIE mechanisms in nursing homes and the results are encouraging and illustrate the need for initiatives to drive other organizations to invest in HIEs.[5]

For LTC facilities, there are barriers to the implementation of HIEs aside from the complete lack of financial incentives. The integration of software among organizations is a prominent issue, as well as differing information needs.[5] Compensation for the coordination of chronic care among the geriatric population may not suffice if the cost of generating billing and documentation exceeds the capitated reimbursement for the treatment of the condition. In this regard, HIEs can reduce unnecessary documentation and tests by improving communication among providers and reducing waste.[23] Healthcare insurers, including the federal and state governments who pay for a majority of healthcare for older adults, must reduce waste to make a profit.

If organizations do not receive the proper and necessary information, the HIE is only an expense and not a benefit. It is imperative that LTC facilities, which can most benefit from the smooth transfer of information, implement HIE mechanisms to stop the unnecessary harm caused by the poor transfer of information.


#

Limitations

Limitations stemmed from the nature of current studies, which assess LTC implementation of EHRs and HIEs in general. Because the implementation of HIT in LTC organizations is low compared with other organizations, studies that assess the true notion of patient handoff from acute care to LTC facilities or from LTC facilities to higher levels of care are limited and premature in their scope.

While selection bias is a large threat to any study, we controlled for that threat by adopting techniques from AMSTAR. We ensured multiple reviewers independently evaluated articles and held regular consensus meetings to keep everyone calibrated on the purpose and scope of this review.

One of the filters used in the search for articles was “free text” which eliminated seven articles from consideration. This is a limitation because the impact of those seven articles on the outcome of our systematic literature review is unknown. Additional funding would enable access to these articles and if findings are significantly different, a study should be republished.

Another limitation is that this systematic literature review fell short of a meta-analysis. A sufficient level of original data from the studies was not collected. The science behind the studies analyzed was not sufficiently robust. Our time was short for this round of research, so we held short at the systematic review.


#
#

Conclusion

Health information exchanges are imperative to the future of care in the LTC sector. With the growth of the aging population, organizations must be ready to provide accurate quality care to ensure the well-being of patients. The benefits of an HIE cannot be underestimated, given that the elderly population presents more comorbidities and chronic condition than the rest of the population. Current barriers to the proper implementation of an HIE include network pitfalls and competing stakeholder needs, as well as the financial component of implementing a useable HIE.

The status of EHR implementation in LTC facilities varies throughout the country with HIEs lagging even further behind.[37] Significant research has been conducted on hospital readmissions from the LTC setting, but not enough research has been conducted on successful mechanisms to prevent readmissions.[38] Future research should focus on the need to increase financial incentives for LTC facilities to implement HIE mechanisms, as well as the actuarial effect of readmissions on LTC facilities as financial penalties become more stringent. This systematic review serves as a good foundation for extensive research to establish best practices.


#

Multiple Choice Questions

  1. The HITECH Act applied incentives for the adoption of HIT for which of the following organizations?

    • Acute-care hospitals.

    • All size of physician practices.

    • Long-term care.

    • a and b only.

    Correct Answer: The correct answer is option d.

  2. Which is the largest barrier to the adoption of HIT in LTC organizations?

    • Cost.

    • Privacy/security.

    • Organizational culture.

    • Market conditions.

    Correct Answer: The correct answer is option a.


#
#

Conflict of Interest

None.

Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was not reviewed by Institutional Review Board because it is exempted, IAW 45CFR46.



Address for correspondence

Clemens Scott Kruse, PhD, MHA, MSIT, MBA, FACHE
School of Health Administration, Texas State University
CHP Room 2050A, 601 University Drive, San Marcos, TX 78666
United States   


Zoom Image
Fig. 1 PRISMA checklist.
Zoom Image
Fig. 2 Search criteria with inclusive and exclusion criteria.
Zoom Image
Appendix A Kappa statistic calculation