Appl Clin Inform 2017; 08(03): 794-809
DOI: 10.4338/ACI-2016-12-RA-0210
Research Article
Schattauer GmbH

Comparison of EHR-based diagnosis documentation locations to a gold standard for risk stratification in patients with multiple chronic conditions

Shelby Martin
1  Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA
,
Jesse Wagner
1  Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA
,
Nicoleta Lupulescu-Mann
2  Center for Health Systems Effectiveness, Oregon Health & Science University, Portland, OR, USA
,
Katrina Ramsey
3  School of Public Health, Division of Biostatistics, Oregon Health & Science University, Portland, OR, USA
,
Aaron A. Cohen
1  Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA
,
Peter Graven
2  Center for Health Systems Effectiveness, Oregon Health & Science University, Portland, OR, USA
,
Nicole G. Weiskopf
1  Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA
,
David A. Dorr
1  Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA
› Author Affiliations
Funding The project described was supported by AHRQ grant number 1R21HS023091–01. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Agency for Healthcare Research and Quality.
Further Information

Publication History

21 December 2016

21 May 2017

Publication Date:
20 December 2017 (online)

Summary

Objective: To measure variation among four different Electronic Health Record (EHR) system documentation locations versus ‘gold standard’ manual chart review for risk stratification in patients with multiple chronic illnesses.

Methods: Adults seen in primary care with EHR evidence of at least one of 13 conditions were included. EHRs were manually reviewed to determine presence of active diagnoses, and risk scores were calculated using three different methodologies and five EHR documentation locations. Claims data were used to assess cost and utilization for the following year. Descriptive and diagnostic statistics were calculated for each EHR location. Criterion validity testing compared the gold standard verified diagnoses versus other EHR locations and risk scores in predicting future cost and utilization.

Results: Nine hundred patients had 2,179 probable diagnoses. About 70% of the diagnoses from the EHR were verified by gold standard. For a subset of patients having baseline and prediction year data (n=750), modeling showed that the gold standard was the best predictor of outcomes on average for a subset of patients that had these data. However, combining all data sources together had nearly equivalent performance for prediction as the gold standard.

Conclusions: EHR data locations were inaccurate 30% of the time, leading to improvement in overall modeling from a gold standard from chart review for individual diagnoses. However, the impact on identification of the highest risk patients was minor, and combining data from different EHR locations was equivalent to gold standard performance.

Martin S, Wagner J, Lupulescu-Mann N et al. Comparison of EHR-based diagnosis documentation locations to a gold standard for risk stratification in patients with multiple chronic conditions . Appl Clin Inform 2017; 8: 794–809 https://doi.org/10.4338/ACI-2016-12-RA-0210

Clinical Relevance Statement

This study is relevant to organizations and clinical teams engaged in examining their population of patients and to determine the best way to identify ongoing risks of adverse health outcomes and the resultant hospital stays and costs. It finds that using data from clinical information systems to summarize risks from common clinical diagnoses related to these outcomes leads to variable results in the presence of diagnoses but limited impact on risk prediction. This impact could also be addressed by combining all the potential locations of diagnoses, but the final estimations of risks are moderate and clinicians should be wary of their use in care.


Human Subject Research Approval

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by Oregon Health and Science University’s Institutional Review Board.