Methods Inf Med 2020; 59(06): 219-226
DOI: 10.1055/s-0041-1729951
Original Article

Accuracy of Asthma Computable Phenotypes to Identify Pediatric Asthma at an Academic Institution

Mindy K. Ross#
1  Department of Pediatrics, University of California Los Angeles, Los Angeles, California, United States
,
Henry Zheng#
2  Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, United States
,
Bing Zhu
2  Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, United States
,
Ailina Lao
3  University of California Los Angeles, Los Angeles, California, United States
,
Hyejin Hong
3  University of California Los Angeles, Los Angeles, California, United States
,
Alamelu Natesan
1  Department of Pediatrics, University of California Los Angeles, Los Angeles, California, United States
,
Melina Radparvar
1  Department of Pediatrics, University of California Los Angeles, Los Angeles, California, United States
,
Alex A.T. Bui
2  Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, United States
› Author Affiliations
Funding Javier Sanz and the UCLA Clinical and Translational Science Institute (CTSI) Data Core; Dr. Sleiman, CHOP. The authors would like to acknowledge the UCLA CTSI (UL TR001881) and the Institute for Precision Health (IPH) for their support in this project.

Abstract

Objectives Asthma is a heterogenous condition with significant diagnostic complexity, including variations in symptoms and temporal criteria. The disease can be difficult for clinicians to diagnose accurately. Properly identifying asthma patients from the electronic health record is consequently challenging as current algorithms (computable phenotypes) rely on diagnostic codes (e.g., International Classification of Disease, ICD) in addition to other criteria (e.g., inhaler medications)—but presume an accurate diagnosis. As such, there is no universally accepted or rigorously tested computable phenotype for asthma.

Methods We compared two established asthma computable phenotypes: the Chicago Area Patient-Outcomes Research Network (CAPriCORN) and Phenotype KnowledgeBase (PheKB). We established a large-scale, consensus gold standard (n = 1,365) from the University of California, Los Angeles Health System's clinical data warehouse for patients 5 to 17 years old. Results were manually reviewed and predictive performance (positive predictive value [PPV], sensitivity/specificity, F1-score) determined. We then examined the classification errors to gain insight for future algorithm optimizations.

Results As applied to our final cohort of 1,365 expert-defined gold standard patients, the CAPriCORN algorithms performed with a balanced PPV = 95.8% (95% CI: 94.4–97.2%), sensitivity = 85.7% (95% CI: 83.9–87.5%), and harmonized F1 = 90.4% (95% CI: 89.2–91.7%). The PheKB algorithm was performed with a balanced PPV = 83.1% (95% CI: 80.5–85.7%), sensitivity = 69.4% (95% CI: 66.3–72.5%), and F1 = 75.4% (95% CI: 73.1–77.8%). Four categories of errors were identified related to method limitations, disease definition, human error, and design implementation.

Conclusion The performance of the CAPriCORN and PheKB algorithms was lower than previously reported as applied to pediatric data (PPV = 97.7 and 96%, respectively). There is room to improve the performance of current methods, including targeted use of natural language processing and clinical feature engineering.

Ethical Considerations

A waiver of consent was obtained from the UCLA Institutional Review Board (IRB) committee (IRB #18–002015).


# Co-first authors.


Supplementary Material



Publication History

Received: 21 December 2020

Accepted: 15 April 2021

Publication Date:
14 July 2021 (online)

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany