Download PDF
Methods Inf Med 2022; 61(01/02): 038-045
DOI: 10.1055/a-1817-7008
Original Article

A Methodological Approach to Validate Pneumonia Encounters from Radiology Reports Using Natural Language Processing

AlokSagar Panny
1   Center for Oral-Systemic Health, Marshfield Clinic Research Institute, Marshfield, Wisconsin, United States
,
Harshad Hegde
1   Center for Oral-Systemic Health, Marshfield Clinic Research Institute, Marshfield, Wisconsin, United States
,
Ingrid Glurich
1   Center for Oral-Systemic Health, Marshfield Clinic Research Institute, Marshfield, Wisconsin, United States
,
Frank A. Scannapieco
2   Department of Oral Biology, School of Dental Medicine, State University of New York at Buffalo, Buffalo, New York, United States
,
Jayanth G. Vedre
3   Department of Critical Care Medicine, Marshfield Clinic Health System, Marshfield, Wisconsin, United States
,
Jeffrey J. VanWormer
4   Center for Clinical Epidemiology and Population Health, Marshfield Clinic Research Institute, Marshfield, Wisconsin, United States
,
Jeffrey Miecznikowski
5   Department of Biostatistics, School of Public Health and Health Professions, State University of New York at Buffalo, Buffalo, New York, United States
,
Amit Acharya
1   Center for Oral-Systemic Health, Marshfield Clinic Research Institute, Marshfield, Wisconsin, United States
6   Advocate Aurora Research Institute, Advocate Aurora Health, Downers Grove, Illinois, United States
› Author Affiliations Funding Research reported in this publication was supported by the National Institute of Dental and Craniofacial Research of the National Institutes of Health under Award Number 1R03DE027020–01A1. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
› Further Information
    Permissions and Reprints

Abstract

Introduction Pneumonia is caused by microbes that establish an infectious process in the lungs. The gold standard for pneumonia diagnosis is radiologist-documented pneumonia-related features in radiology notes that are captured in electronic health records in an unstructured format.

Objective The study objective was to develop a methodological approach for assessing validity of a pneumonia diagnosis based on identifying presence or absence of key radiographic features in radiology reports with subsequent rendering of diagnostic decisions into a structured format.

Methods A pneumonia-specific natural language processing (NLP) pipeline was strategically developed applying Clinical Text Analysis and Knowledge Extraction System (cTAKES) to validate pneumonia diagnoses following development of a pneumonia feature–specific lexicon. Radiographic reports of study-eligible subjects identified by International Classification of Diseases (ICD) codes were parsed through the NLP pipeline. Classification rules were developed to assign each pneumonia episode into one of three categories: “positive,” “negative,” or “not classified: requires manual review” based on tagged concepts that support or refute diagnostic codes.

Results A total of 91,998 pneumonia episodes diagnosed in 65,904 patients were retrieved retrospectively. Approximately 89% (81,707/91,998) of the total pneumonia episodes were documented by 225,893 chest X-ray reports. NLP classified and validated 33% (26,800/81,707) of pneumonia episodes classified as “Pneumonia-positive,” 19% as (15401/81,707) as “Pneumonia-negative,” and 48% (39,209/81,707) as “episode classification pending further manual review.” NLP pipeline performance metrics included accuracy (76.3%), sensitivity (88%), and specificity (75%).

Conclusion The pneumonia-specific NLP pipeline exhibited good performance comparable to other pneumonia-specific NLP systems developed to date.

Keywords

pneumonia - natural language processing - knowledge bases

Supplementary Material



Publication History

Received: 17 August 2021

Accepted: 02 April 2022

Accepted Manuscript online:
05 April 2022

Article published online:
19 August 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Glurich I, Shimpi N, Scannapieco F, Vedre J, Acharya A. Interdisciplinary care model: pneumonia and oral health. In: Acharya A, Powell V, Torres-Urquidy M, Posteraro R, Thyvalikakath T. eds. Integration of Medical and Dental Care and Patient Data. 2nd ed. Cham: Springer; 2019: 123-139
    Google Scholar
  • 2 Franco J. Community-acquired pneumonia. Radiol Technol 2017; 88 (06) 621-636
    PubMedGoogle Scholar
  • 3 Franquet T. Imaging of community-acquired pneumonia. J Thorac Imaging 2018; 33 (05) 282-294
    CrossrefPubMedGoogle Scholar
  • 4 Drahos J, Vanwormer JJ, Greenlee RT, Landgren O, Koshiol J. Accuracy of ICD-9-CM codes in identifying infections of pneumonia and herpes simplex virus in administrative data. Ann Epidemiol 2013; 23 (05) 291-293
    CrossrefPubMedGoogle Scholar
  • 5 Dublin S, Baldwin E, Walker RL. et al. Natural Language Processing to identify pneumonia from radiology reports. Pharmacoepidemiol Drug Saf 2013; 22 (08) 834-841
    CrossrefPubMedGoogle Scholar
  • 6 Mendonça EA, Haas J, Shagina L, Larson E, Friedman C. Extracting information on pneumonia in infants using natural language processing of radiology reports. J Biomed Inform 2005; 38 (04) 314-321
    CrossrefPubMedGoogle Scholar
  • 7 Chapman WW, Fiszman M, Dowling JN, Chapman BE, Rindflesch TC. Identifying respiratory findings in emergency department reports for biosurveillance using MetaMap. Stud Health Technol Inform 2004; 107 (Pt 1): 487-491
    PubMedGoogle Scholar
  • 8 Elkin PL, Froehling D, Wahner-Roedler D. et al. NLP-based identification of pneumonia cases from free-text radiological reports. AMIA Annu Symp Proc 2008; 2008: 172-176
    PubMedGoogle Scholar
  • 9 Hegde H, Shimpi N, Glurich I, Acharya A. Tobacco use status from clinical notes using Natural Language Processing and rule based algorithm. Technol Health Care 2018; 26 (03) 445-456
    CrossrefPubMedGoogle Scholar
  • 10 Savova GK, Masanz JJ, Ogren PV. et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc 2010; 17 (05) 507-513
    CrossrefPubMedGoogle Scholar
  • 11 Hegde H, Shimpi N, Panny A, Glurich I, Christie P, Acharya A. Development of non-invasive diabetes risk prediction models as decision support tools designed for application in the dental clinical environment. Inform Med Unlocked 2019; 17: 100254
    CrossrefPubMedGoogle Scholar
  • 12 Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004; 32 (Database issue): D267-D270
    CrossrefPubMedGoogle Scholar
  • 13 Van Rossum G. Python reference manual. Amsterdam; January 1995. Accessed July 30, 2021 at: https://ir.cwi.nl/pub/5008
    PubMedGoogle Scholar
  • 14 Liu V, Clark MP, Mendoza M. et al. Automated identification of pneumonia in chest radiograph reports in critically ill patients. BMC Med Inform Decis Mak 2013; 13: 90
    CrossrefPubMedGoogle Scholar
  • 15 Hegde H, Glurich I, Panny A. et al. Identifying pneumonia sub-types from electronic health records using rule-based algorithms. Methods Inf Med 2022; (e-pub ahead of print). DOI: 10.1055/a-1801-2718.
    Article in Thieme ConnectPubMedGoogle Scholar