Appl Clin Inform 2016; 07(04): 1051-1068
DOI: 10.4338/ACI-2016-08-RA-0129
Research Article
Schattauer GmbH

Identification of Long Bone Fractures in Radiology Reports Using Natural Language Processing to support Healthcare Quality Improvement

Robert W. Grundmeier
1  Department of Biomedical and Health Informatics, The Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
2  Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, United States
,
Aaron J. Masino
1  Department of Biomedical and Health Informatics, The Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
,
T. Charles Casper
3  Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, Utah, United States
,
Jonathan M. Dean
3  Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, Utah, United States
,
Jamie Bell
3  Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, Utah, United States
,
Rene Enriquez
3  Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, Utah, United States
,
Sara Deakyne
4  Children’s Hospital Colorado, Denver, Colorado, United States
,
James M. Chamberlain
5  Division of Emergency Medicine, Children’s National Health System, Washington, District of Columbia, United States
,
Elizabeth R. Alpern
6  Department of Pediatrics, Ann and Robert H. Lurie Children’s Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, Illinois, United States
,
The Pediatric Emergency Care Applied Research Network› Author Affiliations
Funding This project work was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS020270. The PECARN infrastructure was supported by the Health Resources and Services Administration (HRSA), the Maternal and Child Health Bureau (MCHB), and the Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.
Further Information

Correspondence to:

Robert W. Grundmeier, MD
The Children’s Hospital of Philadelphia
3535 Market Street
Suite 1024
Philadelphia
PA 19104
Phone: 215-590-2857   

Publication History

received: 01 August 2016

accepted: 26 September 2016

Publication Date:
18 December 2017 (online)

 

Summary

Background Important information to support healthcare quality improvement is often recorded in free text documents such as radiology reports. Natural language processing (NLP) methods may help extract this information, but these methods have rarely been applied outside the research laboratories where they were developed.

Objective To implement and validate NLP tools to identify long bone fractures for pediatric emergency medicine quality improvement.

Methods Using freely available statistical software packages, we implemented NLP methods to identify long bone fractures from radiology reports. A sample of 1,000 radiology reports was used to construct three candidate classification models. A test set of 500 reports was used to validate the model performance. Blinded manual review of radiology reports by two independent physicians provided the reference standard. Each radiology report was segmented and word stem and bigram features were constructed. Common English “stop words” and rare features were excluded. We used 10-fold cross-validation to select optimal configuration parameters for each model. Accuracy, recall, precision and the F1 score were calculated. The final model was compared to the use of diagnosis codes for the identification of patients with long bone fractures.

Results There were 329 unique word stems and 344 bigrams in the training documents. A support vector machine classifier with Gaussian kernel performed best on the test set with accuracy=0.958, recall=0.969, precision=0.940, and F1 score=0.954. Optimal parameters for this model were cost=4 and gamma=0.005. The three classification models that we tested all performed better than diagnosis codes in terms of accuracy, precision, and F1 score (diagnosis code accuracy=0.932, recall=0.960, precision=0.896, and F1 score=0.927).

Conclusions NLP methods using a corpus of 1,000 training documents accurately identified acute long bone fractures from radiology reports. Strategic use of straightforward NLP methods, implemented with freely available software, offers quality improvement teams new opportunities to extract information from narrative documents.

Citation: Grundmeier RW, Masino AJ, Casper TC, Dean JM, Bell J, Enriquez R, Deakyne S, Chamberlain JM, Alpern ER. Identification of long bone fractures in radiology reports using natural language processing to support healthcare quality improvement.


#

 


#

Conflicts of Interest

The authors have no conflicts of interest relevant to the work described in this article.


Correspondence to:

Robert W. Grundmeier, MD
The Children’s Hospital of Philadelphia
3535 Market Street
Suite 1024
Philadelphia
PA 19104
Phone: 215-590-2857