Methods Inf Med
DOI: 10.1055/s-0039-3402069
Original Article
Georg Thieme Verlag KG Stuttgart · New York

Information Extraction from Echocardiography Reports for a Clinical Follow-up Study—Comparison of Extracted Variables Intended for General Use in a Data Warehouse with Those Intended Specifically for the Study

Mathias Kaspar
1  Comprehensive Heart Failure Center, Würzburg University Hospital, Würzburg, Germany
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Caroline Morbach
1  Comprehensive Heart Failure Center, Würzburg University Hospital, Würzburg, Germany
3  Department of Internal Medicine I, Würzburg University, Würzburg, Germany
,
Georg Fette
1  Comprehensive Heart Failure Center, Würzburg University Hospital, Würzburg, Germany
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Maximilian Ertl
4  Service Centre Medical Informatics, Würzburg University, Würzburg, Germany
,
Lea K. Seidlmayer
1  Comprehensive Heart Failure Center, Würzburg University Hospital, Würzburg, Germany
3  Department of Internal Medicine I, Würzburg University, Würzburg, Germany
,
Jonathan Krebs
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Georg Dietrich
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Leon Liman
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Frank Puppe
2  Chair for Artificial Intelligence and Applied Informatics, Würzburg University, Würzburg, Germany
,
Stefan Störk
1  Comprehensive Heart Failure Center, Würzburg University Hospital, Würzburg, Germany
3  Department of Internal Medicine I, Würzburg University, Würzburg, Germany
› Author Affiliations
Funding This work was supported by the German Ministry of Education and Research (BMBF), Berlin (#01EO1004, #01EO1504).
Further Information

Publication History

22 March 2019

12 November 2019

Publication Date:
30 January 2020 (online)

Abstract

Background The interest in information extraction from clinical reports for secondary data use is increasing. But experience with the productive use of information extraction processes over time is scarce. A clinical data warehouse has been in use at our university hospital for several years, which also provides an information extraction of echocardiography reports developed for general use.

Objectives This study aims to illustrate the difficulties encountered, while using data from a preexisting information extraction process for a large clinical study. To compare the data from the preexisting process with the data obtained from a specially developed process designed to improve the quality and completeness of the study data.

Methods We extracted the echocardiography variables for 440 patients from the general-use information extraction of the data warehouse (678 reports). Then we developed an information extraction process for the same variables but specifically for this study, with the aim to extract as much information as possible from the text. The extracted data of both processes were compared with a newly created gold standard defined by a cardiologist with long-standing experience in heart failure.

Results Among 57 echocardiography variables considered relevant for the study, 50 were documented in the routine text reports and could be extracted. Twenty of the required variables were not provided by the general-use extraction process, some others were not provided correctly. The median macro F1-score (precision, recall) across the 30 variables for which values were extracted was 0.81 (0.94, 0.77). Across all 50 variables, as relevant for the study, median macro F1-score was only 0.49 (0.56, 0.46). Employing the study-specific approach considerably improved the quality and completeness of the variables, resulting in F1-scores of 0.97 (0.98, 0.96) across all variables.

Conclusion Data from information extractions can be used for large clinical studies. However, preexisting information extraction processes should be treated with caution, as the time and effort spent defining each variable in the information extraction process may not be clear.