Methods Inf Med 2018; 57(04): 168-176
DOI: 10.3414/ME17-01-0133
Original Articles
Georg Thieme Verlag KG Stuttgart · New York

Quality of the ICD-11 Beta Draft from the German Perspective: Evaluation Based on the Alphabet of ICD-10-GM 2017

Jürgen Stausberg
1  Institute for Medical Informatics, Biometry and Epidemiology (IMIBE), Faculty of Medicine, University Duisburg-Essen, Germany
› Author Affiliations
Funding This work was partially funded by the German Federal Ministry for Health under contract D105–01_06_ICD-11_TMF_GMDS.
Further Information

Publication History

received: 30 November 2017

accepted: 27 February 2018

Publication Date:
24 September 2018 (online)


Objectives: The German Association for Medical Informatics, Biometry and Epidemiology implemented a field test for the ICD-11 Beta Draft. Aim was to analyze completeness and appropriateness of the ICD-11 Beta Draft in its entire breadth.

Methods: Starting point was the synonym thesaurus (“Alphabet”) of the German modification of ICD-10. The Alphabet included a list of diagnoses terms that supports the coding of diagnoses with ICD-10. A sample of 60,328 diagnosis terms was drawn to be mapped to the ICD-11 Beta Draft. A subsample of 13,975 diagnosis terms was prepared for assessing reliability. First, the coders had to assign a diagnosis term from the sample to an appropriate English one. This included the automatic selection of the respective code from the ICD-11 Beta Draft. Secondly, the coders had to answer questions regarding completeness, appropriateness, and other issues.

Results: Finally, 49,184 results from 36 coders were available for the analysis. Problems with completeness were indicated in 4.7% of the results, problems with appropriateness in 5.3%. On the level of chapters, Cohen’s kappa reached grade “fair” at a maximum. The coders agreed in 31.4% of the terms.

Conclusions: Problems with the ICD-11 Beta Draft appeared to be moderate. Completeness was high, reliability was low as it is known for ICD-10. Concerns with the structure of the ICD-11 Beta Draft were noted, e. g. for neoplasms. A post processing of the ICD-11 Beta Draft seems to be sufficient with regard to the content. Methodologically, a thorough review of the structure might be advisable.