Download PDF
Methods Inf Med 2006; 45(03): 253-266
DOI: 10.1055/s-0038-1634079
Original Article
Schattauer GmbH

A Framework for Characterizing Terminological Systems

R. Cornet
1   Department of Medical Informatics, Academic Medical Center, Universiteit van Amsterdam, Amsterdam, The Netherlands
,
N. F. de Keizer
1   Department of Medical Informatics, Academic Medical Center, Universiteit van Amsterdam, Amsterdam, The Netherlands
,
A. Abu-Hanna
1   Department of Medical Informatics, Academic Medical Center, Universiteit van Amsterdam, Amsterdam, The Netherlands
› Author Affiliations
Further Information

Publication History

Publication Date:
06 February 2018 (online)

    Permissions and Reprints

Summary

Objectives: The notion of a terminological system (TS) is complex due to the broad range of systems, applications, and clinical domains. A uniform approach to describe the characteristics of TSs is lacking. This impedes furthering understanding, applicability, mutual comparison and development of TSs. For these reasons we propose a terminological systems characterization framework.

Methods: Relevant issues pertaining to TSs and terminology servers have been extracted from literature describing requirements and functionality of TSs. From these issues, features have been distilled and further refined. A categorization has been developed to provide a convenient arrangement of these features.

Results: The framework distinguishes between application-dependent and application-independent features of TSs. Definitions are provided for measures of content coverage, which was identified as the only application-dependent feature. Application-independent features are categorized along two axes: their respective type of TS and the particular element within that system, i.e. the formalism, the content, or the functionality. For each feature we provide an explicit question, the answer to which yields a feature value. The framework has been applied to SNOMED CT and the CLUE browser.

Conclusions: We present and apply a framework to support a feature-based characterization of terminological systems. Standardized methods for content coverage studies reduce the effort of assessing the applicability of a TS for a specific clinical setting. A two-axial categorization provides a convenient arrangement of the large number of application-independent features. Application of the framework increases comparability of terminological systems. This framework may also help TS developers determine how their system can be improved.

Keywords

Terminological systems - evaluation - methods - definitions
 

  • References

  • 1 de Keizer NF, Abu-Hanna A, Zwetsloot-Schonk JH. Understanding terminological systems. I: Terminology and typology. Methods Inf Med 2000; 39 (01) 16-21.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Rossi Mori A, Consorti F, Galeazzi E. Standards to support development of terminological systems for healthcare telematics. Methods Inf Med 1998; 37 (04) (05) 551-63.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 3 International Statistical Classification of Diseases and Related Health Problems, 1989 Revision Geneva: World Health Organization. 1992
    PubMedGoogle Scholar
  • 4 Rogers FB. Medical subject headings. Bulletin of the Medical Library Association 1963; 51: 114-6.
    PubMedGoogle Scholar
  • 5 Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug safety: an international journal of medical toxicology and drug experience 1999; 20 (02) 109-17.
    CrossrefPubMedGoogle Scholar
  • 6 McDonald CJ, Huff SM, Suico JG, Hill G, Leavelle D, Aller R. et al LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clinical Chemistry 2003; 49 (04) 624-33.
    CrossrefPubMedGoogle Scholar
  • 7 Coté RA, Rothwell DJ, Palotay JL, Beckett RS, Brochu L. editors SNOMED International: the systematized nomenclature of human and veterinary medicine. Vols I-IV. Northfield, IL: : College of American Pathologists; 1993
    Google Scholar
  • 8 Spackman K. SNOMED RT and SNOMED CT. Promise of an international clinical terminology. MD Computing 2000; 17 (06) 29
    PubMedGoogle Scholar
  • 9 Rector AL, Nowlan WA. The GALEN project. Computer Methods and Programs in Biomedicine 1994; 45 (01) (02) 75-8.
    CrossrefPubMedGoogle Scholar
  • 10 Harris MA, Clark J, Ireland A, Lomax J, Ashburner M, Foulger R. et al The Gene Ontology (GO) database and informatics resource. Nucleic Acids Research 2004; 32 (Database issue) D 258-61.
    CrossrefPubMedGoogle Scholar
  • 11 Rosse C, Mejino JLV. V. A reference ontology for biomedical informatics: the Foundational Model of Anatomy. Journal of Biomedical Informatics 2003; 36 (06) 478-500.
    CrossrefPubMedGoogle Scholar
  • 12 Schuyler PL, Hole WT, Tuttle MS, Sherertz DD. The UMLS Metathesaurus: representing different views of biomedical concepts. Bulletin of the Medical Library Association 1993; 81 (02) 217-22.
    PubMedGoogle Scholar
  • 13 Ingenerf J, Giere W. Concept-oriented standardization and statistics-oriented classification: continuing the classification versus nomenclature controversy. Methods Inf Med 1998; 37 (04) (05) 527-39.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 14 Cimino JJ. Desiderata for controlled medical vocabularies in the twenty-first century. Methods Inf Med 1998; 37 (04) (05) 394-403.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 ISO/TC215 WG 3 Standard Specification for Quality Indicators for Controlled Health Vocabularies; 2000 July. Report No.: TS17117
    PubMedGoogle Scholar
  • 16 Hales JW, Schoeffler KM. Barriers to Evaluation of Clinical Vocabularies. In Cesnik B, McCray AT, Scherrer JR. editors Medinfo1998: Proceedings of the 9th World Congress on Medical Informatics;1998; Seoul, Korea. Amsterdam, The Netherlands: IOS Press 1998; 680-4.
    PubMedGoogle Scholar
  • 17 NCVHS SSS. Summary and Analysis of Terminology Questionnaires Submitted by Developers of Candidate Terminologies for PMRI Standards. draft 2003; April 17
    PubMedGoogle Scholar
  • 18 OMG Lexicon Query Service Specification: Object Management Group; 2000 July. Report No.: 00-06-31.pdf
    PubMedGoogle Scholar
  • 19 Smith B. Beyond Concepts: Ontology as Reality Representation. In Varzi A, Vieu L. editors Proceedings of FOIS 2004. International Conference on Formal Ontology and Information Systems; 2004; Turin: Amsterdam, The Netherlands: IOS Press 2004; 73-84.
    PubMedGoogle Scholar
  • 20 Bodenreider O, Smith B, Burgun A. The Ontology- Epistemology Divide: A Case Study in Medical Terminology. In Varzi A, Vieu L. editors Proceedings of FOIS 2004. International Conference on Formal Ontology and Information Systems; 2004; Turin. Amsterdam, The Netherlands: IOS Press 2004; 185-95.
    PubMedGoogle Scholar
  • 21 Supekar K. A Peer-review Approach for Ontology Evaluation. In Noy NF. editor. 8th International Protégé Conference; July 18-21, 2005. Madrid, Spain 2005; 77-9.
    PubMedGoogle Scholar
  • 22 Supekar K, Patel C, Lee Y. Characterizing quality of knowledge on semantic web. In Barr V, Markov Z. editors Proceedings of the Seventeenth International FLAIRS Conference; May 2004 Miami, FL: AAAI Press; 2004: 220-8.
    Google Scholar
  • 23 Cornet R. Towards Structured Requirements for Terminological Systems and Servers. In Patel V, Rogers R, Haux R. editors Medinfo 2001: Proceedings of the 10th World Congress on Medical Informatics; 2001; London, UK: IOS Press, Amsterdam, The Netherlands 2001; 295
    PubMedGoogle Scholar
  • 24 CEN/TC251 Medical Informatics – Categorical structures of systems of concepts – Model for representation of semantics. European Prestandard. Brussels, Belgium: European Committee for Standardization; October 1997. Report No.: ENV 12264
    PubMedGoogle Scholar
  • 25 Guarino N. Formal Ontology in Information Systems. In Guarino N. editor Proceedings of FOIS’98; June 1998; Trento, Italy: IOS Press, Amsterdam, The Netherlands 1998; 3-15.
    PubMedGoogle Scholar
  • 26 Arts DGT, Cornet R, de Jonge E, de Keizer NF. Methods for Evaluation of Medical Terminological Systems. Methods Inf Med 2005; 44: 616-25.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 27 Bodenreider O, Burgun A, Botti G, Fieschi M, Le Beux P, Kohler F. Evaluation of the Unified Medical Language System as a medical knowledge source. Journal of the American Medical Informatics Association 1998; 5 (01) 76-87.
    CrossrefPubMedGoogle Scholar
  • 28 Bakken Henry S, Holzemer WL, Reilly CA, Campbell KE. Terms used by nurses to describe patient problems: can SNOMED III represent nursing concepts in the patient record?. Journal of the American Medical Informatics Association 1994; 1 (01) 61-74.
    CrossrefPubMedGoogle Scholar
  • 29 Bodenreider O, Mitchell JA, McCray AT. Evaluation of the UMLS as a terminology and knowledge resource for biomedical informatics. In Kohane IS. Proceedings of the 2002 AMIA Annual Symposium; 2002; San Antonio, TX, USA. Philadelphia, PA: Hanley and Belfus Inc., USA 2002; 61-5.
    PubMedGoogle Scholar
  • 30 Bowles KH. Application of the Omaha System in acute care. Research in Nursing & Health 2000; 23 (02) 93-105.
    CrossrefPubMedGoogle Scholar
  • 31 Campbell JR, Carpenter P, Sneiderman C, Cohn S, Chute CG, Warren J. Phase II Evaluation of Clinical Coding Schemes: Completeness, Taxonomy, Mapping, Definitions, and Clarity. Journal of the American Medical Informatics Association 1997; 4 (03) 238-51.
    CrossrefPubMedGoogle Scholar
  • 32 Chute CG, Cohn SP, Campbell KE, Oliver DE, Campbell JR. For The Computer-Based Patient Record Institute’s Work Group on Codes & Structures. The content coverage of clinical classifications. Journal of the American Medical Informatics Association 1996; 3 (03) 224-33.
    PubMedGoogle Scholar
  • 33 Cimino JJ, Patel VL, Kushniruk AW. Studying the human-computer-terminology interface. Journal of the American Medical Informatics Association 2001; 8 (02) 163-73.
    CrossrefPubMedGoogle Scholar
  • 34 Hardiker NR, Rector AL. Structural validation of nursing terminologies. Journal of the American Medical Informatics Association 2001; 8 (03) 212-21.
    CrossrefPubMedGoogle Scholar
  • 35 Humphreys BL, McCray AT, Cheh ML. Evaluating the coverage of controlled health data terminologies: report on the results of the NLM/AHCPR large scale vocabulary test. Journal of the American Medical Informatics Association 1997; 4 (06) 484-500.
    CrossrefPubMedGoogle Scholar
  • 36 Warnekar P, Carter J. HIV Terms Coverage by a Commercial Nomenclature. In Musen MA, Friedman CP, Teich JM. editors Proceedings of the 2003 AMIA Annual Symposium; 2003; Washington, DC, USA. Philadelphia, PA, USA: Hanley and Belfus Inc 2003; 1046
    PubMedGoogle Scholar
  • 37 Wasserman H, Wang J. An Applied Evaluation of SNOMED CT as a Clinical Vocabulary for the Computerized Diagnosis and Problem. List. In Musen MA, Friedman CP, Teich JM. editors Proceedings of the 2003 AMIA Annual Symposium; 2003; Washington, DC, USA. Philadelphia, PA, USA: Hanley and Belfus Inc 2003; 699-703.
    PubMedGoogle Scholar
  • 38 Arts D, De Keizer N, De Jonge E, Cornet R. Comparison of methods for evaluation of a medical terminological system. In Fieschi M, Coiera E, Li J. editors Proceedings from Medinfo 2004, San Francisco, CA, USA. Amsterdam The Netherlands: IOS Press; 2004: 467-71.
    Google Scholar
  • 39 Rector AL. Clinical terminology: why is it so hard?. Methods Inf Med 1999; 38 (04) (05) 239-52.
    Article in Thieme ConnectPubMedGoogle Scholar