PDF herunterladen
Methods Inf Med 2000; 39(04/05): 311-318
DOI: 10.1055/s-0038-1634398
Original Article
Schattauer GmbH

Validation of ICTERUS, a Knowledge-based Expert System for Jaundice Diagnosis

G. Molino
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
M. Marzuoli
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
F. Molino
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
S. Battista
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
F. Bar
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
M. Torchio
1   Laboratory of Clinical Informatics, San Giovanni Battista Hospital of Turin, Turin, Italy
,
S. M. Lavelle
2   Department of Experimental Medicine, Galway University, Ireland
,
G. Corless
2   Department of Experimental Medicine, Galway University, Ireland
,
N. Cappello
3   Department of Genetics, Biology and Biochemistry, University of Turin, Turin, Italy
› Institutsangaben
Weitere Informationen

Publikationsverlauf

Publikationsdatum:
08. Februar 2018 (online)

    Lizenzen und Reprints

Abstract:

The study aimed to describe an example of the assessment and validation of knowledge-based clinical expert systems. The paper focuses on ICTERUS, an expert system for jaundice diagnosis. It describes system design, the methodology applied for upgrading and validating the program, and the most important outcomes of the validation procedure. The clinical validation of the system on a very large European database (Euricterus Project) shows that diagnostic conclusions are reliable in about 70% of eligible cases. This figure appears acceptable for a system which provides decision support only on the basis of clinical data, assuming that the final decision is achieved under user responsibility. Expected biases, limitations and inconsistencies in the practical application of the system are discussed.

Keywords

Jaundice - Liver Diseases - Knowledge-based Expert System - System Validation - Diagnostic Accuracy
 

  • REFERENCES

  • 1 Chandrasekeran B. On evaluating AI systems for medical diagnosis. AI Magazine 1983; 4: 34-7.
    PubMedSuche in Google Scholar
  • 2 Foster D, Buchanan B. Broadening our approach to evaluating medical information systems. Proc 15th SCAMC, Washington DC: 1991: 8-12.
    Suche in Google Scholar
  • 3 Heathfield HA, Wyatt J. Philosophies for the design and development of clinical decision-support systems. Method Inform Med 1993; 32: 1-8.
    PubMedSuche in Google Scholar
  • 4 Hilden J, Habbema DF. Evaluation of clinical decision-aids: more to think about. Med Inform 1990; 15: 275-84.
    PubMedSuche in Google Scholar
  • 5 Miller PL, Sitting DF. The evaluation of clinical decision support systems: what is necessary versus what is interesting. Med Inform 1990; 15: 185-90.
    PubMedSuche in Google Scholar
  • 6 Nguyen T, Perkins W, Laffey T. et al. Knowledge base verification. AI Magazine Summer 1987; 8: 69-75.
    PubMedSuche in Google Scholar
  • 7 O’Neil M, Glowinski A. Evaluating and validating very large knowledge-based systems. Med Inform 1990; 15: 237-52.
    PubMedSuche in Google Scholar
  • 8 Perrault L, Wiederhold G. System design and evaluation. In: Shortliffe et al (Eds). Med Inform. 1990: 151-77.
    Suche in Google Scholar
  • 9 Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240: 1285-93.
    CrossrefPubMedSuche in Google Scholar
  • 10 Friedman C, Wyatt J. Evaluation methods in medical informatics. New York: Springer Verlag; 1997
    Suche in Google Scholar
  • 11 Van der Loo RP. Overview of published assessment and evaluation studies. In: Assessment and Evaluation of Information Technologies in Medicine. van Gennip EMSJ, and Talmon JL. Eds Amsterdam: IOS Press; 1995: 261-81.
    Suche in Google Scholar
  • 12 Talmon JL, and Van der Loo RP. Literature on assessment of information technology and medical KBS evaluation: studies and methodologies. In: Assessment and Evaluation of Information Technologies in Medicine. van Gennip EMSJ, and Talmon JL. Eds. Amsterdam: IOS Press; 1995: 283-327.
    Suche in Google Scholar
  • 13 Molino G, Molino F. Assessment and evaluation of knowledge-based expert systems for medical diagnosis. In: Assessment and Evaluation of Information. van Gennip EMSJ, and Talmon JL. Eds. Amsterdam: IOS Press; 1994: 151-66.
    Suche in Google Scholar
  • 14 Lindberg G. Studies on diagnostic decision making in jaundice. Stockholm: Karolinska Institutes (Thesis); 1982: 1-60.
    Suche in Google Scholar
  • 15 Matzen P, Malchow-Møller A, Hilden J. et al. Differential diagnosis of jaundice: a pocket diagnostic chart. Liver 1984; 4: 360-71.
    PubMedSuche in Google Scholar
  • 16 Boom R, Gonzalez C, Fridman L. et al. Looking for “indicants” in the differential diagnosis of jaundice. Med Decis Making 1986; 6: 36-41.
    CrossrefPubMedSuche in Google Scholar
  • 17 Malchow-Møller A, Thomsen C, Matzen P. et al. Computer diagnosis in jaundice. Bayes’ rules founded on 1002 consecutive cases. J Hepatol 1986; 3: 156-63.
    PubMedSuche in Google Scholar
  • 18 O’Beirn SF, Given HF, Lavelle SM. Experiences with compiling a diagnostic data base on jaundice presenting to a general hospital. Irish J Med Sci 1987; 156: 93-7.
    CrossrefPubMedSuche in Google Scholar
  • 19 Malchow-Møller A, Thomsen C, Hilden J. et al. A decision tree for early differentiation between obstructive jaundice and non-obstructive jaundice. Scand J Gastroenterol 1988; 23: 391-401.
    CrossrefPubMedSuche in Google Scholar
  • 20 Segaar RW, Wilson JHP, Habbema JDF. et al. Transferring a diagnostic decision aid for jaundice. Neth J Med 1988; 33: 5-15.
    PubMedSuche in Google Scholar
  • 21 Miller RA. Medical diagnostic decision support systems - past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1994; 1: 8-27.
    CrossrefPubMedSuche in Google Scholar
  • 22 Pauker SG, Gorry GA, Kassirer JP, Schwartz WB. Towards the simulation of clinical cognition. Taking a present illness by computer. Am J Med, 1976; 60: 981-96.
    CrossrefPubMedSuche in Google Scholar
  • 23 Masarie FE, Miller RA, Myers JD. INTER-NIST-1 properties: representing common sense and good medical practice in a computerized medical knowledge base. Comput Biomed Res, 1985; 18: 458-79.
    CrossrefPubMedSuche in Google Scholar
  • 24 Barnett GO, Cimino JJ, Hupp JA, Hoffer EP. DXplain. An evolving diagnostic decision-support system. J Am Med Inform Assoc 1987; 258: 67-74.
    PubMedSuche in Google Scholar
  • 25 Miller RA, Masarie Jr FE. Use of the Quick Medical Reference (QMR) program as a tool for medical education. Method Inform Med 1989; 28: 340-5.
    PubMedSuche in Google Scholar
  • 26 Warner Jr HR. Iliad: moving medical decision-making into new frontiers. Method Inform Med 1989; 28: 370-2.
    PubMedSuche in Google Scholar
  • 27 Molino G, Ballarè M, Aurucci PE. et al. Application of artificial intelligence techniques to a well defined clinical problem: jaundice diagnosis. Int J Biomed Comp 1990; 12: 189-202.
    PubMedSuche in Google Scholar
  • 28 Euricterus Project Management Group – Objective medical decision-making: clinical database for diagnosis of jaundice (EURICTERUS). In: Advances in Biomedical Engineering. Beneken JEW, and Thévenin V. Eds. Amsterdam: IOS Press; 1993: 35-64.
    Suche in Google Scholar
  • 29 Molino G, Molino F, Furia D. et al. Computer-aided diagnosis in jaundice: comparison of the knowledge-based and probabilistic approaches. Method Inform Med 1996; 35: 41-51.
    PubMedSuche in Google Scholar
  • 30 Quaglini S, Stefanelli M, Barosi G. et al. A performance evaluation of the expert system ANEMIA. Comp Biomed Res 1988; 21: 307-23.
    CrossrefPubMedSuche in Google Scholar
  • 31 Hilden J, Habbema JDF, Bjerregaard B. The measurement of performance in probabilistic diagnosis. II. Trustworthiness of the exact values of the diagnostic probabilities. Method Inform Med 1978; 17: 227-37.
    PubMedSuche in Google Scholar
  • 32 Hilden J, Habbema JDF, Bjerregaard B. The measurement of performance in probabilistic diagnosis. III. Methods based on continuous functions of the diagnostic probabilities. Method Inform Med 1978; 17: 238-46.
    PubMedSuche in Google Scholar
  • 33 Hanley JA, McNeil BJ. The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve. Radiology 1982; 143: 26-36.
    PubMedSuche in Google Scholar