Comparison of Rheumatological Diagnoses by a Bayesian Program and by Physicians

Abstract

A Bayesian decision support system was developed for the diagnosis of rheumatic disorders. Knowledge in this system is represented as evidential weights of findings. Simple weights were calculated as the logarithm of likelihood ratios on the basis of 1,000 consecutive patients from a rheumatological clinic. The effect of various methods to improve performance of the system by modification of the weights was studied. Three methods had a mathematical basis; a fourth consisted of weights adapted by a human expert, which allowed inclusion of diagnostic rules such as defined in widely accepted criteria sets. The system’s performance was measured in a test population of 570 different cases from the same clinic and compared with predictions of diagnostic outcome made by rheumatologists. The weights from a human expert gave optimal results (sensitivity 65% and specificity 96%), that were close to the physicians’ predictions (sensitivity 64% and specificity 98%). The methods to measure the performance of the various models used in this study emphasize sensitivity, specificity and the use of receiver operating characteristics.

Key-Words

Computer-Assisted Diagnosis - Rheumatology - Evaluation - Knowledge Acquisition - Bayes’ Theorem

• REFERENCES

• 1 Kassirer JR. Our stubborn quest for diagnostic certainty. A cause of excessive testing. N Engl J Med 1989; 320: 1489-91.
• 2 Bankowitz RA, McNeil MA, Challinor SM, Miller RA. Effect of a computer-assisted general medicine diagnostic consultation service on housestaff diagnostic strategy. Meth Inf Med 1989; 28: 352-6.
• 3 Dannenberg AL, Shapiro AR, Fries JF. Enhancement of clinical predictive ability by computer consultation. Meth Inf Med 1979; 18: 10-4.
• 4 Kingsland LC. The evaluation of medical expert systems: experience with the Al/Rheum knowledge based consultant system in rheumatology. Proc Ninth Annual SCAMC. Washington DC: IEEE Computer Society Press; 1985: 292-5.
• 5 Mathew B, Norris D, Hendry D, Waddell G. Artificial intelligence in the diagnosis of low back pain and sciatica. Spine 1988; 13: 168-72.
• 6 Porter JF, Kingsland LC, Lindberg DAB. et al. The AI/RHEUM knowledge-based computer consultant system in rheumatology. Performance in the diagnosis of 59 connective tissue disease patients from Japan. Arthritis Rheum 1988; 31: 219-26.
• 7 Schewe S, Herzer P, Kruger K. Prospective Application of an Expert System for the Medical History of Joint Pain. Klin Wochenschr 1990; 68: 466-71.
• 8 Bernelot HJMoens. Development and testing of a computer program for the diagnosis of joint pain (in Dutch). Proceedings MIC 1988; 201-6.
• 9 Bernelot HJMoens, Van der Korst JK. Development and validation of a computer program using Bayes’ theorem to support diagnosis of rheumatic disorders. Acc for publ in Ann Rheum Dis..
• 10 Burer JH, Groot A, Van de Laar MA, Olsthoorn PG, Van Soesbergen RM, Van der Korst JK. Bacterial joint infections: a retrospective study for the source of the infection (in Dutch). Ned Tijdschr Geneeskd 1989; 133: 1693-6.
• 11 Spiegelhalter DJ. Statistical Methodology for evaluating gastrointestinal symptoms. Clin Gastroenterology 1985; 14: 489-515.
• 12 Chard T. The effect of dependence on the performance of Bayes’ theorem: an evaluation using a computer simulation. Comp Progr Biomed 1989; 29: 15-9.
• 13 Hilden J. Statistical diagnosis based on conditional independence does not require it. Comp Biol Med 1984; 14: 429-35.
• 14 Spiegelhalter DJ, Knill-Jones RP. Statistical and knowledge-based approaches to clinical decision support systems, with an application in gastro-enterology. J R Stat Soc 1984; 147: 35-77.
• 15 Harrell FE. The LOGIST Procedure. SAS statistical software manual. Chapter 23: 269-93.
• 16 The American Rheumatism Association Glossary Committee. Dictionary of the Rheumatic diseases. Published by Contact Associates International Ltd; New York: 1982. Volumes 1 & 2.
• 17 Habbema JDF, Hilden J, Bjerregaard B. The measurement of performance in probabilistic diagnosis (parts 1, 2, and 3). Meth Inf Med 1978; 17: 217-46.
• 18 Habbema JDF, Hilden J, Bjerregaard B. The measurement of performance in probabilistic diagnosis (part 4 and 5). Meth Inf Med 1981; 20: 80-100.
• 19 Zagoria RJ, Reggia JA. Transferability of medical decision support systems based on Bayesian classification. Med Decis Making 1983; 03: 501-9.
• 20 Shapiro AR. The evaluation of clinical predictions: a method and initial application. N Engl J Med 1977; 296: 1509-14.
• 21 Adlassnig K-P, Scheithauer W. Performance evaluation of medical expert systems using ROC curves. Comp Biomed Res 1989; 22: 297-313.
• 22 De Dombal FT, Horrocks JC. Use of Receiver Operating Characteristic (ROC) curves to evaluate computer confidence threshold and clinical performance in the diagnosis of appendicitis. Meth Inf Med 1978; 17: 157-61.
• 23 Adlassnig K-P, Kolarz G. Representation and semiautomatic acquisition of medical knowledge in Cadiag-1 and Cadiag-2. Comp Biomed Res 1986; 19: 63-79.
• 24 Miller RA, Pople HE, Myers J. Internist-I, an experimental computer-based diagnostic consultant for general internal medicine. N Engl J Med 1983; 307: 468-76.