A Method for Assessing the Impact of a Computer-Based Decision Support System on Health Care Outcomes

Abstract:

The wide variation in utilization of diagnostic resources has not been decreased by the proliferation of new diagnostic technologies. We wish to test the hypothesis that the introduction of a medical decision support system into clinical practice could potentially lead to more efficient use of diagnostic information, and therefore lead to a reduction in overall laboratory use and cost of care. We have devised and are currently implementing a randomized controlled trial of a computer based decision support system, the University of Pittsburgh version of Quick Medical Reference (QMR).The main purpose of the study is to determine the effect of the QMR program on specific outcome measures: length of hospital stay, number and types of diagnostic tests ordered, and overall charges. An important part of this evaluation is relating the initial level of diagnostic uncertainty expressed by the admitting housestaff team to utilization of diagnostic resources. The purpose of this paper is to describe the methodology for carrying out this controlled trial, and to describe our initial experiences with its implementation.

Key-Words

Computer-Assisted Diagnosis - Differential Diagnosis - Randomized Controlled Trial - Medical Care Evaluation - Outcomes Process and Assessment

• REFERENCES

• 1 Wennberg JE, Gittleshon A. Small area variations in health care delivery. Science 1973; 18: 1102-8.
• 2 Chassin MR, Brook RH, Park RE. et al. Regional variations in the use of common surgical services by the medicare population. N Engl J Med 1986; 314: 285-90.
• 3 Sui AL, Sonnenberg FA, Manning WG. Inappropriate use of hospitals in a randomized trial of health insurance plans. N Engl J Med 1986; 315: 1259-66.
• 4 Wennberg JE. Freeman Are hospital services rationed in New Haven or overutilized in Boston?. Lancet. 1987 23 May.
• 5 Williamson JW, German PS, Weis R. et al. Health science information management and continuing education of physicians. Ann Intern Med 1989; 110: 151-60.
• 6 Haynes RB, Sackett MD, Tugwell P. Problems in the handling of clinical and research evidence by medical practitioners. Arch Intern Med 1983; 143: 1971-5.
• 7 Osheroff JA, Forsythe DE, Buchanan BG. et al. Understanding physicians’ information needs: An analysis of questions posed during clinical teaching in internal medicine. Ann Intern Med 1991; 114: 576-81.
• 8 Covell DG, Uman GC, Manning PR. Information needs in office practice: Are they being met?. Ann Intern Med 1985; 103: 596-9.
• 9 Young MJ, Poses RM. Can physicians be rational about diagnostic tests?. Clin Lab Med 1984; 04: 25-9.
• 10 Allman RM, Steinberg EP, Keruly JL. et al. Physician tolerance for uncertainty: use of liver-spleen scan to detect metastases. JAMA 1985; 254: 246-8.
• 11 Griner PF, Glaser RJ. Misuse of laboratory tests and diagnostic procedures. N Engl J Med 1982; 307: 1336-9.
• 12 Schwantz WB, Patil RS, Szolovits P. Artificial intelligence in medicine: Where do we stand?. N Engl J Med 1987; 316: 685-8.
• 13 Shortliffe EH. Computer programs to support clinical decision making. JAMA 1987; 258: 61-6.
• 14 Nelson SJ, Blois MS, Tuttle MS. et al. Evaluating Reconsider. A computer program for diagnostic prompting. J Med Systems 1985; 09: 292-5.
• 15 Barnett GO, Cimino JJ, Hupp JA, Hoffer EP. DXplain. An evolving diagnostic decision support system, JAMA 1987; 258: 67-74.
• 16 Barnett GO, Chimino JJ, Hupp JA. et al. DXplain: Experience with knowledge acquisition and program evaluation. Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care; IEEE Computer Society Press 1987: 150-4.
• 17 Pryor TA, Gardner RM, Clayton PD. et al. The HELP System. J Med Systems 1983; 07: 87-102.
• 18 Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990; 322: 1499-504.
• 19 McDonald CJ, Hui SL, Smith DM. et al. Reminders to physicians from an introspective computer medical record. Ann Int Med 1984; 10 (01) 130-8.
• 20 Kent DL, Shortliffe EH, Carlson RW, Bishoff MB, Jacobs CD. et al. Improvements in data collection through physician use of a computer-based chemotherapy Treatment consultant. J Clin Oncology 1985; 03: 1409-17.
• 21 Evans RS, Larsen RA, Burke JP. et al. Computer surveillance of hospital-acquired infections and antibiotic use. JAMA 1986; 256: 1007-11.
• 22 Adams ID, Chan M, Clifford PC. et al. Computer aided diagnosis of acute abdominal pain: a multicentre study. Br Med J 1986; 293: 800-4.
• 23 Miller RA, McNeil SC, Challinor SM. et al. The INTERNIST-l/Quick Medical Reference project -status report. West J Med 1986; 145: 816-22.
• 24 Bankowitz RA, McNeil MA, Challinor SM. A computer-assisted medical diagnostic consultation service: Implementation and evaluation of a prototype. Ann Intern Med 1989; 110: 824-32.
• 25 Bankowitz RA, McNeil MA, Challinor SM, Miller RA. The effect of a computer-assisted general medicine diagnostic consultation service on house-staff diagnostic strategy. Methods Inform Med 1989; 28: 352-6.
• 26 Bankowitz RA. The role of diagnostic uncertainty in health care outcomes and its potential for modification with decision support. European Federation for Medical Informatics, Tenth International Congress (MIE 91).
• 27 Thomas WJ, Ashcraft MLF. Measuring severity of illness: Six severity systems and their ability to explain cost variations. Inquiry 1991; 28: 39-55.
• 28 Gerrity MS, De Vellis RF, Earp JA. Physicians’ reactions to uncertainty in patient care. A new measure and new insights. Med Care 1990; 08: 724-36.
• 29 Miller PL. Issues in the evaluation of artificial intelligence systems in medicine. Proceedings of the Ninth Annual Symposium on Computer Applications in Medical Care. IEEE Computer Society Press 1985: 281-5.
• 30 Greer AL. The state of the art versus the state of the science. The diffusion of new medical technologies into practice. Int J Technol Assessment Health Care 1988; 04: 5-26.
• 31 Forsythe DE, Buchanan BG. Broadening our approach to evaluating medical information systems. Fifteenth Annual Symposium on Computer Applications in Medical Care. McGraw Hill 1991; 8-12.
• 32 Sackett DL, Haynes RB, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine. Toronto: Little Brown and Company; 1985: 181-8.