Methods Inf Med 2005; 44(01): 44-56
DOI: 10.1055/s-0038-1633922
Original Article
Schattauer GmbH

An Inventory of Evaluation Studies of Information Technology in Health Care

Trends in Evaluation Research 1982-2002
E. Ammenwerth
1   UMIT – University for Health Sciences, Medical Informatics and Technology, Institute for Health Information Systems, Hall in Tyrol, Austria
N. de Keizer
2   Department of Medical Informatics, Academic Medical Center, University of Amsterdam, The Netherlands
› Author Affiliations
Further Information

Publication History

Received: 02 April 2004

accepted: 27 October 2004

Publication Date:
06 February 2018 (online)


Objectives: During the last years the significance of evaluation studies as well as the interest in adequate methods and approaches for evaluation has grown in medical informatics. In order to put this discussion into historical perspective of evaluation research, we conducted a systematic review on trends in evaluation research of information technology in health care from 1982 to 2002.

Methods: The inventory is based on a systematic literature search in PubMed. Abstracts were included when they described an evaluation study of a computer-based component in health care. We identified 1035 papers from 1982 to 2002 and indexed them based on a multi-axial classification describing type of information system, study location, evaluation strategy, evaluation methods, evaluation setting, and evaluation focus.

Results and Conclusions: We found interesting developments in evaluation research in the last 20 years. For example, there has been a strong shift from medical journals to medical informatics journals. With regard to methods, explanatory research and quantitative methods have dominated evaluation studies in the last 20 years. Since 1982, the number of lab studies and technical evaluation studies has declined, while the number of studies focusing on the influence of information technology on quality of care processes or outcome of patient care has increased. We interpret this shift as a sign of maturation of evaluation research in medical informatics.