Download PDF
Methods Inf Med 2006; 45(01): 125-136
DOI: 10.1055/s-0038-1634049
Original Article
Schattauer GmbH

Factors Influencing Success and Failure of Health Informatics Systems[*]

A Pilot Delphi Study
J. Brender
1   Institute of Health Technology and Science, University of Aalborg, and Virtual Centre for Health Informatics, Aalborg, Denmark
,
E. Ammenwerth
2   Institute for Health Information Systems, UMIT – University of Health Sciences, Medical Informatics and Technology, Innsbruck, Austria
,
P. Nykänen
3   Department of Computer Sciences, Tampere University, Tampere, Finland
,
J. Talmon
4   Department of Medical Informatics, Maastricht University, Maastricht, The Netherlands
› Author Affiliations
Further Information

Publication History

Publication Date:
06 February 2018 (online)

    Permissions and Reprints

Summary

Objectives: The aim is to gain information on factors influencing success and failure for Health Informatics applications from a group of medical informaticians.

Methods: Based on the presentations at a special topic conference on success and failure in Health ICT and analysis of the proceedings, we conducted a Delphi study on success and failure aspects.

Results: A total of 110 success factors and 27 failure criteria were identified, distributed on categories like functional, organizational, behavioral, technical, managerial, political, cultural, legal, strategy, economy, education and user acceptance. These factors and criteria were rated for six different system types. Unanimously it was agreed that “collaboration and co-operation” and “setting goals and courses” are “essential for the success” of clinical systems, and “user acceptance” for educational systems. Similarly, the score “essential in order to avoid a failure” were given unanimously on clinical systems for “response rate and other performance measures” and on administrative systems for “not understanding the organizational context” with “not understanding or foreseeing the extent to which the new IT-system affects the organization, its structure and/or work procedures” as the highest scoring sub-item.

Conclusions: All success factors and failure criteria were considered relevant by the Delphi expert panel. There is no small set of relevant factors or indicators, but success or failure of a Health ICT depends on a large set of issues. Further, clinical systems and decision support systems depend on more factors than other systems.

Keywords

Success - failure - systems development - Delphi

* Additional data available for free at URL http://www.methods-online.com


 

  • References

  • 1 Briggs RO, deVreede G-J. Nunamaker JF, Sprague RH. Special issue: Information systems success. Editorial. J Management Information Systems 2003; 19 (04) 5-8.
    PubMedGoogle Scholar
  • 2 Lyytinen K. Different perspectives on Information Systems: Problems and solutions. ACM Computing Surveys 1987; 19 (01) 5-46.
    CrossrefPubMedGoogle Scholar
  • 3 Lucas HC. Implementation, the key to successful information systems. New York: Columbia University Press; 1981
    Google Scholar
  • 4 Newman M, Sabherwal R. Determinants of commitment to information systems development: A longitudinal investigation. MIS Quarterly 1996; 20 (01) 23-54.
    CrossrefPubMedGoogle Scholar
  • 5 Lorenzi NM, Riley RT. Organizational issues = change. Int J of Medical Informatics 2003; 69: 197-203.
    PubMedGoogle Scholar
  • 6 Seddon PB, Staples S, Patnayakuni R, Bowtell M. Dimensions of information systems success. Communications of the Association for Information Systems, 1999; 2 article 20 http://cais.isworld.org/articles/2–20/article.html 47 5.9.2004.
    PubMedGoogle Scholar
  • 7 Avital M. Reexamining information systems success through the information technology professionals perspective. Sprouts: Working papers on information environments, systems and organizations. http://weatherhead.cwru.edu/sprouts/2003/030206.pdf 15 10.9.2004
    PubMedGoogle Scholar
  • 8 Ammenwerth E, Brender J, Nykänen P, Prokosch H-U, Rigby M, Talmon J. Visions and strategies to improve evaluation of health information systems – Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. Int J Med Informatics 2004; 73 (06) 479-91.
    CrossrefPubMedGoogle Scholar
  • 9 DeLone WH, McLean ER. Information systems success: The quest for dependent variable. Information Systems Research 1992; 3 (01) 60-95.
    CrossrefPubMedGoogle Scholar
  • 10 DeLone WH, McLean ER. Information systems success revisited. Proceedings of the Annual Hawaii International Conference on Systems Sciences 2002 (HICSS-35 02) IEEE Computer Society; 2002: 238-48.
    Google Scholar
  • 11 DeLone WH, McLean ER. The DeLeone and McLean Model of Information Systems Success: A ten-year update. J Management Information Systems 2003; 19 (04) 9-30.
    PubMedGoogle Scholar
  • 12 van der Meijden MJ, Tange HJ, Troost J, Hasman A. Determinants of success of Inpatient Clinical Information Systems: A literature review. J Am Medical Informatics Assoc 2003; 10 (03) 235-43.
    PubMedGoogle Scholar
  • 13 Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. From the field: Overcoming Barriers to adopting and implementing Computerized Physician Order Entry Systems in U.S. hospitals. Health Affairs 2004; 23 (04) 1184-90.
    PubMedGoogle Scholar
  • 14 Bikson TK, Eveland JD. Technology Transfer as a Framework for Understanding Social Impacts of Computerization. In: Smith MJ, Salvendy G. Work With Computers: Organizational, Management, Stress and Health Aspects. Proceedings of the 3rd Conference on Human-Computer Interaction. Amsterdam:: Elsevier.; 1989. 1 28-37.
    Google Scholar
  • 15 Crosswell PL. Obstacles to GIS Implementation and Guidelines to Increase the Opportunities for Success. URISA J 1991; 3 (01) 43-56.
    PubMedGoogle Scholar
  • 16 Without Change There is no Progress – Coping with Chaos, a Global Survey. Price Waterhouse 1997
    Google Scholar
  • 17 Brender J. Evaluation of Health Information Applications in a Lifecycle Perspective. In: Blobel B, Gell G, Hildebrandt C, Engelbrecht R. Contribution of Medical Informatics to Health, Integrated Clinical Data and Knowledge to support Primary, Secondary, Tertiary and Home Care. Proc. European Federation for Medical Informatics Special Topic Conference, June 13–16th, 2004. Munich, Berlin:: Akademische Verlagsgesellschaft Aka GmbH; 2004: 7-14.
    Google Scholar
  • 18 Jørgensen T. Measuring effects. In: van Gennip EMSJ, Talmon J. Assessment and Evaluation of Information Technologies. Amsterdam:: IOS Press.; Studies in Health Technology and Informatics. 1995. 17 99-109.
    Google Scholar
  • 19 Stoop A, Berg M. Integrating quantitative and qualitative methods in patient care information system evaluation: guidance for the organizational decision maker. Methods Inf Med 2003; 42 (04) 458-62.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 20 Blobel B, Gell G, Hildebrandt C, Engelbrecht R. Contribution of Medical Informatics to Health, Integrated Clinical Data and Knowledge to support Primary, Secondary, Tertiary and Home Care. Proc. European Federation for Medical Informatics Special Topic Conference, June 13-16, 2004 Munich, Berlin:: Akademische Verlagsgesellschaft Aka GmbH; 2004
    Google Scholar
  • 21 Willems JL, Arnaud P, Van Bemmel JH, Degani R, MacFarlane PW, Zywuetz Ch. Common Standards for quantitative electrocardiography: Goals and Main Results. Meth Inf Med 1990; 29 (04) 263-71.
    PubMedGoogle Scholar
  • 22 Van Ast JF, Talmon JL, Renier WO, Hasman A. Knowledge elicitation among experts. Chapter 5 of “Diagnostic Reference Frames for Saizures” PhD Thesis of JF van Ast. Maastricht University; 2005
    Google Scholar
  • 23 Kors JA, Sittig AC, van Bemmel JH. The Delphi Method to validate diagnostic knowledge in computerized ECG interpretation. Methods Inform Med 1990; 29: 44-50.
    PubMedGoogle Scholar
  • 24 Dalkey NC. The Delphi Method: An Experimental Study of Group Opinion – Prepared for United States Air Force Project. Rand, Santa Monica (CA); 1969
    Google Scholar
  • 25 Linstone HA, Turoff M. The Delphi Method, Techniques and Applications. Addison-Wesley Publishing Company, Reading (MA); 1975
    Google Scholar
  • 26 Adler A, Ziglio E. Gazing into the Oracle – The Delphi Method and its Application to Social Policy and Public Health. Jessical Kingsley Publishers, London; 1996
    Google Scholar
  • 27 Crisp J, Pelletier D, Duffield C, Adams A, Nagy S. The Delphi Method?. Nursing Research 1997; 46 (02) 116-8.
    CrossrefPubMedGoogle Scholar
  • 28 Buzan T. The Mind Map Book. Plume; 1996
    Google Scholar
  • 29 Fröhlich D, Gill C, Krieger H. Workplace Involvement in Technological Innovation in the European Community, Vol I:. Roads to Participation. European Foundation for the Improvement of Living and Working Conditions Dublin; 1993
    Google Scholar
  • 30 Kaplan B, Shaw N. Future Directions in Evaluation Research: People, Organizational, and Social Issues. Methods Inf Med 2004; 43: 215-31.
    Article in Thieme ConnectPubMedGoogle Scholar