Appl Clin Inform 2019; 10(05): 810-819
DOI: 10.1055/s-0039-1697905
State of the Art/Best Practice Paper
Georg Thieme Verlag KG Stuttgart · New York

Towards a Maturity Model for Clinical Decision Support Operations

Evan W. Orenstein
1   Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States
2   Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, Georgia, United States
,
Naveen Muthu
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Asli O. Weitkamp
5   Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
,
Daria F. Ferro
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Mike D. Zeidlhack
6   Phrase Health Inc., Philadelphia, Pennsylvania, United States
,
Jason Slagle
5   Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
,
Eric Shelov
3   Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
4   Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
,
Marc C. Tobias
6   Phrase Health Inc., Philadelphia, Pennsylvania, United States
› Author Affiliations
Further Information

Publication History

04 May 2019

14 August 2019

Publication Date:
30 October 2019 (online)

Abstract

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main “pillars”: “Content Creation,” “Analytics and Reporting,” and “Governance and Management.” Each pillar contains five levels—advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A “roof” represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Authors' Contributions

E.W.O., N.M., A.O.W., D.F.F., M.D.Z., J.S., E.S., and M.C.T. wrote the manuscript. N.M., M.D.Z., and M.C.T. performed primary data collection. E.W.O., N.M., D.F.F., M.D.Z., and M.C.T. analyzed the data and developed the proposed CDS maturity model.


Protection of Human and Animal Subjects

No human subjects were involved in this project.


 
  • References

  • 1 Bright TJ, Wong A, Dhurjati R. , et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
  • 2 Roshanov PS, You JJ, Dhaliwal J. , et al; CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (88) 88
  • 3 Nieuwlaat R, Connolly SJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6: 90
  • 4 Souza NM, Sebaldt RJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (87) 87
  • 5 Sahota N, Lloyd R, Ramakrishna A. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (91) 91
  • 6 Hemens BJ, Holbrook A, Tonkin M. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (89) 89
  • 7 Roshanov PS, Fernandes N, Wilczynski JM. , et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657
  • 8 Osheroff JA, Teich JM, Levick D. , et al. Improving Outcomes with Clinical Decision Support: an Implementer's Guide. 2nd ed. Chicago, IL: HIMSS; 2012
  • 9 Van de Velde S, Kunnamo I, Roshanov P. , et al; GUIDES expert panel. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implement Sci 2018; 13 (01) 86
  • 10 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
  • 11 Millard Krause T. CMIO/CHIO Summit: managing clinical decision support and improving workflow. Healthcare Informatics Magazine (2016). Available at: https://www.healthcare-informatics.com/article/ehr/cmiochio-summit-managing-clinical-decision-support-and-improving-workflow . Accessed November 9, 2018
  • 12 Carvalho JV, Rocha Á, Abreu A. Maturity models of healthcare information systems and technologies: a literature review. J Med Syst 2016; 40 (06) 131
  • 13 Knosp BM, Barnett WK, Anderson NR, Embi PJ. Research IT maturity models for academic health centers: early development and initial evaluation. J Clin Transl Sci 2018; 2 (05) 289-294
  • 14 HIMSS Analytics - North America. Electronic Medical Record Adoption Model. Available at: https://www.himssanalytics.org/emram . Accessed November 12, 2018
  • 15 Analytics HIMSS. Continuity of Care Maturity Model. Available at: https://www.himssanalytics.org/ccmm . Accessed: November 23, 2018
  • 16 van de Wetering R, Batenburg R. Towards a theory of PACS deployment: an integrative PACS maturity framework. J Digit Imaging 2014; 27 (03) 337-350
  • 17 Corrigan D, McDonnell R, Zarabzadeh A, Fahey T. A multistep maturity model for the implementation of electronic and computable diagnostic clinical prediction rules (eCPRs). EGEMS (Wash DC) 2015; 3 (02) 1153
  • 18 Staggers N, Rodney M, Alafaireet P. , et al. Promoting Usability in Health Organizations: Initial Steps and Progress Toward a Healthcare Usability Maturity Model. Chicago, IL: Health Information Management Systems Society; 2011: 1-55
  • 19 Innovation Pathways Maturity Model HIMSS. Available at: https://www.himss.org/library/innovation/pathways-maturity-model . Accessed November 23, 2018
  • 20 Lipsitz LA. Understanding health care as a complex system: the foundation for unintended consequences. JAMA 2012; 308 (03) 243-244
  • 21 Ash JS, Smith III AC, Stavri PZ. Performing subjectivist studies in the qualitative traditions responsive to users. In: Friedman CP, Wyatt JC. , eds. Evaluation Methods in Medical Informatics. 2nd ed. New York, NY: Springer-Verlag; 2006: 267-300
  • 22 Marshall MN. Sampling for qualitative research. Fam Pract 1996; 13 (06) 522-525
  • 23 Tobias MC, Orenstein EW, Muthu N. Decision support for decision support: a novel system to prioritize improvement efforts, identify safety hazards, and measure improvement. AMIA Annu Symp Proc 2018; 2018: 2246
  • 24 Orenstein EW, Weitkamp AO, Rosenau P, Mallozzi C, Tobias MC. Towards a maturity model for clinical decision support operations: an interactive panel. In: American Medical Informatics Association Clinical Informatics Conference; April 30–May 2, 2019; Atlanta, GA
  • 25 Carayon P, Schoofs Hundt A, Karsh BT. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
  • 26 Holden RJ, Carayon P, Gurses AP. , et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11) 1669-1686
  • 27 Jones BE, Collingridge DS, Vines CG. , et al. CDS in a learning health care system: identifying physicians' reasons for rejection of best-practice recommendations in pneumonia through computerized clinical decision support. Appl Clin Inform 2019; 10 (01) 1-9
  • 28 Aaron S, McEvoy DS, Ray S, Hickman TTT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 26 (01) 37-43
  • 29 Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc 2011; 18 (03) 276-281
  • 30 Wright A, Hickman TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
  • 31 Yoshida E, Fei S, Bavuso K, Lagor C, Maviglia S. The value of monitoring clinical decision support interventions. Appl Clin Inform 2018; 9 (01) 163-173
  • 32 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
  • 33 Thayer JG, Miller JM, Fiks AG, Tague L, Grundmeier RW. Assessing the safety of custom web-based clinical decision support systems in electronic health records: a case study. Appl Clin Inform 2019; 10 (02) 237-246
  • 34 Institute of Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press; 2007
  • 35 Lindgren H, Lu MH, Hong Y, Yan C. Applying the zone of proximal development when evaluating clinical decision support systems: a case study. Stud Health Technol Inform 2018; 247: 131-135
  • 36 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
  • 37 Nielsen J. Corporate UX maturity: stages 5–8. 2006. Available at: https://www.nngroup.com/articles/ux-maturity-stages-5-8/ . Accessed November 23, 2018
  • 38 Wright A, Feblowitz JC, Pang JE. , et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
  • 39 Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev 2009; 8 (03) CD001096
  • 40 Mutafelija B, Stromberg H. Systematic Process Improvement Using ISO 9001:2000 and CMMI. Norwood, MA: Artech House; 2003
  • 41 Sanders D, Burton DA, Prott D. The healthcare analytics adoption model: a framework and roadmap. 2016 . Available at: https://downloads.healthcatalyst.com/wp-content/uploads/2013/11/analytics-adoption-model-Nov-2013.pdf . Accessed September 2, 2019
  • 42 Tarakci M, Ates NY, Porck JP, van Knippenberg D, Groenen PJF, de Haas M. Strategic consensus mapping: a new method for testing and visualizing strategic consensus within and between teams. Strateg Manage J 2014; 35: 1053-1069
  • 43 Okoli C, Pawlowski S. The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 2004; 42: 15-29