Subscribe to RSS
Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital HospitalFunding This study was funded by the Digital Health CRC, grant no.: STARS 0034.
Objective A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.
Methods Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.
Results A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.
Conclusion Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.
Keywordslearning health care system - electronic health records and systems - clinical decision support - hospital information systems - clinical data management - dashboard - digital hospital
Conception and design: H.C.L. and C.M.S. Data collection: J.D.P. and C.M.S. Data extraction: H.C.L., A.K.R., J.M., and A.V.D.V. Quality Assessment: J.A.A. and H.C.L. Data analysis and interpretation: H.C.L., J.A.A., A.V.D.V., and C.M.S. Drafting the manuscript: H.C.L., J.A.A., A.V.D.V., and C.M.S. This article has co–first authorship by three authors. H.C.L., J.A.A., and A.V.D.V. contributed equally and have the right to their name first in their CV. Critical revision of article: H.C.L., J.A.A., A.V.D.V., A.K.R., J.M., O.J.C., J.D.P., M.A.B., T.H., S.S., and C.M.S. All authors contributed to the article and approved the submitted version.
Protection of Human and Animal Subjects
Human and/or animal subjects were not involved in completing the present review.
* Marked Authors Are Co–First Authors.
Received: 29 July 2021
Accepted: 09 January 2022
Article published online:
06 April 2022
© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
- 1 Charles D, Gabriel M, Furukawa MF. Adoption of electronic health record systems among U. S. non -federal acute care hospitals: 2008–2015. Accessed October 7, 2021 at: https://www.healthit.gov/sites/default/files/briefs/2015_hospital_adoption_db_v17.pdf
- 2 Carroll JS, Quijada MA. Redirecting traditional professional values to support safety: changing organisational culture in health care. Qual Saf Health Care 2004; 13 (Suppl. 02) ii16-ii21
- 3 Aggarwal A, Aeran H, Rathee M. Quality management in healthcare: The pivotal desideratum. J Oral Biol Craniofac Res 2019; 9 (02) 180-182
- 4 Oaten J, Stayner G, Ballard J. et al. Baby deaths: hospital failures: an independent investigation has found a series of failures may have contributed to the deaths of 7 babies at a regional Victorian hospital. Accessed July 21, 2021 at: https://search-informit-org.ezproxy.library.uq.edu.au/doi/10.3316/tvnews.tsm201510160050
- 5 Barnett A, Winning M, Canaris S, Cleary M, Staib A, Sullivan C. Digital transformation of hospital quality and safety: real-time data for real-time action. Aust Health Rev 2019; 43 (06) 656-661
- 6 Sullivan C, Staib A, McNeil K, Rosengren D, Johnson I. Queensland digital health clinical charter: a clinical consensus statement on priorities for digital health in hospitals. Aust Health Rev 2020; 44 (05) 661-665
- 7 Mandl KD, Kohane IS, McFadden D. et al. Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS): architecture. J Am Med Inform Assoc 2014; 21 (04) 615-620
- 8 Olsen LA, Aisner D, McGinnis JM. eds; Institute of Medicine (US) Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press (US); 2007
- 9 Platt JE, Raj M, Wienroth M. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res 2020; 22 (03) e17026
- 10 Li X, Zhao X, Pu W, Chen P, Liu F, He Z. Optimal decisions for operations management of BDAR: a military industrial logistics data analytics perspective. Comput Ind Eng 2019; 137: 106100
- 11 Sullivan C, Staib A, Khanna S. et al. The National Emergency Access Target (NEAT) and the 4-hour rule: time to review the target. Med J Aust 2016; 204 (09) 354
- 12 Auliya RAknuranda I, Tolle H. A systematic literature review on healthcare dashboards development: trends, issues, methods, and frameworks. Adv Sci Lett 2018; 24 (11) 8632-8639
- 13 Buttigieg SC, Pace A, Rathert C. Hospital performance dashboards: a literature review. J Health Organ Manag 2017; 31 (03) 385-406
- 14 Dowding D, Randell R, Gardner P. et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84 (02) 87-100
- 15 Khairat SSDA, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
- 16 Maktoobi SMelchiori M. A brief survey of recent clinical dashboards. Accessed January 25, 2022 at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1073.1848&rep=rep1&type=pdf
- 17 West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
- 18 Wilbanks BALP, Langford PA. A review of dashboards for data analytics in nursing. Comput Inform Nurs 2014; 32 (11) 545-549
- 19 Veritas Health Innovation. Covidence: systematic review. Accessed January 25, 2022 at: https://unimelb.libguides.com/sysrev/covidence
- 20 Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract 2012; 18 (04) 746-752
- 21 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
- 22 Cox ZLP, Lewis CMNPCC, Lai P, Lenihan DJMD. Validation of an automated electronic algorithm and “dashboard” to identify and characterize decompensated heart failure admissions across a medical center. Am Heart J 2017; 183: 40-48
- 23 Fletcher GS, Aaronson BA, White AA, Julka R. Effect of a real-time electronic dashboard on a rapid response system. J Med Syst 2017; 42 (01) 5
- 24 Fuller TE, Garabedian PM, Lemonias DP. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon 2020; 85: 103047
- 25 Mlaver E, Schnipper JL, Boxer RB. et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12) 676-685
- 26 Paulson SS, Dummett BA, Green J, Scruth E, Reyes V, Escobar GJ. What do we do after the pilot is done? Implementation of a hospital early warning system at scale. Jt Comm J Qual Patient Saf 2020; 46 (04) 207-216
- 27 Schall Jr MC, Cullen L, Pennathur P, Chen H, Burrell K, Matthews G. Usability evaluation and implementation of a health information technology dashboard of Evidence-Based Quality Indicators. Comput Inform Nurs 2017; 35 (06) 281-288
- 28 Ye C, Wang O, Liu M. et al. A real-time early warning system for monitoring inpatient mortality risk: prospective study using electronic medical record data. J Med Internet Res 2019; 21 (07) e13719
- 29 Franklin A, Gantela S, Shifarraw S. et al. Dashboard visualizations: supporting real-time throughput decision-making. J Biomed Inform 2017; 71: 211-221
- 30 Ibrahim H, Sorrell S, Nair SC, Al Romaithi A, Al Mazrouei S, Kamour A. Rapid development and utilization of a clinical intelligence dashboard for frontline clinicians to optimize critical resources during Covid-19. Acta Inform Med 2020; 28 (03) 209-213
- 31 Kurtzman G, Dine J, Epstein A. et al. Internal medicine resident engagement with a laboratory utilization dashboard: mixed methods study. J Hosp Med 2017; 12 (09) 743-746
- 32 Merkel MJ, Edwards R, Ness J. et al. statewide real-time tracking of beds and ventilators during coronavirus disease 2019 and beyond. Crit Care Explor 2020; 2 (06) e0142
- 33 Staib A, Sullivan C, Jones M, Griffin B, Bell A, Scott I. The ED-inpatient dashboard: Uniting emergency and inpatient clinicians to improve the efficiency and quality of care for patients requiring emergency admission to hospital. Emerg Med Australas 2017; 29 (03) 363-366
- 34 Yoo J, Jung KY, Kim T. et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR Mhealth Uhealth 2018; 6 (11) e10666
- 35 Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30 (12) 2310-2317
- 36 Health IT. gov. 2020–2025 federal health it strategic plan 2020. Accessed October 5, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-10/Federal%20Health%20IT%20Strategic%20Plan_2020_2025.pdf
- 37 Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci 2016; 11 (01) 33
- 38 CFIR Research Team-Center for Clinical Management Research. Consolidated framework for implementation research. Accessed July 2, 2021 at: https://cfirguide.org/
- 39 Hunter SC, Kim B, Mudge A. et al. Experiences of using the i-PARIHS framework: a co-designed case study of four multi-site implementation projects. BMC Health Serv Res 2020; 20 (01) 573
- 40 Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learn Health Syst 2019; 4 (01) e10201
- 41 Waitman LR, Phillips IE, McCoy AB. et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf 2011; 37 (07) 326-332
- 42 Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021; 12 (04) 888-896