CC BY-NC-ND 4.0 · Appl Clin Inform 2021; 12(04): 888-896
DOI: 10.1055/s-0041-1735183
Case Report

Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring

Liza Prudente Moorman
1   Clinical Implementation Specialist, Advanced Medical Predictive Devices, Diagnostics, and Displays (AMP3D), Charlottesville, Virginia, United States
› Author Affiliations


A new development in the practice of medicine is Artificial Intelligence-based predictive analytics that forewarn clinicians of future deterioration of their patients. This proactive opportunity, though, is different from the reactive stance that clinicians traditionally take. Implementing these tools requires new ideas about how to educate clinician users to facilitate trust and adoption and to promote sustained use. Our real-world hospital experience implementing a predictive analytics monitoring system that uses electronic health record and continuous monitoring data has taught us principles that we believe to be applicable to the implementation of other such analytics systems within the health care environment. These principles are mentioned below:

• To promote trust, the science must be understandable.

• To enhance uptake, the workflow should not be impacted greatly.

• To maximize buy-in, engagement at all levels is important.

• To ensure relevance, the education must be tailored to the clinical role and hospital culture.

• To lead to clinical action, the information must integrate into clinical care.

• To promote sustainability, there should be periodic support interactions after formal implementation.

Protection of Human and Animal Subjects

No human subjects were involved in the project.

Publication History

Received: 10 April 2021

Accepted: 12 July 2021

Article published online:
22 September 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981; 9 (08) 591-597
  • 2 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (01) 17
  • 3 Davis JP, Wessells DA, Moorman JR. Coronavirus disease 2019 calls for predictive analytics monitoring-a new kind of illness scoring system. Crit Care Explor 2020; 2 (12) e0294
  • 4 Baig MM, GholamHosseini H, Gutierrez J, Ullah E, Lindén M. Early detection of prediabetes and T2DM using wearable sensors and internet-of-things-based monitoring applications. Appl Clin Inform 2021; 12 (01) 1-9
  • 5 Moorman JR, Carlo WA, Kattwinkel J. et al. Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial. J Pediatr 2011; 159 (06) 900-6.e1
  • 6 Moss TJ, Lake DE, Calland JF. et al. Signatures of subacute potentially catastrophic illness in the ICU: model development and validation. Crit Care Med 2016; 44 (09) 1639-1648
  • 7 Holder AL, Clermont G. Using what you get: dynamic physiologic signatures of critical illness. Crit Care Clin 2015; 31 (01) 133-164
  • 8 Beaulieu-Jones BK, Yuan W, Brat GA. et al. Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians?. NPJ Digit Med 2021; 4 (01) 62
  • 9 Pearson TA, Califf RM, Roper R. et al. Precision health analytics with predictive analytics and implementation research: JACC state-of-the-art review. J Am Coll Cardiol 2020; 76 (03) 306-320
  • 10 Lund O, Andersen B, Christensen MK. Old habits die hard: a case study on how new ways of teaching colonoscopy affect the habitus of experienced clinicians. Int J Med Educ 2016; 7: 297-308
  • 11 Keim-Malpass J, Kitzmiller RR, Skeeles-Worley A. et al. Advancing continuous predictive analytics monitoring: moving from implementation to clinical action in a learning health system. Crit Care Nurs Clin North Am 2018; 30 (02) 273-287
  • 12 Ruminski CM, Clark MT, Lake DE. et al. Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit. J Clin Monit Comput 2019; 33 (04) 703-711
  • 13 Kitzmiller RR, Vaughan A, Skeeles-Worley A. et al. Diffusing an innovation: clinician perceptions of continuous predictive analytics monitoring in intensive care. Appl Clin Inform 2019; 10 (02) 295-306
  • 14 Proctor E, Silmere H, Raghavan R. et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011; 38 (02) 65-76
  • 15 Laï MC, Brian M, Mamzer MF. Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France. J Transl Med 2020; 18 (01) 14
  • 16 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
  • 17 Lee JD, See KA. Trust in automation: designing for appropriate reliance. Hum Factors 2004; 46 (01) 50-80
  • 18 Politano AD, Riccio LM, Lake DE. et al; Predictive Monitoring in Patients with Trauma (PreMPT) Group. Predicting the need for urgent intubation in a surgical/trauma intensive care unit. Surgery 2013; 154 (05) 1110-1116
  • 19 Moss TJ, Clark MT, Calland JF. et al. Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: A retrospective cohort study. PLoS One 2017; 12 (08) e0181448
  • 20 Spaeder MC, Moorman JR, Tran CA. et al. Predictive analytics in the pediatric intensive care unit for early identification of sepsis: capturing the context of age. Pediatr Res 2019; 86 (05) 655-661
  • 21 Blackwell JN, Keim-Malpass J, Clark MT. et al. Early detection of in-patient deterioration: one prediction model does not fit all. Crit Care Explor 2020; 2 (05) e0116
  • 22 Glass G, Hartka TR, Keim-Malpass J, Enfield KB, Clark MT. Dynamic data in the ED predict requirement for ICU transfer following acute care admission. J Clin Monit Comput 2021; 35 (03) 515-523
  • 23 Kappen TH, van Loon K, Kappen MAM. et al. Barriers and facilitators perceived by physicians when using prediction models in practice. J Clin Epidemiol 2016; 70: 136-145
  • 24 Dharampal N, Cameron C, Dixon E, Ghali W, Quan ML. Attitudes and beliefs about the surgical safety checklist: just another tick box?. Can J Surg 2016; 59 (04) 268-275
  • 25 Kastner M, Li J, Lottridge D, Marquez C, Newton D, Straus SE. Development of a prototype clinical decision support tool for osteoporosis disease management: a qualitative study of focus groups. BMC Med Inform Decis Mak 2010; 10: 40
  • 26 Lemieux-Charles L, McGuire WL. What do we know about health care team effectiveness? A review of the literature. Med Care Res Rev 2006; 63 (03) 263-300
  • 27 Haynes AB, Weiser TG, Berry WR. et al; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009; 360 (05) 491-499
  • 28 Pham JC, Aswani MS, Rosen M. et al. Reducing medical errors and adverse events. Annu Rev Med 2012; 63: 447-463
  • 29 Müller M, Jürgens J, Redaèlli M, Klingberg K, Hautz WE, Stock S. Impact of the communication and patient hand-off tool SBAR on patient safety: a systematic review. BMJ Open 2018; 8 (08) e022202
  • 30 Korach ZT, Cato KD, Collins SA. et al. Unsupervised machine learning of topics documented by nurses about hospitalized patients prior to a rapid-response event. Appl Clin Inform 2019; 10 (05) 952-963
  • 31 Linnander E, McNatt Z, Boehmer K, Cherlin E, Bradley E, Curry L. Changing hospital organisational culture for improved patient outcomes: developing and implementing the leadership saves lives intervention. BMJ Qual Saf 2021; 30 (06) 475-483
  • 32 Heath ML, Porter TH. Physician leadership and health information exchange: literature review. BMJ Health Care Inform 2019; 26 (01) e100080
  • 33 Reader TW, Flin R, Mearns K, Cuthbertson BH. Developing a team performance framework for the intensive care unit. Crit Care Med 2009; 37 (05) 1787-1793
  • 34 van Diggele C, Roberts C, Burgess A, Mellis C. Interprofessional education: tips for design and implementation. BMC Med Educ 2020; 20 (Suppl. 02) 455
  • 35 Weinstein RS, Brandt BF, Gilbert JHV, Schmitt MH. Bridging the quality chasm: interprofessional teams to the rescue?. Am J Med 2013; 126 (04) 276-277
  • 36 Pronovost P, Berenholtz S, Dorman T, Lipsett PA, Simmonds T, Haraden C. Improving communication in the ICU using daily goals. J Crit Care 2003; 18 (02) 71-75
  • 37 Jain M, Miller L, Belt D, King D, Berwick DM. Decline in ICU adverse events, nosocomial infections and cost through a quality improvement initiative focusing on teamwork and culture change. Qual Saf Health Care 2006; 15 (04) 235-239
  • 38 Starmer AJ, Sectish TC, Simon DW. et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA 2013; 310 (21) 2262-2270
  • 39 Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care 2004; 13 (Suppl. 01) i85-i90
  • 40 Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy 2003; 8 (02) 94-99
  • 41 Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003; 362 (9391): 1225-1230
  • 42 McLean SF. Case-based learning and its application in medical and health-care fields: a review of worldwide literature. J Med Educ Curric Dev 2016;3JMECD.S20377
  • 43 Fairchild KD, Schelonka RL, Kaufman DA. et al. Septicemia mortality reduction in neonates in a heart rate characteristics monitoring trial. Pediatr Res 2013; 74 (05) 570-575
  • 44 Schelonka RL, Carlo WA, Bauer CR. et al. Mortality and neurodevelopmental outcomes in the heart rate characteristics monitoring randomized controlled trial. J Pediatr 2020; 219: 48-53
  • 45 Kappen TH, Vergouwe Y, van Klei WA, van Wolfswinkel L, Kalkman CJ, Moons KGM. Adaptation of clinical prediction models for application in local settings. Med Decis Making 2012; 32 (03) E1-E10
  • 46 Smolen P, Zhang Y, Byrne JH. The right time to learn: mechanisms and optimization of spaced learning. Nat Rev Neurosci 2016; 17 (02) 77-88
  • 47 Wiggins GA, Sanjekdar A. Learning and consolidation as re-representation: revising the meaning of memory. Front Psychol 2019; 10: 802
  • 48 Shachak A, Barnsley J, Tu K, Jadad AR, Lemieux-Charles L. Understanding end-user support for health information technology: a theoretical framework. Inform Prim Care 2011; 19 (03) 169-172