The Clinical Utility of Traditional and Machine Learning Alarms during the Care of Acutely Ill Patients

Abstract

Objectives

Despite low-level evidence, acutely ill patients are often continuously monitored. This creates high false alarm rates and alarm fatigue with unclear clinical effectiveness. We compare metrics, including alarm burden, area under the receiver operator characteristic curve (auROC), sensitivity, and specificity for threshold, score (i.e., National Early Warning Score [NEWS]), and machine learning (ML) alarms.

Methods

We retrospectively annotated continuous biometric data for acutely ill patients receiving hospital care at home for clinical utility (change in clinical management) or a safety composite using the electronic health record. Threshold alarms for heart rate (HR), respiratory rate (RR), and fall were set pragmatically by clinical teams; the score alarm was the NEWS, and the ML alarm was an unsupervised ML algorithm that detected anomalies in HR, RR, and activity. Our primary outcome was alarm burden (alarms/patient-hour). Secondary outcomes included alarm performance.

Results

We studied 526 patients of median age 71 (interquartile range [IQR]: 25), 60.3% female, 45.1% White. Compared with threshold alarms (0.132 alarms/patient-hour), alarm burden was lower with score and ML alarms (0.005 score alarms/patient-hour; 0.032 ML alarms/patient-hour; p < 0.001 for both, compared with threshold). The positive predictive value for identifying clinical utility was 0.073 for threshold, 0.247 for score, and 0.181 for ML. The auROC for identifying the safety composite was 0.557 for threshold, 0.578 for score, and 0.656 for ML.

Conclusion

Score and ML alarms decreased alarm burden with higher overall performance in recognizing clinically important events. Our findings suggest that the use of score or ML alarms holds promise in reducing alarm fatigue while improving recognition of clinically important events, although all alarms require improvement.

Keywords

alert fatigue - artificial intelligence - clinical decision support - remote monitoring - process management tools - clinical information systems - alerting - monitoring and surveillance

Protection of Human and Animal Subjects

The study protocol was prespecified and approved by the Mass General Brigham institutional review board.

Publication History

Received: 20 March 2025

Accepted after revision: 16 February 2026

Article published online:
03 March 2026

• References

• 1 Sendelbach S, Funk M. Alarm fatigue: A patient safety concern. AACN Adv Crit Care 2013; 24 (04) 378-386 , quiz 387–388
• 2 Atzema C, Schull MJ, Borgundvaag B, Slaughter GRD, Lee CK. ALARMED: Adverse events in low-risk patients with chest pain receiving continuous electrocardiographic monitoring in the emergency department. A pilot study. Am J Emerg Med 2006; 24 (01) 62-67
• 3 Bliss JP, Gilson RD, Deaton JE. Human probability matching behaviour in response to alarms of varying reliability. Ergonomics 1995; 38 (11) 2300-2312
• 4 Joint Commission. The Joint Commission Sentinel Event Alert. Accessed February 19, 2026 at: https://digitalassets.jointcommission.org/api/public/content/f65e5c9df2b94000a99445e0a7877007?v=6b2e5ed0
• 5 Yu D, Obuseh M, DeLaurentis P. Quantifying the impact of infusion alerts and alarms on nursing workflows: A retrospective analysis. Appl Clin Inform 2021; 12 (03) 528-538
• 6 Sun C, Bao M, Pu C. et al. Machine alarm fatigue among hemodialysis nurses in 29 tertiary hospitals. Appl Clin Inform 2024; 15 (03) 533-543
• 7 Pettinati MJ, Vattis K, Mitchell H, Rosario NA, Levine DM, Selvaraj N. The role of continuous monitoring in acute-care settings for predicting all-cause 30-day hospital readmission: A pilot study. Heliyon 2025; 11 (02) e41994
• 8 Top 10 health technology hazards: Are you protecting your patients from these high-priority risks?. Health Devices 2007; 36 (11) 345-351
• 9 ECRI Institute. Top 10 Health Technology Hazards for 2020. J Radiol Nurs 2020; 39 (01) 1-16
• 10 ECRI Institute. ECRI Top 10 Health Technology Hazards for 2015. Biomed Saf Stand 2015; 45 (03) 23-24
• 11 Behar J, Oster J, Li Q, Clifford GD. ECG signal quality during arrhythmia and its application to false alarm reduction. IEEE Trans Biomed Eng 2013; 60 (06) 1660-1666
• 12 Liu Z, Khojandi A, Mohammed A. et al. HeMA: A hierarchically enriched machine learning approach for managing false alarms in real time: A sepsis prediction case study. Comput Biol Med 2021; 131: 104255
• 13 Ouchi K, Liu S, Tonellato D, Keschner YG, Kennedy M, Levine DM. Home hospital as a disposition for older adults from the emergency department: Benefits and opportunities. J Am Coll Emerg Physicians Open 2021; 2 (04) e12517
• 14 Levine DM, Ouchi K, Blanchfield B. et al. Hospital-level care at home for acutely ill adults: A pilot randomized controlled trial. J Gen Intern Med 2018; 33 (05) 729-736
• 15 Levine DM, Ouchi K, Blanchfield B. et al. Hospital-level care at home for acutely ill adults: A randomized controlled trial. Ann Intern Med 2020; 172 (02) 77-85
• 16 Levine DM, Mitchell H, Rosario N. et al. Acute care at home during the COVID-19 pandemic surge in Boston. J Gen Intern Med 2021; 36 (11) 3644-3646
• 17 Federman AD, Soones T, DeCherrie LV, Leff B, Siu AL. Association of a bundled hospital-at-home and 30-day postacute transitional care program with clinical outcomes and patient experiences. JAMA Intern Med 2018; 178 (08) 1033-1040
• 18 Levine DM, Pian J, Mahendrakumar K, Patel A, Saenz A, Schnipper JL. Hospital-level care at home for acutely ill adults: A qualitative evaluation of a randomized controlled trial. J Gen Intern Med 2021; 36 (07) 1965-1973
• 19 Levine DM, Desai MP, Ross JB, Como N, Holley S. Scoping and testing rural acute care at home: a simulation analysis. BMJ Innov 2021; 7 (03) 539-547
• 20 Leff B, Burton L, Mader SL. et al. Hospital at home: Feasibility and outcomes of a program to provide hospital-level care at home for acutely ill older patients. Ann Intern Med 2005; 143 (11) 798-808
• 21 Cai S, Intrator O, Chan C. et al. Association of costs and days at home with transfer hospital in home. JAMA Netw Open 2021; 4 (06) e2114920
• 22 Levine DM, DeCherrie LV, Siu A. et al; Hospital at Home Users Group Practice Standards Council. Practice standards for acute hospital care at home. J Am Geriatr Soc 2025; 73 (07) 2037-2045
• 23 Cherukara A, Rudski L, Grinman M, Levine DM. Bringing heart care home: management of acute cardiovascular pathologies in the Home Hospital. Eur Heart J Acute Cardiovasc Care 2025; 14 (06) 364-374
• 24 Levine DM, Findeisen S, Desai MP. et al. Hospital at home worldwide: Program and clinician characteristics from the World Hospital at Home Congress survey. J Am Geriatr Soc 2024; 72 (12) 3824-3832
• 25 Levine DM, Souza J, Schnipper JL, Tsai TC, Leff B, Landon BE. Acute hospital care at home in the United States: The early national experience. Ann Intern Med 2024; 177 (01) 109-110
• 26 Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013; 84 (04) 465-470
• 27 Downey CL, Chapman S, Randell R, Brown JM, Jayne DG. The impact of continuous versus intermittent vital signs monitoring in hospitals: A systematic review and narrative synthesis. Int J Nurs Stud 2018; 84: 19-27
• 28 van Loon K, van Zaane B, Bosch EJ, Kalkman CJ, Peelen LM. Non-invasive continuous respiratory monitoring on general hospital wards: A systematic review. PLoS ONE 2015; 10 (12) e0144626
• 29 Voepel-Lewis T, Parker ML, Burke CN. et al. Pulse oximetry desaturation alarms on a general postoperative adult unit: A prospective observational study of nurse response time. Int J Nurs Stud 2013; 50 (10) 1351-1358
• 30 Watkins T, Whisman L, Booker P. Nursing assessment of continuous vital sign surveillance to improve patient safety on the medical/surgical unit. J Clin Nurs 2016; 25 (1-2): 278-281
• 31 Gross B, Dahl D, Nielsen L. Physiologic monitoring alarm load on medical/surgical floors of a community hospital. Biomed Instrum Technol 2011; 45 (suppl): 29-36
• 32 Brown H, Terrence J, Vasquez P, Bates DW, Zimlichman E. Continuous monitoring in an inpatient medical-surgical unit: A controlled clinical trial. Am J Med 2014; 127 (03) 226-232
• 33 Hravnak M, Devita MA, Clontz A, Edwards L, Valenta C, Pinsky MR. Cardiorespiratory instability before and after implementing an integrated monitoring system. Crit Care Med 2011; 39 (01) 65-72
• 34 Taenzer AH, Pyke JB, McGrath SP, Blike GT. Impact of pulse oximetry surveillance on rescue events and intensive care unit transfers: A before-and-after concurrence study. Anesthesiology 2010; 112 (02) 282-287
• 35 Jeskey M, Card E, Nelson D. et al. Nurse adoption of continuous patient monitoring on acute post-surgical units: Managing technology implementation. J Nurs Manag 2011; 19 (07) 863-875
• 36 Langhorne P, Stott D, Knight A, Bernhardt J, Barer D, Watkins C. Very early rehabilitation or intensive telemetry after stroke: A pilot randomised trial. Cerebrovasc Dis 2010; 29 (04) 352-360
• 37 Banks J, McArthur J, Gordon G. Flexible monitoring in the management of patient care processes: A pilot study. Lippincotts Case Manag 2000; 5 (03) 94-103 , quiz 104–106
• 38 Prgomet M, Cardona-Morrell M, Nicholson M. et al. Vital signs monitoring on general wards: Clinical staff perceptions of current practices and the planned introduction of continuous monitoring technology. Int J Qual Health Care 2016; 28 (04) 515-521
• 39 Welch J. An evidence-based approach to reduce nuisance alarms and alarm fatigue. Biomed Instrum Technol 2011; 45 (suppl): 46-52
• 40 Leenen JPL, Leerentveld C, van Dijk JD, van Westreenen HL, Schoonhoven L, Patijn GA. Current evidence for continuous vital signs monitoring by wearable wireless devices in hospitalized adults: Systematic review. J Med Internet Res 2020; 22 (06) e18636