PDF herunterladen
Semin Neurol
DOI: 10.1055/a-2769-6703
Review Article

Operationalizing AI in Stroke Alerts: Balancing Sensitivity and Specificity in Predicting Acute Cerebrovascular Disease

Autor*innen

  • Margaret H. Downes

    1   Icahn School of Medicine at Mount Sinai, New York, New York, United States

  • Asala N. Erekat

    2   Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, New York, United States
    3   Clinical Neuro-informatics Center, Icahn School of Medicine at Mount Sinai, New York, New York, United States

  • Benjamin R. Kummer 

    2   Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, New York, United States
    3   Clinical Neuro-informatics Center, Icahn School of Medicine at Mount Sinai, New York, New York, United States
    4   Windreich Department of Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai, New York, New York, United States
    5   Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai, New York, New York, United States
Weitere Informationen
   Lizenzen und Reprints

Abstract

Acute stroke alerts are frequently triggered by conditions unrelated to cerebrovascular disease, resulting in false positives that burden clinical teams and contribute to diagnostic ambiguity. At a large academic center, we developed ScanNER v2, a machine learning (ML) model based on large-language models (LLMs) and structured clinical data to predict the presence of acute cerebrovascular disease (ACD) in approximately 16,000 stroke alerts occurring over 10 years with an area under the receiver-operating curve and F1 score of 0.72 and overall positive predictive value of 0.68. In this perspective, we outline a practical framework for operationalizing this model within hospital-based stroke systems. We first describe our health-system experience developing and validating an AI-enabled pipeline, named “ScanNER 2,” then take the point of view of two implementation angles (high sensitivity and high specificity), outlining the operational and clinical tradeoffs for each approach. We also highlight challenges related to implementation, clinical governance, workflow integration, and equity, emphasizing guardrails required for responsible deployment. As stroke centers increasingly adopt AI-assisted tools, this type of thought experiment is essential to ensure that such ML-based innovations effectively enhance the core mission of delivering timely, high-quality acute stroke care.

Keywords

machine learning - LLM - large language model - implementation - acute stroke - predictive modeling - diagnostic acceleration


Publikationsverlauf

Eingereicht: 20. November 2025

Angenommen: 09. Dezember 2025

Accepted Manuscript online:
11. Dezember 2025

Artikel online veröffentlicht:
23. Dezember 2025

© 2025. Thieme. All rights reserved.

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

 

  • References

  • 1 Powers WJ, Rabinstein AA, Ackerson T. et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2019; 50 (12) e344-e418 Erratum in: Stroke. 2019;50(12):e440-e441
    Suche in Google Scholar
  • 2 Kim T, Jeong HY, Suh GJ. Clinical differences between stroke and stroke, mimics in code stroke patients. J Korean Med Sci 2022; 37 (07) e54
    Suche in Google Scholar
  • 3 Aroor S, Singh R, Goldstein LB. BE-FAST (Balance, Eyes, Face, Arm, Speech, Time): reducing the proportion of strokes missed using the FAST mnemonic. Stroke 2017; 48 (02) 479-481
    Suche in Google Scholar
  • 4 Cumbler E, Simpson J. Code stroke: multicenter experience with in-hospital stroke alerts. J Hosp Med 2015; 10 (03) 179-183
    Suche in Google Scholar
  • 5 Erekat AN, Stein LDelman BN. et al. Natural language processing to automate cerebrovascular event identification in stroke alerts. Stroke Vasc Intervent Neurol 2025; 5 (06) e001846
    Suche in Google Scholar
  • 6 Long B, Koyfman A. Clinical mimics: an emergency medicine–focused review of stroke mimics. J Emerg Med 2017; 52 (02) 176-183
    Suche in Google Scholar
  • 7 Appukutty AJ, Skolarus LE, Springer MV, Meurer WJ, Burke JF. Increasing false positive diagnoses may lead to overestimation of stroke incidence, particularly in the young: a cross-sectional study. BMC Neurol 2021; 21 (01) 152
    Suche in Google Scholar
  • 8 Erekat AN, Downes MH, Stein LK. et al. Predicting acute cerebrovascular events in stroke alerts using large-language models and structured data, medArxiv.org. 2025 . Preprint
    Suche in Google Scholar
  • 9 Trevethan R. Sensitivity, specificity, and predictive values: foundations and pitfalls. Front Public Health 2017; 5: 307
    Suche in Google Scholar
  • 10 Sung SF, Hung LC, Hu YH. Developing a stroke alert trigger for clinical decision support at emergency triage using machine learning. Int J Med Inform 2021; 152: 104505
    Suche in Google Scholar
  • 11 Abedi V, Misra D, Chaudhary D. et al. Machine learning–based prediction of stroke in emergency departments. Ther Adv Neurol Disord 2024;17:17562864241239108
  • 12 Maxim LD, Niebo R, Utell MJ. Screening tests: a review with examples. Inhal Toxicol 2014; 26 (13) 811-828
    Suche in Google Scholar
  • 13 Saver JL, Fonarow GC, Smith EE. et al. Time to treatment with intravenous tissue plasminogen activator and outcome from acute ischemic stroke. JAMA 2013; 309 (23) 2480-2488
    Suche in Google Scholar
  • 14 Wang F, Beecy A. Implementing AI models in clinical workflows: a roadmap. BMJ Evid Based Med 2025; 30 (05) 285-287
    Suche in Google Scholar
  • 15 Kwong JCC, Erdman L, Khondker A. et al. The silent trial - the bridge between bench-to-bedside clinical AI applications. Front Digit Health 2022;4:929508
  • 16 Cai T, Ni H, Yu M. et al. DeepStroke: an efficient stroke screening framework for emergency rooms with multimodal adversarial deep learning. Med Image Anal 2022; 80: 102522
    Suche in Google Scholar
  • 17 Albrecht M, Shanks D, Shah T. et al. Enhancing clinical documentation with ambient artificial intelligence: a quality improvement survey assessing clinician perspectives on work burden, burnout, and job satisfaction. JAMIA Open 2025; 8 (01) ooaf013
    Suche in Google Scholar
  • 18 Rosenthal J, Beecy A, Sabuncu MR. Rethinking clinical trials for medical AI with dynamic deployments of adaptive systems. NPJ Digit Med 2025; 8 (01) 252
    Suche in Google Scholar