Improving Stroke Risk Prediction in the General Population: A Comparative Assessment of Common Clinical Rules, a New Multimorbid Index, and Machine-Learning-Based Algorithms

Abstract

Background There are few large studies examining and predicting the diversified cardiovascular/noncardiovascular comorbidity relationships with stroke. We investigated stroke risks in a very large prospective cohort of patients with multimorbidity, using two common clinical rules, a clinical multimorbid index and a machine-learning (ML) approach, accounting for the complex relationships among variables, including the dynamic nature of changing risk factors.

Methods We studied a prospective U.S. cohort of 3,435,224 patients from medical databases in a 2-year investigation. Stroke outcomes were examined in relationship to diverse multimorbid conditions, demographic variables, and other inputs, with ML accounting for the dynamic nature of changing multimorbidity risk factors, two clinical risk scores, and a clinical multimorbid index.

Results Common clinical risk scores had moderate and comparable c indices with stroke outcomes in the training and external validation samples (validation—CHADS₂: c index 0.812, 95% confidence interval [CI] 0.808–0.815; CHA₂DS₂-VASc: c index 0.809, 95% CI 0.805–0.812). A clinical multimorbid index had higher discriminant validity values for both the training/external validation samples (validation: c index 0.850, 95% CI 0.847–0.853). The ML-based algorithms yielded the highest discriminant validity values for the gradient boosting/neural network logistic regression formulations with no significant differences among the ML approaches (validation for logistic regression: c index 0.866, 95% CI 0.856–0.876). Calibration of the ML-based formulation was satisfactory across a wide range of predicted probabilities. Decision curve analysis demonstrated that clinical utility for the ML-based formulation was better than that for the two current clinical rules and the newly developed multimorbid tool. Also, ML models and clinical stroke risk scores were more clinically useful than the “treat all” strategy.

Conclusion Complex relationships of various comorbidities uncovered using a ML approach for diverse (and dynamic) multimorbidity changes have major consequences for stroke risk prediction. This approach may facilitate automated approaches for dynamic risk stratification in the significant presence of multimorbidity, helping in the decision-making process for risk assessment and integrated/holistic management.

Keywords

machine learning - cardiovascular/noncardiovascular multimorbidity - stroke

Data Availability

Data are available as presented in the article. According to U.S. laws and corporate agreements, our own approvals to use the Anthem and IngenioRx data sources for the current study do not allow us to distribute or make patient data directly available to other parties.

The review process for this paper was fully handled by Christian Weber, Editor in Chief.

Publication History

Received: 19 March 2021

Accepted: 23 March 2021

Accepted Manuscript online:
25 March 2021

Article published online:
28 May 2021

• References

• 1 Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet 2012; 380 (9836): 37-43
• 2 Muth C, Blom JW, Smith SM. et al. Evidence supporting the best clinical management of patients with multimorbidity and polypharmacy: a systematic guideline review and expert consensus. J Intern Med 2019; 285 (03) 272-288
• 3 Sposato LA, Lam M, Allen B, Richard L, Shariff SZ, Saposnik G. First-ever ischemic stroke and increased risk of incident heart disease in older adults. Neurology 2020; 94 (15) e1559-e1570
• 4 Borre ED, Goode A, Raitz G. et al. Predicting thromboembolic and bleeding event risk in patients with non-valvular atrial fibrillation: a systematic review. Thromb Haemost 2018; 118 (12) 2171-2187
• 5 Tajeu GS, Booth III JN, Colantonio LD. et al. Incident cardiovascular disease among adults with blood pressure <140/90 mm Hg. Circulation 2017; 136 (09) 798-812
• 6 Chao TF, Liao JN, Tuan TC. et al. Incident co-morbidities in patients with atrial fibrillation initially with a CHA2DS2-VASc score of 0 (males) or 1 (females): implications for reassessment of stroke risk in initially ‘low-risk’ patients. Thromb Haemost 2019; 119 (07) 1162-1170
• 7 Chao TF, Lip GYH, Lin YJ. et al. Incident risk factors and major bleeding in patients with atrial fibrillation treated with oral anticoagulants: a comparison of baseline, follow-up and delta HAS-BLED scores with an approach focused on modifiable bleeding risk factors. Thromb Haemost 2018; 118 (04) 768-777
• 8 Vetrano DL, Roso-Llorach A, Fernández S. et al. Twelve-year clinical trajectories of multimorbidity in a population of older adults. Nat Commun 2020; 11 (01) 3223
• 9 Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 2014; 35 (29) 1925-1931
• 10 Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 2014; 33 (03) 517-535
• 11 Grant SW, Collins GS, Nashef SAM. Statistical primer: developing and validating a risk prediction model. Eur J Cardiothorac Surg 2018; 54 (02) 203-208
• 12 Pencina MJ, D'Agostino Sr RB. Evaluating discrimination of risk prediction models: the C statistic. JAMA 2015; 314 (10) 1063-1064
• 13 Gallacher KI, Jani BD, Hanlon P, Nicholl BI, Mair FS. Multimorbidity in stroke. Stroke 2019; 50 (07) 1919-1926
• 14 Marengoni A, Roso-Llorach A, Vetrano DL. et al. Patterns of multimorbidity in a population-based cohort of older people: sociodemographic, lifestyle, clinical, and functional differences. J Gerontol A Biol Sci Med Sci 2020; 75 (04) 798-805
• 15 Guo Y, Lane DA, Chen Y, Lip GYH. Mobile health technology facilitates population screening and integrated care management in patients with atrial fibrillation. Eur Heart J 2020; 41 (17) 1617-1619
• 16 Guo Y, Guo J, Shi X. et al; mAF-App II Trial investigators. Mobile health technology-supported atrial fibrillation screening and integrated care: a report from the mAFA-II trial Long-term Extension Cohort. Eur J Intern Med 2020; 82: 105-111