External Validation of the OAC³-PAD Risk Score for Patients Treated with Lower Extremity Endovascular Revascularisation

Introduction Various bleeding risk scores have been conceived to assess the bleeding risk in patients with cardiac disease, however the OAC³-PAD bleeding risk score is only the second one specifically designed to assess the bleeding risk in patients with peripheral arterial disease (PAD) [1]. We aimed to externally validate the OAC³-PAD risk score for PAD patients treated exclusively with endovascular revascularisation.

Method We retrospectively reviewed the medical documentation of all 1 612 patients who underwent successful endovascular revascularisation at the Clinical Department of Vascular Diseases at the University Medical Centre Ljubljana in a 5-year period. Performance of the risk score was tested in two steps [2]. We first assessed the Cox proportional hazards (CPH) model upon which the score is based using discrimination, calibration, and a scaled Brier score. Secondly, we tested the risk score itself by calculating the patients’ risk scores and grouping patients into the four risk groups. Survival analysis was performed using Kaplan-Meier curves and discrimination was tested using log-rank analysis.

Results Uno’s IPCW corrected AUC, a measure of discrimination of the CPH model, was 0.83 (95% CI 0.76 – 0.90). The scaled Brier score was 3.27% (95% CI 0.65% – 4.40%), indicating poor overall performance. 1 434 patients were included in the main analysis, of whom 33 (2.3%) experienced a major bleeding event. Major bleeding rates in the low, low-to-moderate, moderate-to-high, and high risk groups were 0.4% (3/736 patients), 0.8% (2/243 patients), 5.8% (15/258 patients), and 6.6% (13/197 patients), respectively. The Kaplan-Meier curves presenting the event-free survival of patients for all four groups are shown in [Fig. 1]. Observed and expected event rates are shown in [Fig. 2].

Fig. 1 Kaplan-Meier curves for the four bleeding risk groups in our patient cohort.; Kaplan-Meier curves showing the event-free survival of our patients in the low, low-to-moderate, moderate-to-high, and high risk group as determined by the OAC³-PAD risk score.

Fig. 2 Observed and expected major bleeding rates as obtained from the Kaplan-Meier curves and th; Observed 1-year major bleeding rates as obtained from the Kaplan-Meier curves for our validation cohort and expected 1-year major bleeding rates as obtained from the CPH model upon which the OAC³-PAD risk score is based.

Log-rank analysis showed successful discrimination between either of the two lower risk groups and one of the higher risk groups: low vs moderate-to-high risk (p<.001), low vs high risk (p<.001), low-to-moderate vs moderate-to-high risk (p=.003), and low-to-moderate vs high risk (p<.001). However, the score failed to discriminate between the low and low-to-moderate, as well as between the moderate-to-high and high risk group.

Conclusion While discrimination of the OAC³-PAD CPH model was adequate, calibration was poor. The score overestimated the bleeding risk in low-risk patients while underestimating the risk in high-risk patients. Although the score failed to stratify PAD patients after percutaneous revascularisation into the four respective risk groups, it can be clinically useful in distinguishing low risk patients from high risk patients.

• References

• 1 Behrendt C-A, Kreutzburg T, Nordanstig J, Twine CP, Marschall U, Kakkos S. et al. The OAC3-PAD Risk Score Predicts Major Bleeding Events one Year after Hospitalisation for Peripheral Artery Disease. European Journal of Vascular and Endovascular Surgery 2022; 63 (03) 503-10
• 2 McLernon DJ, Giardiello D, Van Calster B, Wynants L, van Geloven N, van Smeden M. et al. Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models. Ann Intern Med 2023; 176 (01) 105-14