Abstract
Objective A recent study leveraging machine learning methods found that postpartum hemorrhage
(PPH) can be predicted accurately at the time of labor admission in the U.S. Consortium
for Safe Labor (CSL) dataset, with a C-statistic as high as 0.93. These CSL models
were developed in older data (2002–2008) and used an estimated blood loss (EBL) of
≥1,000 mL to define PPH. We sought to externally validate these models using a more
recent cohort of births where blood loss was measured using quantitative blood loss
(QBL) methods.
Study Design Using data from 5,261 deliveries between February 1, 2019 and May 11, 2020 at a single
tertiary hospital, we mapped our electronic health record (EHR) data to the 55 predictors
described in previously published CSL models. PPH was defined as QBL ≥1,000 mL within
24 hours after delivery. Model discrimination and calibration of the four CSL models
were measured using our cohort. In a secondary analysis, we fit new models in our
study cohort using the same predictors and algorithms as the original CSL models.
Results The original study cohort had a substantially lower rate of PPH, 4.8% (7,279/228,438)
versus 25% (1,321/5,261), possibly due to differences in measurement. The CSL models
had lower discrimination in our study cohort, with a C-statistic as high as 0.57 (logistic
regression). Models refit in our study cohort achieved better discrimination, with
a C-statistic as high as 0.64 (random forest). Calibration improved in the refit models
as compared with the original models.
Conclusion The CSL models' accuracy was lower in a contemporary EHR where PPH is assessed using
QBL. As institutions continue to adopt QBL methods, further data are needed to understand
the differences between EBL and QBL to enable accurate prediction of PPH.
Key Points
-
Machine learning methods may help predict PPH.
-
EBL models do not generalize when QBL is used.
-
Blood loss estimation alters model accuracy.
Keywords
electronic health record - external validation - machine learning - postpartum hemorrhage
- prediction model
