Prospective Temporal Validation of the Neonatal Bleeding Risk (NeoBRis) Index

Critically ill neonates constitute a very fragile population at high risk of bleeding and in need of transfusions. Prediction scores and bleeding assessment tools, which could help estimating the risk/benefit ratio of transfusions, are crucial for supporting clinical decisions and bleeding management in this vulnerable population.[1] [2] [3] [4] Recently, our research group developed and published in the journal a multivariable prediction tool for 24-hour bleeding risk in critically ill neonates (Neonatal Bleeding Risk [NeoBRis] index).[5] This assessment tool includes extrinsically activated rotational thromboelastometry (ROTEM) parameters (EXTEM A10 and LI60 [amplitude recorded at 10 minutes and lysis index at 60 minutes, respectively]), platelet count, and creatinine plasma levels, and has demonstrated excellent performance (area under the receiver operating characteristic, ROC, curve [AUC]: 0.908; 95% confidence interval [CI]: 0.870–0.946). The aim of the current study was to perform a prospective temporal validation of this prognostic index.

We conducted and reported the study in agreement with the TRIPOD statement.[6] The validation cohort of the NeoBRis index consisted of all consecutive patients admitted to the neonatal intensive care unit (NICU) of General Hospital of Nikaia, Piraeus, Greece, over 6 months, from January 2020 to July 2020. The derivation cohort had been previously described,[5] and consisted of 332 full-term and preterm neonates with sepsis, suspected sepsis, and/or perinatal hypoxia, hospitalized in the same NICU from July 2014 to January 2019. Cohort enrollment followed the same inclusion and exclusion criteria of the derivation cohort, as well as the same sample processing procedures.[5]

To evaluate the performance of the NeoBRis index in the validation cohort, we calculated the score corresponding to each patient by assigning points as reported in the original publication.[5] We assigned to each patient: −10.4 points per each point of A10, +6.4 points per each point of LI60, +250 points if the creatinine level was ≥1.5 mg/dL, +300 points if platelets were <50,000 cells/μL, or, alternatively, +90 points if platelets were between 50,000 and 150,000 cells/μL. We obtained the predicted probability of bleeding by applying logistic regression to the index and performed ROC analyses to calculate the AUC. To assess the overall performance of the model in this new population, we calculated the Brier score. This score can range from 0 (for a perfect model) to 0.25 (for a noninformative model).[7] All tests were two-sided. Stata software was used for statistical modeling and analysis (Stata Corp., College Station, Texas, United States).

This cohort study included 137 consecutive, critically ill, full-term, and preterm neonates. The population consisted predominantly of males (63.5%) with a median gestational age of 37 weeks (interquartile range [IQR]: 33–38) and a median birth weight of 2,560 g (IQR: 1,700–3,170). About half of the included newborns (n = 67; 48.9%) were preterm (<37 weeks of gestation). The mean age was 11.7 days (standard deviation [SD] = 30.3), and the median was 3 days (IQR: 2–10). The baseline characteristics, the biochemical and the hematological measurements, of both the development and validation cohorts are shown in [Table 1]. The validation cohort was composed mostly of patients with suspected sepsis (67.2%), whereas many newborns included in the derivation cohort suffered perinatal hypoxia (50.2%). Out of 137 neonates, 15 (10.9%) experienced bleeding within 24 hours of ROTEM testing (with eight patients experiencing severe bleeding as defined by a NeoBAT [Neonatal Bleeding Assessment Tool][8] score ≥3). The proportion of patients experiencing bleeding in the derivation sample was much higher (34.3%).

Table 1 Characteristics of the derivation and validation cohorts

	Derivation cohort (n = 332)	Validation cohort (n = 137)
	Mean ± SD; median (IQR) or n (%)
Gender (males)	213 (64.2%)	87 (63.5%)
Gestational age (wk)	35.2 ± 4.40; 37 (32–39)	35.3 ± 4.17; 37 (33–38)
Birth weight (g)	2,437 ± 1,015; 2,585 (1,480–3,290)	2,430 ± 896; 2,560 (1,700–3,170)
Cesarean section	216 (65.1%)	100 (73.0%)
Perinatal conditions
Suspected sepsis	46 (13.9%)	92 (67.2%)
Sepsis	121 (36.5%)	9 (6.6%)
Respiratory distress syndrome	169 (50.9%)	75 (54.7%)
Intrauterine growth retardation	41 (12.4%)	18 (13.1%)
Perinatal hypoxia	167 (50.3%)	37 (27%)
Acute renal failure	65 (19.6%)	5 (3.7%)
Disseminated intravascular coagulopathy	65 (19.6%)	10 (7.3%)
Necrotizing enterocolitis	12 (3.6%)	6 (4.4%)
Laboratory parameters
WBC (×103, cells/µL)	14.9 ± 8.22; 13.6 (9.56–18.5)	14.5 ± 13.6; 12.2 (9.13–16.5)
Neutrophils (%)	60.0 ± 17.4; 64 (48.0–73.0)	54.3 ± 17.1; 55 (43.2–67.3)
Platelets (×103, cells/µL)	197 ± 129; 208 (81.5–280)	262 ± 140; 256 (166–341)
C-reactive protein (mg/L)	36.0 ± 46.8; 15.6 (3.40–52.5)	26.8 ± 41.3; 9.6 (3.6–26.1)
SGOT (IU/L)	116 ± 256; 60.0 (38.0–100)	130 ± 621; 47 (31.0–72.0)
SGPT (IU/L)	47.9 ± 113; 19.0 (13.0–36.0)	48.4 ± 152; 19.0 (11.0–35.0)
Total bilirubin (mg/dL)	8.21 ± 7.34; 6.35 (4.40–9.90)	6.00 ± 4.21; 5.40 (3.10–7.60)
Direct bilirubin (mg/dL)	1.81 ± 5.17; 0.30 (0.20–0.50)	0.44 ± 0.99; 0.30 (0.20–0.40)
Blood urea nitrogen (mg/dL)	44.5 ± 39.5; 33.0 (20.0–55.0)	30.7 ± 19.1; 26.0 (18.0–39.0)
Creatinine (mg/dL)	0.69 ± 0.47; 0.60 (0.40–0.90)	0.66 ± 0.53; 0.60 (0.40–0.80)
EXTEM ROTEM parameters
CT	100 ± 451; 54.0 (46.0–67.0)	57.0 ± 40.5; 50.0 (44.0–57.0)
A10	47.6 ± 15.1; 52.0 (37.0–59.0)	53.3 ± 15.0; 55.0 (44.0–65.0)
A20	53.0 ± 15.0; 57.0 (44.0–63.5)	57.8 ± 14.0; 60.0 (49.0–69.0)
A30	54.1 ± 14.8; 57.0 (45.0–64.0)	58.3 ± 13.7; 61.0 (50.0–68.0)
CFT	208 ± 585; 93.5 (67.5–157)	131 ± 227; 77.0 (51.5–112)
MCF, mm	55.6 ± 15.1; 58.0 (47.0–65.0)	59.3 ± 14.3; 61.0 (50.0–69.0)
ALPHA-angle	71.1 ± 11.5; 74.0 (68.0–78.0)	72.8 ± 13.5; 77.0 (70.0–81.0)
LI60 (%)	94.2 ± 6.48; 95.0 (92.0–98.0)	93.8 ± 7.74; 95.0 (92.0–98.0)
ML (%)	9.84 ± 12.0; 8.00 (2.00–12.00)	10.3 ± 9.73; 9.00 (5.00–13.00)
Type of hemorrhage (clinical outcome)
Intraventricular hemorrhage	96 (28.9%)	14 (10.2%)
Gastrointestinal hemorrhage	39 (11.8%)	3 (2.2%)
Pulmonary hemorrhage	15 (4.5%)	6 (4.4%)
Venipuncture-site hemorrhage	82 (24.7%)	15 (11.0%)
Urinary tract hemorrhage	13 (3.9%)	2 (1.5%)

Abbreviations: A10, clot amplitude at 10 minutes; A20, clot amplitude at 20 minutes; A30, clot amplitude at 30 minutes; CFT, clot formation time; CT, clotting time; IQR, interquartile range; LI60, lysis index at 60 minutes; MCF, maximum clot firmness; ML, maximal lysis; SD, standard deviation.

After applying the scoring algorithm to the patients of the validation cohort (n = 134; missing data prevented computing the index in three patients), we plotted the AUC ([Fig. 1]). We obtained an AUC of 0.938 (95% CI: 0.874–0.999) in agreement with the AUC of 0.908 (95% CI: 0.870–0.946) observed in the derivation sample, which indicates an excellent model discrimination. The Brier score was close to zero (0.0582), indicating optimal performance.

Fig. 1 Area under the receiver-operating characteristic (ROC) curve of the NeoBRis index in the validation cohort.

Although the NeoBRis score was developed as a tool to predict the immediate (i.e., 24 hours) risk of any bleeding event, we tested its capacity to predict severe bleeding events (i.e., NeoBAT score ≥ 3). We obtained an excellent AUC of 0.952 (95% CI: 0.911–0.993), suggesting that the NeoBRis score can accurately predict also the risk of severe bleeding.

The prospective temporal validation of the NeoBRis index corroborated the initial excellent performance of our prediction model on a subsequent population of critically ill neonates admitted to the same NICU. Of note, the incidence of bleeding was much higher in the derivation sample (34.3%) as compared with the validation cohort (10.9%). In fact, the validation cohort mostly consisted of neonates with suspected sepsis, whereas the derivation cohort included mostly neonates with confirmed sepsis or perinatal hypoxia, clinical states that have been associated with coagulation abnormalities.[9] [10] This further strengthens the validation process, which has been performed on an effectively different population of patients.

Most of previously developed prediction models for risk assessment of hemorrhage in neonates use only clinical variables and exclusively allow for a baseline risk assessment.[11] [12] [13] The disadvantage of baseline prediction models is that they do not take into account the clinical course of the neonate, which can change substantially over time, and may have a profound impact on the bleeding risk. Recently, Fustolo-Gunnink et al[2] developed a dynamic prediction model for bleeding risk in thrombocytopenic preterm neonates including platelet count and clinical variables. Thrombocytopenia is an established risk factor for clinically significant bleeding in neonates; however, a poor association between platelet count and hemorrhage in neonates has been reported.[9] [10] [14] [15] There is insufficient evidence to assess whether platelet counts are causally related to major bleeding, or whether platelet transfusions reduce bleeding risk in thrombocytopenic preterm neonates.[15] Moreover, the results of a recent randomized trial showed that prophylactic transfusion of platelets below the threshold of 50 × 10⁹/L was associated with an increased risk of bleeding and mortality as compared with a threshold of 25 × 10⁹/L.[3] Current evidence supports that several other factors may have an impact on the neonatal bleeding risk,[14] [16] thus de-emphasizing the role of platelet count alone and highlighting the need for improved and individualized guidance on platelet transfusion in neonates. At this point, we should note that the NeoBRis index was developed to evaluate the immediate risk of any bleeding; decisions about the best threshold to apply platelet transfusions to critically ill neonates are premature. Establishing which threshold is the most safe and effective to reduce morbidity and mortality in these neonates needs large and well-designed, comparative prospective studies.

The NeoBRis index, by including platelet counts, creatinine plasma levels, and EXTEM variables (A10, LI60), may also provide an insight into the hemostatic profile of critically ill neonates. Although this index was developed as a tool to predict any bleeding event, suggestive evidence shows that it can well predict also severe bleeding. If confirmed in larger populations, this finding would be of high clinical relevance. The validation process is the crucial step to allow widespread clinical practice application. Although the temporal validation has confirmed excellent performance of this tool, further external validation by other NICUs is needed before the NeoBRis index can support daily clinical practice.

Conflict of Interest

None declared.

^* Both authors have equal contribution.

• References

• 1 Motta M, Del Vecchio A, Chirico G. Fresh frozen plasma administration in the neonatal intensive care unit: evidence-based guidelines. Clin Perinatol 2015; 42 (03) 639-650
• 2 Fustolo-Gunnink SF, Fijnvandraat K, Putter H. et al. Dynamic prediction of bleeding risk in thrombocytopenic preterm neonates. Haematologica 2019; 104 (11) 2300-2306
• 3 Curley A, Stanworth SJ, Willoughby K. et al; PlaNeT2 MATISSE Collaborators. Randomized trial of platelet-transfusion thresholds in neonates. N Engl J Med 2019; 380 (03) 242-251
• 4 Goel R, Josephson C. Recent advances in transfusions in neonates/infants. F1000Res 2018; 7: F1000
• 5 Sokou R, Piovani D, Konstantinidi A. et al. A risk score for predicting the incidence of hemorrhage in critically ill neonates: development and validation study. Thromb Haemost 2021; 121 (02) 131-139
• 6 Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015; 350: g7594
• 7 Steyerberg EW, Vickers AJ, Cook NR. et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21 (01) 128-138
• 8 Venkatesh V, Curley A, Khan R. et al. A novel approach to standardised recording of bleeding in a high risk neonatal population. Arch Dis Child Fetal Neonatal Ed 2013; 98 (03) F260-F263
• 9 Sokou R, Giallouros G, Konstantinidi A. et al. Thromboelastometry for diagnosis of neonatal sepsis-associated coagulopathy: an observational study. Eur J Pediatr 2018; 177 (03) 355-362
• 10 Konstantinidi A, Sokou R, Tsantes AG. et al. Thromboelastometry variables in neonates with perinatal hypoxia. Semin Thromb Hemost 2020; 46 (04) 428-434
• 11 Luque MJ, Tapia JL, Villarroel L. et al; Neocosur Neonatal Network. A risk prediction model for severe intraventricular hemorrhage in very low birth weight infants and the effect of prophylactic indomethacin. J Perinatol 2014; 34 (01) 43-48
• 12 Singh R, Visintainer PF. Predictive models for severe intraventricular hemorrhage in preterm infants. J Perinatol 2014; 34 (10) 802-802
• 13 Gleissner M, Jorch G, Avenarius S. Risk factors for intraventricular hemorrhage in a birth cohort of 3721 premature infants. J Perinat Med 2000; 28 (02) 104-110
• 14 Deschmann E, Saxonhouse MA, Feldman HA, Norman M, Barbian M, Sola-Visner M. Association of bleeding scores and platelet transfusions with platelet counts and closure times in response to adenosine diphosphate (CT-ADPs) among preterm neonates with thrombocytopenia. JAMA Netw Open 2020; 3 (04) e203394-e203394
• 15 Fustolo-Gunnink SF, Huisman EJ, van der Bom JG. et al. Are thrombocytopenia and platelet transfusions associated with major bleeding in preterm neonates? A systematic review. Blood Rev 2019; 36: 1-9
• 16 Muthukumar P, Venkatesh V, Curley A. et al; Platelets Neonatal Transfusion Study Group. Severe thrombocytopenia and patterns of bleeding in neonates: results from a prospective observational study and implications for use of platelet transfusions. Transfus Med 2012; 22 (05) 338-343

Publikationsverlauf

Eingereicht: 13. November 2020

Angenommen: 23. Dezember 2020

Accepted Manuscript online:
24. Dezember 2020

Artikel online veröffentlicht:
16. Februar 2021