Interpretable Machine Learning–Based Prediction of Intraoperative Cerebrospinal Fluid Leakage in Endoscopic Transsphenoidal Pituitary Surgery: A Pilot Study

Pier Paolo Mattogno; Valerio M. Caccavella; Martina Giordano; Quintino G. D'Alessandris; Sabrina Chiloiro; Leonardo Tariciotti; Alessandro Olivi; Liverana Lauretti

doi:10.1055/s-0041-1740621

Journal of Neurological Surgery Part B: Skull Base, Table of Contents

J Neurol Surg B Skull Base 2022; 83(05): 485-495
DOI: 10.1055/s-0041-1740621

Original Article

Interpretable Machine Learning–Based Prediction of Intraoperative Cerebrospinal Fluid Leakage in Endoscopic Transsphenoidal Pituitary Surgery: A Pilot Study

Pier Paolo Mattogno‡^*

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Valerio M. Caccavella ‡^*

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Martina Giordano

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Quintino G. D'Alessandris

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Sabrina Chiloiro

²Department of Endocrinology, Fondazione Policlinico Universitario A. Gemelli Istituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Leonardo Tariciotti

³Unit of Neurosurgery, Fondazione Istituto di Ricovero e Cura a Carattere Scientifico Cà Granda Ospedale Maggiore Policlinico, Milan, Italy

⁴University of Milan, Milan, Italy

,

Alessandro Olivi

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

,

Liverana Lauretti

¹Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy

› Author Affiliations

Abstract

Purpose Transsphenoidal surgery (TSS) for pituitary adenomas can be complicated by the occurrence of intraoperative cerebrospinal fluid (CSF) leakage (IOL). IOL significantly affects the course of surgery predisposing to the development of postoperative CSF leakage, a major source of morbidity and mortality in the postoperative period. The authors trained and internally validated the Random Forest (RF) prediction model to preoperatively identify patients at high risk for IOL. A locally interpretable model-agnostic explanations (LIME) algorithm is employed to elucidate the main drivers behind each machine learning (ML) model prediction.

Methods The data of 210 patients who underwent TSS were collected; first, risk factors for IOL were identified via conventional statistical methods (multivariable logistic regression). Then, the authors trained, optimized, and audited a RF prediction model.

Results IOL reported in 45 patients (21.5%). The recursive feature selection algorithm identified the following variables as the most significant determinants of IOL: Knosp's grade, sellar Hardy's grade, suprasellar Hardy's grade, tumor diameter (on X, Y, and Z axes), intercarotid distance, and secreting status (nonfunctioning and growth hormone [GH] secreting). Leveraging the predictive values of these variables, the RF prediction model achieved an area under the curve (AUC) of 0.83 (95% confidence interval [CI]: 0.78; 0.86), significantly outperforming the multivariable logistic regression model (AUC = 0.63).

Conclusion A RF model that reliably identifies patients at risk for IOL was successfully trained and internally validated. ML-based prediction models can predict events that were previously judged nearly unpredictable; their deployment in clinical practice may result in improved patient care and reduced postoperative morbidity and healthcare costs.

Keywords

random forest - cerebrospinal fluid leak - machine learning - pituitary adenoma - transsphenoidal surgery - pituitary surgery

Full Text

References

References
1 Chen CJ, Ironside N, Pomeraniec IJ. et al. Microsurgical versus endoscopic transsphenoidal resection for acromegaly: a systematic review of outcomes and complications. Acta Neurochir (Wien) 2017; 159 (11) 2193-2207
2 Dhandapani S, Singh H, Negm HM, Cohen S, Anand VK, Schwartz TH. Cavernous sinus invasion in pituitary adenomas: systematic review and pooled data meta-analysis of radiologic criteria and comparison of endoscopic and microscopic surgery. World Neurosurg 2016; 96: 36-46
3 Fatemi N, Dusick JR, Mattozo C. et al. Pituitary hormonal loss and recovery after transsphenoidal adenoma removal. Neurosurgery 2008; 63 (04) 709-718 , discussion 718–719
4 Przybylowski CJ, Dallapiazza RF, Williams BJ. et al. Primary versus revision transsphenoidal resection for nonfunctioning pituitary macroadenomas: matched cohort study. J Neurosurg 2017; 126 (03) 889-896
5 Strickland BA, Lucas J, Harris B. et al. Identification and repair of intraoperative cerebrospinal fluid leaks in endonasal transsphenoidal pituitary surgery: surgical experience in a series of 1002 patients. J Neurosurg 2018; 129 (02) 425-429
6 Zhang C, Ding X, Lu Y, Hu L, Hu G. Cerebrospinal fluid rhinorrhoea following transsphenoidal surgery for pituitary adenoma: experience in a Chinese centre. Acta Otorhinolaryngol Ital 2017; 37 (04) 303-307
7 Karnezis TT, Baker AB, Soler ZM. et al. Factors impacting cerebrospinal fluid leak rates in endoscopic sellar surgery. Int Forum Allergy Rhinol 2016; 6 (11) 1117-1125
8 Mehta GU, Oldfield EH. Prevention of intraoperative cerebrospinal fluid leaks by lumbar cerebrospinal fluid drainage during surgery for pituitary macroadenomas. J Neurosurg 2012; 116 (06) 1299-1303
9 Conger A, Zhao F, Wang X. et al. Evolution of the graded repair of CSF leaks and skull base defects in endonasal endoscopic tumor surgery: trends in repair failure and meningitis rates in 509 patients. J Neurosurg 2018; 130 (03) 861-875
10 Zhou Q, Yang Z, Wang X. et al. Risk factors and management of intraoperative cerebrospinal fluid leaks in endoscopic treatment of pituitary adenoma: analysis of 492 patients. World Neurosurg 2017; 101: 390-395
11 Jakimovski D, Bonci G, Attia M. et al. Incidence and significance of intraoperative cerebrospinal fluid leak in endoscopic pituitary surgery using intrathecal fluorescein. World Neurosurg 2014; 82 (3,4): e513-e523
12 Senders JT, Staples PC, Karhade AV. et al. Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg 2018; 109: 476-486.e1
13 Staartjes VE, Serra C, Muscas G. et al. Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. Neurosurg Focus 2018; 45 (05) E12
14 Staartjes VE, Zattra CM, Akeret K. et al. Neural network-based identification of patients at high risk for intraoperative cerebrospinal fluid leaks in endoscopic pituitary surgery. J Neurosurg 2019; (e-pub ahead of print)
15 Ribeiro MT, Singh S, Guestrin C. Why Should I Trust You?”: Explaining the predictions of any classifier. Accessed December 2, 2021 at: https://aclanthology.org/N16-3020.pdf
16 Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015; 162 (01) 55-63
17 Breiman L. Random forests. Mach Learn 2001; 45: 5-32
18 Liaw A, Wiener M. Classification and regression by random forest. R News 2002; 2: 18-22
19 Cutler A, Cutler DR, Stevens JR. Random forests. In: Zhang C, Ma Y. eds. Ensemble Machine Learning. Boston, MA: Springer; 2012: 157-175
20 Kursa MB, Rudnicki WR. Feature selection with the Boruta package. J Stat Softw 2010; 36 (11) 1-13
21 Degenhardt F, Seifert S, Szymczak S. Evaluation of variable selection methods for random forests and omics data sets. Brief Bioinform 2019; 20 (02) 492-503
22 Sakr S, Elshawi R, Ahmed AM. et al. Comparison of machine learning techniques to predict all-cause mortality using fitness data: the Henry ford exercIse testing (FIT) project. BMC Med Inform Decis Mak 2017; 17 (01) 174
23 Chawla NV, Bowyer K, Hall LO. et al. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 2002; 16 (01) 321-357
24 DiCiccio TJ, Efron B. Bootstrap confidence intervals. Institute of Mathematical Statistics 1996; 11 (03) 189-228
25 Park JH, Choi JH, Kim YI, Kim SW, Hong YK. Modified graded repair of cerebrospinal fluid leaks in endoscopic endonasal transsphenoidal surgery. J Korean Neurosurg Soc 2015; 58 (01) 36-42
26 Hadad G, Bassagasteguy L, Carrau RL. et al. A novel reconstructive technique after endoscopic expanded endonasal approaches: vascular pedicle nasoseptal flap. Laryngoscope 2006; 116 (10) 1882-1886
27 Cavallo LM, Solari D, Somma T, Cappabianca P. The 3F (fat, flap, and flash) technique for skull base reconstruction after endoscopic endonasal suprasellar approach. World Neurosurg 2019; 126: 439-446
28 Ostrom QT, Gittleman H, Xu J. et al. CBTRUS statistical report: Primary brain and other central nervous system tumors diagnosed in the United States in 2009–2013. Neuro Oncol 2017; 18 (Suppl. 05) v1-v75
29 Komotar RJ, Starke RM, Raper DM, Anand VK, Schwartz TH. Endoscopic endonasal compared with microscopic transsphenoidal and open transcranial resection of giant pituitary adenomas. Pituitary 2012; 15 (02) 150-159
30 Patel PN, Stafford AM, Patrinely JR. et al. Risk Factors for Intraoperative and postoperative cerebrospinal fluid leaks in endoscopic transsphenoidal sellar surgery. Otolaryngol Head Neck Surg 2018; 158 (05) 952-960
31 Xue H, Wang X, Yang Z, Bi Z, Liu P. Risk factors and outcomes of cerebrospinal fluid leak related to endoscopic pituitary adenoma surgery. Br J Neurosurg 2020; 34 (04) 447-452
32 Cavallo LM, Messina A, Cappabianca P. et al. Endoscopic endonasal surgery of the midline skull base: anatomical study and clinical considerations. Neurosurg Focus 2005; 19 (01) E2
33 Fleseriu M, Karavitaki N. Non-functioning pituitary adenomas, not all the same and certainly not boring!. Pituitary 2018; 21: 109-110
34 Snow RB, Johnson CE, Morgello S, Lavyne MH, Patterson Jr RH. Is magnetic resonance imaging useful in guiding the operative approach to large pituitary tumors?. Neurosurgery 1990; 26 (05) 801-803
35 Wei L, Lin SA, Fan K, Xiao D, Hong J, Wang S. Relationship between pituitary adenoma texture and collagen content revealed by comparative study of MRI and pathology analysis. Int J Clin Exp Med 2015; 8 (08) 12898-12905
36 Azimi P, Mohammadi HR, Benzel EC, Shahzadi S, Azhari S, Montazeri A. Artificial neural networks in neurosurgery. J Neurol Neurosurg Psychiatry 2015; 86 (03) 251-256
37 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521 (7553): 436-444
38 Fernandez-Delgado M, Cernadas E, Barro S. et al. Do we need hundreds of classifiers to solve real world classification problems?. J Mach Learn Res 2014; 15: 3133-3181
39 Nawar S, Mouazen AM. Comparison between random forests, artificial neural networks and gradient boosted machines methods of on-line Vis-NIR spectroscopy measurements of soil total nitrogen and total carbon. Sensors (Basel) 2017; 17 (10) 2428
40 Senders JT, Staples P, Mehrtash A. et al. An online calculator for the prediction of survival in glioblastoma patients using classical statistics and machine learning. Neurosurgery 2020; 86 (02) E184-E192
41 Ahmad MW, Mourshed M, Rezgui Y. Trees vs Neurons: Comparison between random forest and ANN for high-resolution prediction of building energy consumption. Energy Build 2017; 147: 77-89
42 Reyes M, Meier R, Pereira S. et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol Artif Intell 2020; 2 (03) e190043

Supplementary Material

Supplementary Material