Machine Learning Models for Predicting Postoperative Outcomes following Skull Base Meningioma Surgery

Abstract

Objective While predictive analytic techniques have been used to analyze meningioma postoperative outcomes, to our knowledge, there have been no studies that have investigated the utility of machine learning (ML) models in prognosticating outcomes among skull base meningioma patients. The present study aimed to develop models for predicting postoperative outcomes among skull base meningioma patients, specifically prolonged hospital length of stay (LOS), nonroutine discharge disposition, and high hospital charges. We also validated the predictive performance of our models on out-of-sample testing data.

Methods Patients who underwent skull base meningioma surgery between 2016 and 2019 at an academic institution were included in our study. Prolonged hospital LOS and high hospital charges were defined as >4 days and >$47,887, respectively. Elastic net logistic regression algorithms were trained to predict postoperative outcomes using 70% of available data, and their predictive performance was evaluated on the remaining 30%.

Results A total of 265 patients were included in our final analysis. Our cohort was majority female (77.7%) and Caucasian (63.4%). Elastic net logistic regression algorithms predicting prolonged LOS, nonroutine discharge, and high hospital charges achieved areas under the receiver operating characteristic curve of 0.798, 0.752, and 0.592, respectively. Further, all models were adequately calibrated as determined by the Spiegelhalter Z-test (p >0.05).

Conclusion Our study developed models predicting prolonged hospital LOS, nonroutine discharge disposition, and high hospital charges among skull base meningioma patients. Our models highlight the utility of ML as a tool to aid skull base surgeons in providing high-value health care and optimizing clinical workflows.

Keywords

meningioma - neurovascular structures - central nervous system - overall survival - outcomes

Reporting Guidelines

The authors found no applicable reporting guidelines that would apply to this article. By following the EQUATOR reporting guidelines decision tree, (http://www.equatornetwork.org/wp-content/uploads/2013/11/20160226-RG-decision-tree-for-Wizard-CC-BY-26- February-2016.pdf), we found that none of the most popular checklists are appropriate for our study design.

Publication History

Received: 31 December 2021

Accepted: 20 June 2022

Accepted Manuscript online:
25 June 2022

Article published online:
25 August 2022

• References

• 1 Ostrom QT, Gittleman H, Fulop J. et al. CBTRUS statistical Report: primary brain and central nervous system tumors diagnosed in the United States in 2008–2012. Neuro-oncol 2015; 17 (suppl 4): iv1-iv62
• 2 Wang N, Osswald M. Meningiomas: overview and new directions in therapy. Semin Neurol 2018; 38 (01) 112-120
• 3 Meling TR, Da Broi M, Scheie D, Helseth E. Meningiomas: skull base versus non-skull base. Neurosurg Rev 2019; 42 (01) 163-173
• 4 Voß KM, Spille DC, Sauerland C. et al. The Simpson grading in meningioma surgery: does the tumor location influence the prognostic value?. J Neurooncol 2017; 133 (03) 641-651
• 5 Chen CM, Huang APH, Kuo LT, Tu YK. Contemporary surgical outcome for skull base meningiomas. Neurosurg Rev 2011; 34 (03) 281-296 , discussion 296
• 6 DeMonte F, McDermott M, Al-Mefty O. . Al-Mefty's Meningiomas. 2nd ed. Thieme; 2011
• 7 Saklad M. Grading of patients for surgical procedures. Anesthesiology 1941; 2 (03) 281-284
• 8 Subramaniam S, Aalberg JJ, Soriano RP, Divino CM. New 5-factor modified frailty index using American College of Surgeons NSQIP Data. J Am Coll Surg 2018; 226 (02) 173-181 .e8
• 9 Cahill PJ, Pahys JM, Asghar J. et al. The effect of surgeon experience on outcomes of surgery for adolescent idiopathic scoliosis. J Bone Joint Surg Am 2014; 96 (16) 1333-1339
• 10 Lau D, Deviren V, Ames CP. The impact of surgeon experience on perioperative complications and operative measures following thoracolumbar 3-column osteotomy for adult spinal deformity: overcoming the learning curve. J Neurosurg Spine 2019; 32 (02) 207-220
• 11 Dasenbrock HH, Liu KX, Devine CA. et al. Length of hospital stay after craniotomy for tumor: a National Surgical Quality Improvement Program analysis. Neurosurg Focus 2015; 39 (06) E12
• 12 Lakomkin N, Hadjipanayis CG. Resident participation is not associated with postoperative adverse events, reoperation, or prolonged length of stay following craniotomy for brain tumor resection. J Neurooncol 2017; 135 (03) 613-619
• 13 Muhlestein WE, Akagi DS, Chotai S, Chambless LB. The impact of presurgical comorbidities on discharge disposition and length of hospitalization following craniotomy for brain tumor. Surg Neurol Int 2017; 8: 220
• 14 Kalakoti P, Missios S, Menger R, Kukreja S, Konar S, Nanda A. Association of risk factors with unfavorable outcomes after resection of adult benign intradural spine tumors and the effect of hospital volume on outcomes: an analysis of 18, 297 patients across 774 US hospitals using the National Inpatient Sample (2002-2011). Neurosurg Focus 2015; 39 (02) E4
• 15 Muhlestein WE, Akagi DS, Chotai S, Chambless LB. The impact of race on discharge disposition and length of hospitalization after craniotomy for brain tumor. World Neurosurg 2017; 104: 24-38
• 16 Khalafallah AM, Jimenez AE, Patel P, Huq S, Azmeh O, Mukherjee D. A novel online calculator predicting short-term postoperative outcomes in patients with metastatic brain tumors. J Neurooncol 2020; 149 (03) 429-436
• 17 Brandel MG, Rennert RC, Wali AR. et al. Impact of preoperative endovascular embolization on immediate meningioma resection outcomes. Neurosurg Focus 2018; 44 (04) E6
• 18 Kuhn M. Building predictive models in R using the caret package. J Stat Softw 2008; 28 (05) 1-26
• 19 Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol 2005; 67 (05) 768
• 20 Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res 2012; 13: 281-305
• 21 Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240 (4857): 1285-1293
• 22 Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol 2010; 63 (08) 938-939 , author reply 939
• 23 Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26 (04) 404-413
• 24 Muhlestein WE, Akagi DS, McManus AR, Chambless LB. Machine learning ensemble models predict total charges and drivers of cost for transsphenoidal surgery for pituitary tumor. J Neurosurg 2018; 131 (02) 507-516
• 25 Navarro SM, Wang EY, Haeberle HS. et al. Machine learning and primary total knee arthroplasty: patient forecasting for a patient-specific payment model. J Arthroplasty 2018; 33 (12) 3617-3623
• 26 Jimenez AE, Khalafallah AM, Lam S. et al. Predicting high-value care outcomes after surgery for skull base meningiomas. World Neurosurg 2021; 149: e427-e436
• 27 Little AS, Chapple K. Predictors of resource utilization in transsphenoidal surgery for Cushing disease. J Neurosurg 2013; 119 (02) 504-511
• 28 Zacharia BE, Deibert C, Gupta G. et al. Incidence, cost, and mortality associated with hospital-acquired conditions after resection of cranial neoplasms. Neurosurgery 2014; 74 (06) 638-647
• 29 McKee SP, Yang A, Gray M. et al. Intracranial meningioma surgery: value-based care determinants in New York State, 1995-2015. World Neurosurg 2018; 118: e731-e744
• 30 Chapman EK, Doctor T, Gal JS. et al. The impact of non-elective admission on cost of care and length of stay in anterior cervical discectomy and fusion: a propensity-matched analysis. Spine 2021; 46 (22) 1535-1541
• 31 Ahn J, Iqbal A, Manning BT. et al. Minimally invasive lumbar decompression-the surgical learning curve. Spine J 2016; 16 (08) 909-916
• 32 Huq S, Khalafallah AM, Patel P. et al. Predictive model and online calculator for discharge disposition in brain tumor patients. World Neurosurg 2021; 146: e786-e798
• 33 Lakomkin N, Hadjipanayis CG. Non-routine discharge disposition is associated with post-discharge complications and 30-day readmissions following craniotomy for brain tumor resection. J Neurooncol 2018; 136 (03) 595-604
• 34 Sastry RA, Pertsch NJ, Tang O, Shao B, Toms SA, Weil RJ. Frailty and outcomes after craniotomy for brain tumor. J Clin Neurosci 2020; 81: 95-100
• 35 Muhlestein WE, Akagi DS, Kallos JA. et al. Using a guided machine learning ensemble model to predict discharge disposition following meningioma resection. J Neurol Surg B Skull Base 2018; 79 (02) 123-130
• 36 Kidwai SM, Yang A, Gray ML. et al. Hospital charge variability across New York State: sociodemographic factors in pituitary surgery. J Neurol Surg B Skull Base 2019; 80 (06) 612-619
• 37 Hamill CS, Villwock JA, Sykes KJ, Chamoun RB, Beahm DD. Socioeconomic factors affecting discharge status of patients with uncomplicated transsphenoidal adenohypophysectomy. J Neurol Surg B Skull Base 2018; 79 (05) 501-507
• 38 Abou-Al-Shaar H, Azab MA, Karsy M, Guan J, Couldwell WT, Jensen RL. Assessment of costs in open microsurgery and stereotactic radiosurgery for intracranial meningiomas. World Neurosurg 2018; 119: e357-e365
• 39 Alvin MD, Lubelski D, Alam R. et al. Spine surgeon treatment variability: the impact on costs. Global Spine J 2018; 8 (05) 498-506
• 40 Doumouras AG, Saleh F, Gmora S, Anvari M, Hong D. The value of surgical experience: excess costs associated with the Roux-en-Y gastric bypass learning curve. Surg Endosc 2019; 33 (06) 1944-1951
• 41 Breiman L. Statistical modeling: the two cultures. Stat Sci 2001; 16 (03) 199-215
• 42 Hannun AY, Rajpurkar P, Haghpanahi M. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med 2019; 25 (01) 65-69
• 43 Senior AW, Evans R, Jumper J. et al. Improved protein structure prediction using potentials from deep learning. Nature 2020; 577 (7792): 706-710
• 44 Lockhart R, Taylor J, Tibshirani RJ, Tibshirani R. A significance test for the lasso. Ann Stat 2014; 42 (02) 413-468
• 45 Lee JD, Sun DL, Sun Y, Taylor JE. Exact post-selection inference, with application to the lasso. Ann Stat 2016; 44 (03) 907-927
• 46 Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol 2019; 110: 12-22
• 47 Marcus AP, Marcus HJ, Camp SJ, Nandi D, Kitchen N, Thorne L. Improved prediction of surgical resectability in patients with glioblastoma using an artificial neural network. Sci Rep 2020; 10 (01) 5143
• 48 Bae S, Massie AB, Caffo BS, Jackson KR, Segev DL. Machine learning to predict transplant outcomes: helpful or hype? A national cohort study. Transpl Int 2020; 33 (11) 1472-1480
• 49 Oosterhoff JHF, Gravesteijn BY, Karhade AV. et al; Machine Learning Consortium. Feasibility of machine learning and logistic regression algorithms to predict outcome in orthopaedic trauma surgery. J Bone Joint Surg Am 2022; 104 (06) 544-551