Subscribe to RSS
DOI: 10.1055/s-0043-1762336
Designing and Developing a Novel Deep Computer Vision Platform for Intraoperative Prediction and Analytics in Skull Base Surgery
Introduction: Robust artificial intelligence (AI)-based surgical video analysis platforms could lead to novel insights for intraoperative guidance and analytics.
Objective: In this study, we aim to design and develop an AI-based computer vision architecture for automated pixel-wise object prediction, tracking, and novel analytics during endoscopic skull base surgery, specifically during endoscopic microvascular decompression.
Method: Microvascular decompression surgeries were recorded using a Storz endoscope. A sparse labeling paradigm was used, and training data comprised only 3% of the total video frames. Surgical anatomy including the brain stem, cerebellum, cranial nerves, vascular structures and surgical instruments were annotated. We develop our custom segmentation algorithm on top of the state-of-the-art method SOLOv2 (method 1), using a total of 1287 training frames from surgery videos of 3 patients and test on 81 frames of a novel patient surgery. Additionally, we demonstrate enhanced precision of few shot learning (method 2) in a sparsely labeled dataset paradigm using 7 frames as training from novel patient surgery and test on 74. Mean average precision (mAP) was computed within and across videos as an evaluation metric. Lastly, a novel metric to quantify arterial pulsation-induced nerve deformation is introduced and compared before and after Teflon sponge placement.
Results: Our model consistently outperforms the baseline, for the test set we obtain a segmentation accuracy (measured by mean average precision mAP) for the trigeminal nerve as 39.51. The tool accuracy is 42.10 and overall segmentation accuracy is 12.32. Performance of few-shot learning obtained mAP for the trigeminal nerve as 57, tool accuracy as 48, and overall segmentation accuracy as 27.13. Lastly, our pulsatility index prior to decompression is 9.5, whereas after decompression is 6.0, indicating successful automated quantification of dampening of artery-nerve pulsation after sponge insertion.
Conclusion: In a sparse labeling paradigm, we design and develop a custom-deep computer vision-based instance segmentation architecture to predict and track anatomical structures and surgical objects with high accuracy. We demonstrate good segmentation performance on a novel patient test video and leverage few shot learning to further improve segmentation performance. We create a novel metric, the “pulsatility index,” which can quantify the nerve–artery interface for the first time. Once correlated with outcome measures, real-time AI-based video analysis may have the transformative potential to re-define intraoperative standard of care.






Publication History
Article published online:
01 February 2023
© 2023. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany