Can surgical simulation be used to train detection and classification of neural networks?

[摘要] Computer-assisted interventions (CAI) aim to increase the effectiveness, precision and repeatability of procedures to improve surgical outcomes. The presence and motion of surgical tools is a key information input for CAI surgical phase recognition algorithms. Vision-based tool detection and recognition approaches are an attractive solution and can be designed to take advantage of the powerful deep learning paradigm that is rapidly advancing image recognition and classification. The challenge for such algorithms is the availability and quality of labelled data used for training. In this Letter, surgical simulation is used to train tool detection and segmentation based on deep convolutional neural networks and generative adversarial networks. The authors experiment with two network architectures for image segmentation in tool classes commonly encountered during cataract surgery. A commercially-available simulator is used to create a simulated cataract dataset for training models prior to performing transfer learning on real surgical data. To the best of authors’ knowledge, this is the first attempt to train deep learning models for surgical instrument detection on simulated data while demonstrating promising results to generalise on real data. Results indicate that simulated data does have some potential for training advanced classification methods for CAI systems.

[学科分类] 肠胃与肝脏病学

[关键词] surgery;neural nets;learning (artificial intelligence);image recognition;image classification;image segmentation;biomedical optical imaging;video signal processing;medical image processing;surgical simulation;computer-assisted interventions;CAI surgical phase recognition algorithms;Vision20 based tool detection;deep learning;image recognition;image classification;tool detection;tool segmentation;deep convolutional neural networks;generative adversarial networks;image segmentation;cataract surgery