Learning the representation of instrument images in laparoscopy videos

[摘要] Automatic recognition of instruments in laparoscopy videos poses many challenges that need to be addressed, like identifying multiple instruments appearing in various representations and in different lighting conditions, which in turn may be occluded by other instruments, tissue, blood, or smoke. Considering these challenges, it may be beneficial for recognition approaches that instrument frames are first detected in a sequence of video frames for further investigating only these frames. This pre-recognition step is also relevant for many other classification tasks in laparoscopy videos, such as action recognition or adverse event analysis. In this work, the authors address the task of binary classification to recognise video frames as either instrument or non-instrument images. They examine convolutional neural network models to learn the representation of instrument frames in videos and take a closer look at learned activation patterns. For this task, GoogLeNet together with batch normalisation is trained and validated using a publicly available dataset for instrument count classifications. They compared transfer learning with learning from scratch and evaluate on datasets from cholecystectomy and gynaecology. The evaluation shows that fine-tuning a pre-trained model on the instrument and non-instrument images is much faster and more stable in learning than training a model from scratch.

[学科分类] 肠胃与肝脏病学

[关键词] object detection;video signal processing;learning (artificial intelligence);image classification;gynaecology;surgery;image sequences;convolutional neural nets;image motion analysis;medical image processing;image representation;laparoscopy videos;automatic recognition;recognition approaches;instrument frames;video frames;classification tasks;action recognition;noninstrument images;learned activation patterns;instrument count classifications;transfer learning;adverse event analysis;binary classification;convolutional neural network;GoogLeNet;cholecystectomy;gynaecology;instrument images representation