ARssist: augmented reality on a head-mounted display for the first assistant in robotic surgery

[摘要] In robot-assisted laparoscopic surgery, the first assistant (FA) is responsible for tasks such as robot docking, passing necessary materials, manipulating hand-held instruments, and helping with trocar planning and placement. The performance of the FA is critical for the outcome of the surgery. The authors introduce ARssist , an augmented reality application based on an optical see-through head-mounted display, to help the FA perform these tasks. ARssist offers (i) real-time three-dimensional rendering of the robotic instruments, hand-held instruments, and endoscope based on a hybrid tracking scheme and (ii) real-time stereo endoscopy that is configurable to suit the FA's hand–eye coordination when operating based on endoscopy feedback. ARssist has the potential to help the FA perform his/her task more efficiently, and hence improve the outcome of robot-assisted laparoscopic surgeries.

[学科分类] 肠胃与肝脏病学

[关键词] endoscopes;helmet mounted displays;medical robotics;surgery;augmented reality;stereo image processing;rendering (computer graphics);robot-assisted laparoscopic surgery;robot docking;hand-held instruments;trocar planning;augmented reality application;robotic instruments;optical see-through head-mounted display;ARssist;first assistant;hybrid tracking scheme;FA's hand-eye coordination;real-time stereo endoscopy;endoscopy feedback;real-time three-dimensional rendering