Real-time Software Hand Pose Recognition using Single View Depth Images

[摘要] ENGLISH ABSTRACT: The fairly recent introduction of low-cost depth sensors such as Microsoft's Xbox Kinecthas encouraged a large amount of research on the use of depth sensors for manycommon Computer Vision problems. Depth images are advantageous over normalcolour images because of how easily objects in a scene can be segregated in real-time.Microsoft used the depth images from the Kinect to successfully separate multipleusers and track various larger body joints, but has difficulty tracking smaller jointssuch as those of the fingers. This is a result of the low resolution and noisy nature ofthe depth images produced by the Kinect.The objective of this project is to use the depth images produced by the Kinect toremotely track the user's hands and to recognise the static hand poses in real-time.Such a system would make it possible to control an electronic device from a distancewithout the use of a remote control. It can be used to control computer systems duringcomputer aided presentations, translate sign language and to provide more hygieniccontrol devices in clean rooms such as operating theatres and electronic laboratories.The proposed system uses the open-source OpenNI framework to retrieve the depthimages from the Kinect and to track the user's hands. Random Decision Forests aretrained using computer generated depth images of various hand poses and used toclassify the hand regions from a depth image. The region images are processed usinga Mean-Shift based joint estimator to find the 3D joint coordinates. These coordinatesare finally used to classify the static hand pose using a Support Vector Machine trainedusing the libSVM library. The system achieves a final accuracy of 95.61% when testedagainst synthetic data and 81.35% when tested against real world data.

[发布机构] Stellenbosch University