Creating 3D models using reconstruction techniques

[摘要] ENGLISH ABSTRACT :Virtual reality models of real world environments have a number of compellingapplications, such as preserving the architecture and designs of older buildings. This process can be achieved by using 3D artists to reconstruct theenvironment, however this is a long and expensive process. Thus, this thesisinvestigates various techniques and approaches used in 3D reconstruction ofenvironments using a single RGB-D camera and aims to reconstruct the 3Denvironment to generate a 3D model. This would allow non-technical users toreconstruct environments and use these models in business and simulations,such as selling real-estate, modifying pre-existing structures for renovation andplanning. With the recent improvements in virtual reality technology such asthe Oculus Rift and HTC Vive, a user can be immersed into virtual realityenvironments created from real world structures. A system based on KinectFusion is implemented to reconstruct an environment and track the motion ofthe camera within the environment. The system is designed as a series of selfcontained subsystems that allows for each of the subsystems to be modified,expanded upon or easily replaced by alternative methods. The system is madeavailable as an open source C++ project using Nvidia's CUDA framework toaid reproducibility and provides a platform for future research. The systemmakes use of the Kinect sensor to capture information about the environment.A coarse-to-fine least squares approach is used to estimate the motion of thecamera. In addition, the system employs a frame-to-model approach that usesa view of the estimated reconstruction of the model as the reference frame andthe incoming scene data as the target. This minimises the drift with respectto the true trajectory of the camera. The model is built using a volumetricapproach, with volumetric information implicitly stored as a truncated signeddistance function. The system filters out noise in the raw sensor data by using a bilateral filter. A point cloud is extracted from the volume using an orthogonal ray caster which enables an improved hole-filling approach. Thisallows the system to extract both the explicit and implicit structure from thevolume. The 3D reconstruction is followed by mesh generation based on thepoint cloud. This is achieved by using an approach related to Delaunay triangulation, the ball-pivot algorithm. The resulting system processes frames at30Hz, enabling real-time point cloud generation, while the mesh generation occurs offline. This system is initially tested using Blender to generate syntheticdata, followed by a series of real world tests. The synthetic data is used to testthe presented system's motion tracking against the ground truth. While thepresented system suffers from the effects of drift over long frame sequences, itis shown to be capable of tracking the motion of the camera. This thesis findsthat the ball pivot algorithm can generate the edges and faces for syntheticpoint clouds, however it performs poorly when using the noisy synthetic andreal world data sets. Based on the results obtained it is recommended that theobtained point cloud be preprocessed to remove noise before it is provided tothe mesh generation algorithm and an alternative mesh generation techniqueshould be employed that is more robust to noise.

[发布机构] Stellenbosch University