Automatic 3D Reconstruction From Unstructured Videos Combining Video Summarization and Structure From Motion

[摘要] Social media and collection of large volumes of multimedia data such as images, videos and the accompanying text is of prime importance in today’s society. This is stimulated by the power of the humans to communicate one with the others. A useful paradigm of exploitation of such a huge amount of multimedia volumes is the 3D reconstruction and modelling of sites, historical cultural cities/regions or objects of interest from the short videos captured by simple users mainly for personal or touristic purposes. The main challenge in this research is the unstructured nature of the videos and the fact that they contain many information which is not related with the object the 3D model we ask for but for personal usage such as humans in front of the objects, weather conditions, etc. In this article, we propose an automatic scheme for 3D modelling/reconstruction of objects of interest by collecting pools of short duration videos that have been captured mainly for touristic purposes. Initially a video summarization algorithm is introduced using a discriminant Principal Component Analysis (d-PCA). The goal of this innovative scheme is to extract the frames so that bunches within each video cluster that contains videos of content referring to the same object present the maximum coherency of image data while content across bunches the minimum one. Experimental results on cultural objects indicate the efficiency pf the proposed method to 3D reconstruct assets of interest using an unstructured image content information. □.

[学科分类] 计算机网络和通讯

[关键词] 3D modeling;Social Media;video summarisation;3D Reconstruction;PCA