Improving Light Field Capture Using Hybrid Imaging

[摘要] Light Field imaging gives us the ability to perform post-capture focus manipulation, like refocusing and varying the depth-of-field (DOF). Although this ability is sought after in photography, an inherent problem arises when mapping the 4D light field function on a 2D sensor: reduction in resolution. Current light field cameras face this problem, producing low spatial resolution and hence has a limited DOF control. In contrast, a traditional high resolution camera, like DSLR, provide high spatial resolution and narrow DOF control at capture but no post-capture control.In this work, I propose a hybrid imaging system consisting of the two complementary imaging modalities: light field camera and standard high resolution camera,and show that the combined system enables (a) high-resolution digital refocusing, (b) better DOF control than light field cameras, and (c) render graceful high-resolution viewpoint variations. All of the above abilities were previously unachievable. In order to combine the output of the two modalities, I propose a simple patch-based algorithm that super-resolves the low-resolution views of the light field using the high-resolution patches captured using the high-resolution SLR camera. The algorithm does not require the light field camera and the DSLR to be co-located or for any calibration information regarding the two imaging systems.To depict the abilities of the hybrid imaging system, I built a prototype using a Lytro camera (380x380 pixel spatial resolution) and an 18 megapixel (MP) Canon DSLR camera. Via the prototype, I show 9x improvement in spatial resolution of the final light field (11.7 MP spatial resolution) and the ability to achieve 1/9th of the DOF of the Lytro camera.I show several experimental results on challenging scenes containing occlusions, specularities, and complex non-lambertian materials, demonstrating the effectiveness of our approach.

[发布机构] Rice University

[效力级别] Field