Hybrid 3D Reconstruction for Geometry-Based Free Viewpoint Video

More about the book

This thesis presents a working system for hybrid 3D reconstruction tailored for geometry-based Free Viewpoint Video. Central to the framework is a multi-projector, multi-camera array that enables real-time 3D reconstruction using phase-shifted structured light. The system enhances performance by incorporating disparity information from two color camera pairs for image-guided phase unwrapping. The array consists of six cameras and two projectors, supported by an automated calibration procedure for quick repositioning and recalibration. We also introduce an efficient method to eliminate or mask residual errors in 3D reconstruction after rendering geometry-based imagery for Free Viewpoint Video synthesis. To enhance final video quality, dynamic 3D data from the real-time system is integrated with a highly accurate static background model. A comparison of various 3D scanning, rendering, and display technologies reveals that our system supports stereoscopic, auto-stereoscopic, volumetric, and future holographic displays. To achieve immersive Free Viewpoint Video in the absence of large-scale full parallax holographic displays, we developed a compact head tracking framework that calculates the viewer's head position relative to the display. This allows the 3D data to be rendered in a way that creates the illusion of viewing the scene through a real-world window rather than a flat screen.