Date: July 2017.
Source: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, pp 5573-5582, doi: 10.1109/CVPR.2017.591.
Research Summary: While the ready availability of 3D scan data has influenced computer vision research broadly, less attention has been given to 4D data, meaning 3D scans of moving, non-rigid objects captured over time. To be useful for vision research, such 4D scans require registration, or alignment, to a common topology, making the extension of mesh registration methods to 4D an important problem to solve. No ground-truth datasets existed for the quantitative evaluation and comparison of 4D registration methods, so Dynamic FAUST addresses this gap with a novel dataset of high-resolution 4D scans of human subjects in motion, captured at 60 fps. The paper proposes a new mesh registration method that combines 3D geometry and texture information to register all scans in a sequence to a common reference topology, exploiting texture consistency over both short and long time intervals while accounting for temporal offsets between shape and texture capture. Using geometry alone produces significant alignment errors when motion is fast and non-rigid, a limitation this method overcomes. The resulting dataset of 40,000 real-world raw and aligned meshes extends the original FAUST dataset from static poses into dynamic 4D motion.
3dMD’s Role: Dynamic FAUST’s real-world 4D data of 129 sequences and over 40,000 frames across 10 subjects, was captured at 60 FPS using a dynamic-3D/4D active stereophotogrammetry 3dMDbody22.u System with 22 Modular Camera Units (MCUs) each with a stereo camera pair, color camera, and a speckle projector, supplemented by 12 additional standalone speckle projectors for a total of 34. This research is a dynamic-3D/4D real-world data extension of the original static-3D FAUST dataset published in 2014.
Article: Dynamic FAUST: Registering Human Bodies in Motion.
Authors: Federica Bogo, Javier Romero, Gerard Pons-Moll, Michael J Black. MPI for Intelligent Systems, Perceiving Systems, Tübingen, Germany.