Date: December 2020.
Source: arXiv.org, Computer Vision and Pattern Recognition.
Research Summary: Standard registration algorithms must be applied independently to each surface, requiring careful pre-processing and manual tuning. Shape My Face (SMF) reframes registration as a surface-to-surface translation problem, using a point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model to reliably capture geometric information directly from raw 3D face scans. Unlike prior state-of-the-art learning-based registration methods, SMF only requires raw data to be rigidly aligned with a pre-defined face template, and it produces topologically sound meshes with minimal supervision, faster training time, far fewer trainable parameters, and greater robustness to noise, while generalizing to previously unseen datasets. By learning to register scans, SMF also produces a hybrid linear and non-linear morphable model, and manipulating its latent space enables shape generation and morphing applications such as expression transfer entirely in the wild. The team trains two versions of the model, SMF on seven large-scale face databases, and the larger SMF+ on nine, and evaluates both extensively on real-world data spanning diverse modalities, sensor types, and resolutions.
3dMD’s Role: 3dMD data contributes to this paper in three distinct ways. The base SMF model is trained on a pooled set of seven real-world databases plus synthetic data, three of which are 3dMD-generated: BU-3DFE, the Texas 3D Face Recognition Database, and the Florence 2D/3D dataset, each captured on an active stereophotogrammetry 3dMDface System. The remaining training set data includes laser, structured light, and passive stereo scans, plus synthetic data. The larger SMF+ variant adds two further real-world databases, one of which is MeIn3D, which consists of 12,000 scans and is the same dataset behind LSFM. Separately from training, both SMF and SMF+ are evaluated for generalization on a distinct, held-out 3dMD test set of more than 24,000 real-world scans from over 3,000 individuals.
Article: Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation.
Authors: Mehdi Bahri, Eimear O’ Sullivan, Shunwang Gong, Feng Liu, Xiaoming Liu, Michael M. Bronstein, Stefanos Zafeiriou. Department of Computing, Imperial College London, London, UK.