Date: September 2018.
Source: 15th European Conference on Computer Vision (ECCV2018), Munich, Germany, (pp 704-720).
Research Summary: Learned 3D representations of human faces are useful for tasks like 3D face tracking and reconstruction from images, as well as character generation and animation. Traditional models represent faces using linear subspaces, which cannot capture extreme deformations and non-linear expressions. CoMA (Convolutional Mesh Autoencoders) addresses this with a non-linear model using spectral convolutions directly on a mesh surface, with mesh sampling operations that build a hierarchical representation capturing non-linear shape and expression variation at multiple scales. In a variational setting, the model samples diverse, realistic 3D faces from a multivariate Gaussian distribution. Despite training on a limited real-world dataset, just 20,466 meshes of extreme expressions from 12 subjects, CoMA outperforms state-of-the-art face models with 50% lower reconstruction error while using 75% fewer parameters, and replacing the expression space of an existing parametric face model with CoMA’s autoencoder further reduces reconstruction error.
3dMD’s Role: CoMA’s real-world training data of 20,466 3D meshes from 12 subjects performing extreme facial expressions, was captured at 60 FPS using an active stereophotogrammetry 3dMD system with six (6) Modular Camera Units (MCUs) each incorporating a stereo camera pair, a speckle projector, and a color camera.
Article: Generating 3D Faces using Convolutional Mesh Autoencoders.
Authors: Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black, Max Planck Institute for Intelligent Systems, Perceiving Systems, Tuebingen, Germany.