Date: June 2019.
Source: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.
Research Summary: Earlier 3D face models represented shape and texture using linear methods based on Principal Component Analysis, and even recent non-linear 3D morphable models, or 3DMMs, still processed shape and texture separately, using flat, two-dimensional convolutions on unwrapped texture maps rather than the 3D surface itself. This paper introduces the Coloured Mesh Decoder, or CMD, the first non-linear 3DMM auto-encoder to jointly represent shape and texture directly on the 3D mesh, using geometric deep learning to apply convolutions to mesh surfaces instead of flat images. The resulting model is compact, just 17 megabytes, and decodes a complete, colored 3D face in under half a millisecond, over 2,500 frames per second on a standard computer processor.
3dMD’s Role: The model’s shape and texture components are trained on the same real-world 3dMD dataset used to build UV-GAN’s UVDB, roughly 21,000 raw scans from 3,564 unique identities, each captured with six different facial expressions.
Article: Dense 3D Face Decoding Over 2500FPS: Joint Texture and Shape Convolutional Mesh Decoders.
Authors: Yuxiang Zhou, Jiankang Deng, Irene Kotsia, Stefanos Zafeiriou. Imperial College London, UK.