Date: April 2019.
Source: arXiv.org, Computer Vision and Pattern Recognition.
Research Summary: Generative Adversarial Networks (GANs) are the current method of choice for generating visual data, and have shown exceptional performance producing realistic synthetic images, particularly of human faces. For 3D objects, GANs have fallen short of that success, largely because prior approaches apply 3D convolutional architectures to discrete volumetric representations, which limit shape quality. MeshGAN addresses this by introducing the first intrinsic GAN architecture that operates directly on 3D meshes, building on the ChebNet spectral convolution framework and the Boundary Equilibrium GAN training approach. The model separately learns an identity representation and an expression representation, allowing expressions to be generated for arbitrary identities rather than mixing the two, as earlier mesh autoencoders such as CoMA did. Quantitative and qualitative results on large-scale, real-world 3D facial data show that MeshGAN generates 3D faces with greater realism and richer detail than existing mesh autoencoder approaches.
3dMD’s Role: MeshGAN’s identity model is trained on approximately 12,000 unique identities from a real-world 3dMD dataset, spanning multiple ethnic groups, including Chinese, Caucasian, and Black subjects, and a range of age groups, scanned on a high-resolution 3dMD device. MeshGAN’s expression model is trained separately on the 4DFAB facial expression database.
Article: MeshGAN: Non-linear 3D Morphable Models of Faces.
Authors: Shiyang Cheng, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, Stefanos Zafeiriou. Imperial College London.