Date: June 2018.
Source: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7093-7102.
Research Summary: Fitting a 3D face model to a photo lets researchers unwrap the visible skin into a flat facial texture map, called a UV map. But because a photo only shows one side of a face, part of the UV map is always missing, whichever side was turned away from the camera. UV-GAN solves this with a generative adversarial network, or GAN, trained to fill in the missing regions of a UV map so it looks complete and realistic. Because the completed map covers the whole face, it can then be used to generate the same identity at any pose, giving face recognition systems far more training examples of each person than photos alone could provide. This significantly improves face recognition accuracy on faces photographed from difficult, non-frontal angles.
3dMD’s Role: UV-GAN’s own dataset, UVDB, is built from three sources. Only the first is 3dMD, and only the first is genuine captured 3D data: 3,564 unique identities, each captured with six expressions, 21,384 total UV maps, scanned directly on 3dMD hardware. The other two sources, Multi-PIE and WildUV, are ordinary 2D photos and video, with 3D shape estimated afterward by fitting an existing 3D model onto them, not captured directly. 3dMD’s contribution is UVDB’s only real, physically scanned 3D core.
Article: UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition.
Authors: Jiankang Deng, Shiyang Cheng, Niannan Xue, Yuxiang Zhou, Stefanos Zafeiriou, Imperial College London.