Date: June 2022.
Source: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR): 2761-2770.
Research Summary: 3D face reconstruction from a single image has drawn significant interest in computer vision, with applications spanning realistic 3D avatar creation, pose-invariant face recognition, and face hallucination. Despite growing detail in single-image 3D face reconstruction driven by deep learning, finer, highly deformable components such as the tongue remain absent from every 3D face model in the literature, despite being important to the realism of 3D avatar representation. This paper presents the first end-to-end trainable pipeline that reconstructs the 3D face together with the tongue, made robust to “in-the-wild” images through a novel GAN method built specifically for 3D tongue surface generation. The team also releases the first diverse tongue dataset to the research community, 1,800 real-world raw scans of 700 individuals varying in gender, age, and ethnicity. Extensive quantitative and qualitative experiments show the model robustly and realistically captures 3D tongue structure, even under adverse in-the-wild conditions.
3dMD’s Role: TongueDB, the paper’s dataset of 1,800 real-world 3D tongue scans from 700 subjects, was captured as part of the same 2017 Facial Expression Experiment at the Science Museum, London that produced the MimicME dataset, using a 3dMDface.t system running at 10 frames per second. Subjects performed a range of tongue expressions, and the dataset includes demographic metadata for each subject, gender, age, and ethnicity.
Article: 3D human tongue reconstruction from single “in-the-wild” images.
Authors: Stylianos Ploumpis, Stylianos Moschoglou, Vasileios Triantafyllou, Stefanos Zafeiriou. Imperial College London, UK and Huawei Technologies Co. Ltd.