Date: June 2020.
Source: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
Research Summary: CAPE addresses a gap in 3D body modeling. Existing models are trained on minimally clothed scans and do not generalize well to the complexity of dressed people seen in real images and video, and they lack the expressive power to represent how clothing shape changes with pose. CAPE learns a generative 3D mesh model of clothed people directly from real-world 3D scans with varying pose and clothing, using a conditional Mesh-VAE-GAN to learn how clothing deforms on top of the SMPL body model. The model can be conditioned on both pose and clothing type, allowing it to generate new clothing samples across different body shapes and poses, while a patchwise discriminator approach preserves wrinkle detail. CAPE is the first generative model known to directly dress 3D human body meshes and generalize across poses trained on real-world 3D scans.
3dMD’s Role: CAPE’s own dataset was captured directly on a 3dMD high-resolution body scanner, roughly 80,000 3D real-world scan frames at 60 frames per second across eleven subjects. This is a new subject capture exercise with the 3dMDbody System, not a reuse of ClothCap’s original dataset. The dense-surface 3D/4D datasets behind ClothCap, Dyna, and Dynamic FAUST were all previously captured with a 3dMDbody System.
Article: Learning to Dress 3D People in Generative Clothing.
Authors: Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, Michael J Black. Max Planck Institute for Intelligent Systems, Perceiving Systems Department, Tübingen, Germany.