Date: October 2021.
Source: arXiv, DOI:10.48550/arXiv.2109.15299.
Abstract: Neural shape models can represent complex 3D shapes with a compact latent space. When applied to dynamically deforming shapes such as the human hands, however, they would need to preserve temporal coherence of the deformation as well as the intrinsic identity of the subject. These properties are difficult to regularize with manually designed loss functions. In this paper, we learn a neural deformation model that disentangles the identity-induced shape variations from pose-dependent deformations using implicit neural functions. We perform template-free unsupervised learning on 3D scans without explicit mesh correspondence or semantic correspondences of shapes across subjects. We can then apply the learned model to reconstruct partial dynamic 4D scans of novel subjects performing unseen actions. We propose two methods to integrate global pose alignment with our neural deformation model. Experiments demonstrate the efficacy of our method in the disentanglement of identities and pose. Our method also outperforms traditional skeleton-driven models in reconstructing surface details such as palm prints or tendons without limitations from a fixed template.
3dMD’s Role: The 3DH dataset behind this paper, 183 left-hand scans capturing a wide range of gestures, was captured by a dynamic-3D/4D 3dMDhand5.t capture system with five Modular Camera Units (MCU) with 3dMD’s proprietary reconstruction software. The data collection took place during 3dMD’s 2019/2020 4D Body and Hand Capture Exercise at the Science Museum, London.
Article: Identity-Disentangled Neural Deformation Model for Dynamic Meshes.
Authors: Binbin Xu, Lingni Ma, Yuting Ye, Tanner Schmidt, Christopher D. Twigg, Steven Lovegrove. Facebook Reality Labs and Imperial College London.