Date: October 2021.
Source: arXiv, DOI:10.48550/arXiv.2109.15299.
Research Summary: Neural shape models can represent complex 3D shapes using a compact latent space, but when applied to dynamically deforming shapes such as human hands, they must also preserve temporal coherence of the deformation and the intrinsic identity of the subject, properties that are difficult to enforce with manually designed loss functions. This paper addresses that gap by learning a neural deformation model that disentangles identity-induced shape variation from pose-dependent deformation using implicit neural functions. The method performs template-free, unsupervised learning directly on real-world 3D scans, without requiring explicit mesh or semantic correspondence across subjects, and the resulting model can reconstruct partial dynamic 4D scans of novel subjects performing previously unseen actions. The paper proposes two methods for integrating global pose alignment with the neural deformation model, and experiments confirm the method’s effectiveness at disentangling identity from pose. It also outperforms traditional skeleton-driven models at reconstructing fine surface details such as palm prints and tendons, without the limitations imposed by a fixed template.
3dMD’s Role: The 3DH dataset behind this paper, 183 left-hand scans capturing a wide range of gestures, was captured by a dynamic-3D/4D 3dMDhand5.t capture system with five Modular Camera Units (MCU) with 3dMD’s proprietary reconstruction software. The data collection took place during 3dMD’s 2019/2020 4D Body and Hand Capture Exercise at the Science Museum, London.
Article: Identity-Disentangled Neural Deformation Model for Dynamic Meshes.
Authors: Binbin Xu, Lingni Ma, Yuting Ye, Tanner Schmidt, Christopher D. Twigg, Steven Lovegrove. Facebook Reality Labs and Imperial College London.