Embodied Hands (MANO): Modeling and Capturing Hands and Bodies Together. J Romero, D Tzionas, MJ Black.

Date: November 2017.
Source: ACM Transactions on Graphics (TOG) – Proceedings of ACM SIGGRAPH Asia 2017.
Research Summary: Most methods track and model bodies and hands separately, as detailed hands are difficult to capture accurately in conjunction with the body due to the size difference and occlusion challenges. MANO (hand Model with Articulated and Non-rigid defOrmations) addresses the hand portion of this problem with a low-dimensional, realistic parametric hand model, trained from around 1,000 real-world high-resolution 3D scans of 31 subjects in a wide variety of hand poses, using linear blend skinning with corrective pose blend shapes, an approach similar to the SMPL body model. MANO provides a compact mapping from hand poses to pose blend shape corrections and can be fit to any human hand. MANO is then attached to a standard parameterized 3D body shape model, SMPL, resulting in SMPL+H, a fully articulated body and hand model. SMPL+H is demonstrated by fitting it to real-world 4D sequences of people performing complex, natural activities, producing full-body motion capture with detailed, realistic hand movement not previously achieved.
3dMD’s Role: MANO’s hand shape and pose data, around 1,000 real-world high-resolution 3D scans of 31 subjects in a wide variety of hand poses, was captured using a 3dMDhand System providing an accuracy of 0.2mm root mean square error. Unlike SMPL, SMPL-X, and FLAME, MANO’s shape space does not draw on the CAESAR database. The paper states directly that CAESAR was excluded in favor of building the shape space entirely from this 3dMD hand data. To evaluate the combined SMPL+H model, the team captured 41 real-world 4D body sequences, 27,156 frames total, on a 3dMDbody22.u system running at 60fps. MANO’s hand data went on to inform SMPL-X, GHUM, and SUPR.

Article: Embodied hands: modeling and capturing hands and bodies together.
Authors: Javier Romero, Dimitrios Tzionas, Michael J. Black, Body Labs Inc. and Max Planck Institute for Intelligent Systems, Perceiving Systems, Tuebingen, Germany.