Detailed, accurate, human shape estimation from clothed 3D scan sequences. C Zhang, S Pujades, M Black, G Pons-Moll.

Date: March 2017.
Source: researchgate.net (goo.gl/scholar/d1zUSr)
Abstract: We address the problem of estimating human body shape from 3D scans over time. Reliable estimation of 3D body shape is necessary for many applications including virtual try-on, health monitoring, and avatar creation for virtual reality. Scanning bodies in minimal clothing, however, presents a practical barrier to these applications. We address this problem by estimating body shape under clothing from a sequence of 3D scans. Previous methods that have exploited statistical models of body shape produce overly smooth shapes lacking personalized details. In this paper we contribute a new approach to recover not only an approximate shape of the person, but also their detailed shape. Our approach allows the estimated shape to deviate from a parametric model to fit the 3D scans. We demonstrate the method using high quality 4D data as well as sequences of visual hulls extracted from multi-view images. We also make available a new high quality 4D dataset that enables quantitative evaluation. Our method outperforms the previous state of the art, both qualitatively and quantitatively.
5.2. BUFF: To create BUFF, we use a custom-built multi-camera active stereo system (3dMD LLC, Atlanta, GA) to capture temporal sequences of full-body 3D scans at 60 frames per second. The system uses 22 pairs of stereo cameras, 22 color cameras, 34 speckle projectors and arrays of whitelight LED panels. The projectors and LEDs flash at 120fps to alternate between stereo capture and color capture. The projected texture pattern makes stereo matching more accurate, dense, and reliable compared with passive stereo methods.
The stereo pairs are arranged to give full body capture for a range of activities, enabling us to capture people in motion.
Conclusions: We introduced a novel method to estimate a detailed body shape under clothing from a sequence of 3D scans. Our method exploits the information in a sequence by fusing all clothed registrations into a single frame. This results in very accurate shape estimates. We also contribute a new dataset (BUFF) of high resolution 3D scan sequences of clothed people as well as ground truth minimally-clothed shapes for each subject. BUFF is the first dataset of high quality 4D scans of clothed people; it will enable accurate quantitative evaluation of body shape estimation. Results on BUFF reveal a clear improvement of with respect to state of the art. One of the limitations of the presented approach is the systematic underestimation of female breast shape; this appears to be a limitation of SMPL. SMPL does not take into account soft tissue deformations of the body; future work will incorporate knowledge of soft tissue deformation to obtain even more accurate results. In addition, using the obtained minimally-clothed shapes and cloth alignments we plan to learn a model of cloth deviations from the body.
Article: Detailed, accurate, human shape estimation from clothed 3D scan sequences.
Authors: Chao Zhang, Sergi Pujades, Michael Black, and Gerard Pons-Moll, MPI for Intelligent Systems, Tubingen, Germany.