Perceiving Systems, Computer Vision

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

2021

Conference Paper

avg

ps


Neural implicit surface representations have emerged as a promising paradigm to capture 3D shapes in a continuous and resolution-independent manner. However, adapting them to articulated shapes is non-trivial. Existing approaches learn a backward warp field that maps deformed to canonical points. However, this is problematic since the backward warp field is pose dependent and thus requires large amounts of data to learn. To address this, we introduce SNARF, which combines the advantages of linear blend skinning (LBS) for polygonal meshes with those of neural implicit surfaces by learning a forward deformation field without direct supervision. This deformation field is defined in canonical, pose-independent, space, enabling generalization to unseen poses. Learning the deformation field from posed meshes alone is challenging since the correspondences of deformed points are defined implicitly and may not be unique under changes of topology. We propose a forward skinning model that finds all canonical correspondences of any deformed point using iterative root finding. We derive analytical gradients via implicit differentiation, enabling end-to-end training from 3D meshes with bone transformations. Compared to state-of-the-art neural implicit representations, our approach generalizes better to unseen poses while preserving accuracy. We demonstrate our method in challenging scenarios on (clothed) 3D humans in diverse and unseen poses.

Author(s): Xu Chen and Yufeng Zheng and Michael J. Black and Otmar Hilliges and Andreas Geiger
Book Title: Proc. International Conference on Computer Vision (ICCV)
Pages: 11574--11584
Year: 2021
Month: October
Publisher: IEEE

Department(s): Autonomous Vision, Perceiving Systems
Research Project(s): Implicit Representations
Clothing Capture and Modeling
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

DOI: 10.1109/ICCV48922.2021.01139
Event Name: International Conference on Computer Vision 2021
Event Place: virtual (originally Montreal, Canada)

Address: Piscataway, NJ
ISBN: 978-1-6654-2812-5
State: Published

Links: pdf
pdf 2
supplementary material
project
blog
blog 2
video
video 2
code
Video:
Video:

BibTex

@inproceedings{Chen:ICCV:2021,
  title = {{SNARF}: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes},
  author = {Chen, Xu and Zheng, Yufeng and Black, Michael J. and Hilliges, Otmar and Geiger, Andreas},
  booktitle = {Proc. International Conference on Computer Vision (ICCV)},
  pages = {11574--11584},
  publisher = {IEEE},
  address = {Piscataway, NJ},
  month = oct,
  year = {2021},
  doi = {10.1109/ICCV48922.2021.01139},
  month_numeric = {10}
}