Perceiving Systems, Computer Vision

Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

2016

Conference Paper

ps


We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we fi rst use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fi t it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior pose accuracy with respect to the state of the art.

Author(s): Federica Bogo and Angjoo Kanazawa and Christoph Lassner and Peter Gehler and Javier Romero and Michael J. Black
Book Title: Computer Vision – ECCV 2016
Pages: 561--578
Year: 2016
Month: October
Series: Lecture Notes in Computer Science
Publisher: Springer International Publishing

Department(s): Perceiving Systems
Research Project(s): Optimizing Human Pose and Shape
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: 14th European Conference on Computer Vision
Event Place: Amsterdam, The Netherlands

Links: pdf
Video
Sup Mat video
Code
Project
ppt
Video:
Video:

BibTex

@inproceedings{Bogo:ECCV:2016,
  title = {Keep it {SMPL}: Automatic Estimation of {3D} Human Pose and Shape from a Single Image},
  author = {Bogo, Federica and Kanazawa, Angjoo and Lassner, Christoph and Gehler, Peter and Romero, Javier and Black, Michael J.},
  booktitle = {Computer Vision -- ECCV 2016},
  pages = {561--578},
  series = {Lecture Notes in Computer Science},
  publisher = {Springer International Publishing},
  month = oct,
  year = {2016},
  doi = {},
  month_numeric = {10}
}