3D Human Pose Estimation via Intuitive Physics (CVPR 2023)

19 June 2023

06:30

Estimating 3D humans from images often produces implausible bodies that lean, float, or penetrate the floor. Such methods ignore the fact that bodies are typically supported by the scene. A physics engine can be used to enforce physical plausibility, but these are not differentiable, rely on unrealistic proxy bodies, and are difficult to integrate into existing optimization and learning frameworks. In contrast, we exploit novel intuitive-physics (IP) terms that can be inferred from a 3D SMPL body interacting with the scene. Inspired by biomechanics, we infer the pressure heatmap on the body, the Center of Pressure (CoP) from the heatmap, and the SMPL body’s Center of Mass (CoM). With these, we develop IPMAN, to estimate a 3D body from a color image in a “stable” configuration by encouraging plausible floor contact and overlapping CoP and CoM. Our IP terms are intuitive, easy to implement, fast to compute, differentiable, and can be integrated into existing optimization and regression methods. We evaluate IPMAN on standard datasets and MoYo, a new dataset with synchronized multi-view images, ground-truth 3D bodies with complex poses, body-floor contact, CoM and pressure. IPMAN produces more plausible results than the state of the art, improving accuracy for static poses, while not hurting dynamic ones. Project: https://ipman.is.tue.mpg.de/ Code: https://github.com/sha2nkt/MoYo Paper: https://arxiv.org/pdf/2303.18246.pdf MOYO dataset: https://moyo.is.tue.mpg.de/ Bibtex: @inproceedings{tripathi2023ipman, title = {{3D} Human Pose Estimation via Intuitive Physics}, author = {Tripathi, Shashank and M{\"u}ller, Lea and Huang, Chun-Hao P. and Taheri Omid and Black, Michael J. and Tzionas, Dimitrios}, booktitle = {Conference on Computer Vision and Pattern Recognition ({CVPR})}, year = {2023}, url = {https://ipman.is.tue.mpg.de} }