Perceiving Systems, Computer Vision

SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments

2023

Conference Paper

ps

We present SLOPER4D, a novel scene-aware dataset collected in large urban environments to facilitate the research of global human pose estimation (GHPE) with human-scene interaction in the wild. Employing a head-mounted device integrated with a LiDAR and camera, we record 12 human subjects’ activities over 10 diverse urban scenes from an egocentric view. Frame-wise annotations for 2D key points, 3D pose parameters, and global translations are provided, together with reconstructed scene point clouds. To obtain accurate 3D ground truth in such large dynamic scenes, we propose a joint optimization method to fit local SMPL meshes to the scene and fine-tune the camera calibration during dynamic motions frame by frame, resulting in plausible and scene-natural 3D human poses. Eventually, SLOPER4D consists of 15 sequences of human motions, each of which has a trajectory length of more than 200 meters (up to 1,300 meters) and covers an area of more than 200 square meters (up to 30,000 square meters), including more than 100K LiDAR frames, 300k video frames, and 500K IMU-based motion frames. With SLOPER4D, we provide a detailed and thorough analysis of two critical tasks, including camera-based 3D HPE and LiDAR-based 3D HPE in urban environments, and benchmark a new task, GHPE. The in-depth analysis demonstrates SLOPER4D poses significant challenges to existing methods and produces great research opportunities. The dataset and code are released https://github.com/climbingdaily/SLOPER4D.

Author(s): Dai Yudi and Lin Yitai and Lin Xiping and Wen Chenglu and Xu Lan and Yi Hongwei and Shen Siqi and Ma Yuexin and Wang Cheng
Book Title: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)
Pages: 682-692
Year: 2023
Month: June
Publisher: CVF
Department(s): Perceiving Systems
Bibtex Type: Conference Paper (inproceedings)
Event Name: CVPR 2023
Event Place: Vancouver
State: Accepted
Links: project
dataset
codebase
paper
arXiv

BibTex 

@inproceedings{Dai_2023_CVPR,
  title = {SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments},
  author = {Yudi, Dai and Yitai, Lin and Xiping, Lin and Chenglu, Wen and Lan, Xu and Hongwei, Yi and Siqi, Shen and Yuexin, Ma and Cheng, Wang},
  booktitle = {IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
  pages = {682-692},
  publisher = {CVF},
  month = jun,
  year = {2023},
  doi = {},
  month_numeric = {6}
}