Perceiving Systems, Computer Vision

Layered image motion with explicit occlusions, temporal consistency, and depth ordering

2010

Conference Paper

ps


Layered models are a powerful way of describing natural scenes containing smooth surfaces that may overlap and occlude each other. For image motion estimation, such models have a long history but have not achieved the wide use or accuracy of non-layered methods. We present a new probabilistic model of optical flow in layers that addresses many of the shortcomings of previous approaches. In particular, we define a probabilistic graphical model that explicitly captures: 1) occlusions and disocclusions; 2) depth ordering of the layers; 3) temporal consistency of the layer segmentation. Additionally the optical flow in each layer is modeled by a combination of a parametric model and a smooth deviation based on an MRF with a robust spatial prior; the resulting model allows roughness in layers. Finally, a key contribution is the formulation of the layers using an image dependent hidden field prior based on recent models for static scene segmentation. The method achieves state-of-the-art results on the Middlebury benchmark and produces meaningful scene segmentations as well as detected occlusion regions.

Author(s): Sun, D. and Sudderth, E. and Black, M. J.
Book Title: Advances in Neural Information Processing Systems 23 (NIPS)
Pages: 2226--2234
Year: 2010
Publisher: MIT Press

Department(s): Perceiving Systems
Research Project(s): Layers, Time and Segmentation
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Links: main paper
supplemental material
paper and supplemental material in one pdf file

BibTex

@inproceedings{Sun:NIPS:10,
  title = {Layered image motion with explicit occlusions, temporal consistency, and depth ordering},
  author = {Sun, D. and Sudderth, E. and Black, M. J.},
  booktitle = {Advances in Neural Information Processing Systems 23 (NIPS)},
  pages = {2226--2234},
  publisher = {MIT Press},
  year = {2010},
  doi = {}
}