Perceiving Systems, Computer Vision

Specular flow and the recovery of surface structure

2006

Conference Paper

ps


In scenes containing specular objects, the image motion observed by a moving camera may be an intermixed combination of optical flow resulting from diffuse reflectance (diffuse flow) and specular reflection (specular flow). Here, with few assumptions, we formalize the notion of specular flow, show how it relates to the 3D structure of the world, and develop an algorithm for estimating scene structure from 2D image motion. Unlike previous work on isolated specular highlights we use two image frames and estimate the semi-dense flow arising from the specular reflections of textured scenes. We parametrically model the image motion of a quadratic surface patch viewed from a moving camera. The flow is modeled as a probabilistic mixture of diffuse and specular components and the 3D shape is recovered using an Expectation-Maximization algorithm. Rather than treating specular reflections as noise to be removed or ignored, we show that the specular flow provides additional constraints on scene geometry that improve estimation of 3D structure when compared with reconstruction from diffuse flow alone. We demonstrate this for a set of synthetic and real sequences of mixed specular-diffuse objects.

Author(s): Roth, S. and Black, M.
Book Title: Proc. IEEE Conf. on Computer Vision and Pattern Recognition, CVPR
Volume: 2
Pages: 1869--1876
Year: 2006
Month: June

Department(s): Perceiving Systems
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Address: New York, NY

Links: pdf

BibTex

@inproceedings{Roth:CVPR:2006,
  title = {Specular flow and the recovery of surface structure},
  author = {Roth, S. and Black, M.},
  booktitle = {Proc. IEEE Conf. on Computer Vision and Pattern Recognition, CVPR},
  volume = {2},
  pages = {1869--1876},
  address = {New York, NY},
  month = jun,
  year = {2006},
  doi = {},
  month_numeric = {6}
}