Seeing Physics, or: Physics is for Prediction

摘要

We describe how knowledge of the physics of the scene itself is important to computer vision. Highlevel knowledge of scene physics can help programs see the world, and programs that see and understand this way are useful for planning plan scene interactions. We illustrate these points with two of our most recent knowledge-intensive vision systems. One uses knowledge of physics and function to understand noisy and ambiguous images of gear-train machines; i.e. to report what the machine does. The other uses physical knowledge to guide a robotic eye-hand system to pick up a mug of coffee by its handle. 1 Introduction There are at least two senses in which the phrase "physics-based vision" may be interpreted---namely, the use of physics to describe and attempt to invert the image formation process, and the use of physical methods for object and shape modelling. In our view, this misses the primary use of physics for vision---namely, to describe the scene itself. Physics, after all, was i...