Integrating multiple visual cues for robust tracking

Abstract

Vision-based tracking is a promising technology for tasks such as human-computer interaction and mobile robot navigation. However, distraction and occlusion are major obstacles to robust performance. Though tracking is often framed as strictly an estimation problem, these phenomena also engender a correspondence problem that must be addressed. Without careful consideration of what image data, if any, to associate with a tracked object from frame to frame, the estimation process can become biased and the object lost. In this dissertation, we will describe a framework that incorporates and indeed emphasizes explicit reasoning about data association as a means of improving tracking performance in many difficult visual environments. This framework is built around a hierarchy of three tracking strategies that result from ascribing ambiguous or missing data to the following causes: (1) noise-like visual occurrences; (2) persistent, known scene elements (i.e.  other tracked objects); or (3) persistent, unknown scene elements. First, we introduce a randomized tracking algorithm adapted from an existing probabilistic data association filter (PDAF) that is resistant to clutter and follows agile motion. The algorithm is applied to three different tracking modalities—homogeneous regions, textured regions, and snakes—and extensibly defined so that the inclusion of other methods is straightforward. Second, we add the capacity to track multiple interacting objects by adapting a joint version of the PDAF to vision. This algorithm oversees correspondence choices between same-modality trackers and image features to ensure that they are feasibly distributed. We then derive a related technique that allows mixed tracker modalities, handles object overlaps, and deduces depth orderings. Finally, we represent complex objects as conjunctions of cues that are diverse both geometrically (e.g., a person's face, hands, and torso) and qualitatively (e.g., shape, color, and texture). The use of rigid and hinge constraints between part trackers and multiple attributes to describe individual parts renders the whole object more distinctive, reducing susceptibility to mistracking. Models for tracked targets can be flexibly specified; results are given for a number of objects, including people, cars, airplanes, microscopic cells, and chess pieces.