Visual object tracking: Progress, challenge, and future

摘要

Visual object tracking aims to continuously localize the target object of interest in a video sequence. As one of the most fundamental problems in computer vision, visual object tracking has a long list of critical applications including video surveillance, autonomous driving, human-machine interaction, augmented reality, robotics, etc., in which the tracking system provides the capacity to report target positions in real time for subsequent visual analysis. In the past decades, visual object tracking has been extensively explored and has witnessed considerable progress, especially in deep-learning-based tracking. Despite this, robust tracking remains challenging due to many factors. To provide the community an overview, in this commentary, we will discuss visual tracking from different aspects. Specifically, we will first summarize the recent advancements achieved in visual tracking from the perspectives of algorithm and dataset. Then, we will analyze the challenges that the tracking community faces in developing practical tracking systems. Finally, we will discuss several promising directions for future research on visual tracking. It is worth noting that visual object tracking is a broad problem and consists of many specific topics. In this commentary, we focus on the most popular single-modality (i.e., RGB), bounding-box-based tracking. Figure 1 illustrates the task of visual tracking and the organization of this commentary. Visual object tracking has been largely studied in the past decades. Earlier hand-crafted approaches mainly focus on designing discriminative classification models and/or robust appearance features for tracking. Nevertheless, these trackers usually suffer from various appearance variations under complex scenarios, which limits their performance in achieving high accuracy. Motivated by deep learning, researchers in the tracking community have leveraged deep neural networks (DNNs) for visual tracking in recent years. Specifically, these trackers propose to extract deep features from pre-trained networks and then employ more robust deep representations for tracking, demonstrating superior performance compared with traditional hand-crafted tracking models. Despite this, earlier deep trackers often undergo a heavy computational burden because they require updating the deep model frequently in an online manner, which may result in slow running speed. Addressing this issue, researchers later formulate the tracking task as a forward matching problem. The main idea is to directly leverage DNNs for learning a generic similarity measurement, which matches the designated target object in the initial frame with the current search region to achieve object tracking. As no model update is required once the training completed, these trackers are able to run efficiently in real time with a video graphics card or GPU. Because of the good balance between accuracy and speed, this matching-based tracking framework has become the major trend in visual tracking with many extensions for further improvements. More recently, due to the capacity in modeling global contextual relation, the Transformer architecture1Vaswani A. Shazeer N. Parmar N. et al.Attention is all you need.Adv. Neural Inf. Process. Syst. 2017; 30: 5998-6008Google Scholar has been introduced to object tracking, which greatly improves the object appearance representation and further pushes the frontier of tracking accuracy. Nowadays, Transformer almost becomes a necessity for state-of-the-art trackers. In addition to the tracking algorithm, the tracking dataset has seen remarkable progress in recent years. The tracking dataset serves an important role in advancing visual tracking. Previous datasets are usually utilized for fair evaluation and comparison of different trackers and are therefore small in scale. One representative is the VOT Challenge,2Kristan M. Matas J. Leonardis A. et al.A novel performance evaluation methodology for single-target trackers.IEEE T