An Optimized Multi-Object Tracking with TensorRT

Abstract

Multi-object tracking is a crucial task in computer vision, requiring real-time performance, accurate identification, and tracking multiple objects in a scene. However, it is a challenging process due to the large number of objects to track, the need to maintain their identities over time, and the complexity of deep neural networks. To address these challenges, we present an optimized multi-object tracking pipeline using TensorRT. We leverage ByteTrack, a state-of-the-art multi-object tracking framework, and implement the pipeline in C++ for low latency. We optimized the Yolox neural network used in ByteTrack with TensorRT to produce bounding boxes for each frame. We also focus on the fact that ByteTrack utilizes sparse matrices when applying a Kalman filter to generate tracklets for each object. We evaluate the performance of our approach on the palace.mp4 video dataset, which is used in the ByteTrack demo. Our experimental results show a significant improvement in frame rate, increasing from 10.1fps to 14.3fps, while also reducing memory consumption from 1.4GB to 1.1GB. Overall, our optimized multi-object tracking pipeline demonstrates the effectiveness of combining deep learning with TensorRT and classic computer vision techniques. It provides an efficient and accurate solution for real-world multi-object tracking applications, such as surveillance, robotics, and autonomous vehicles.