Speed-FairMOT: multi-class multi-object tracking for real-time manipulation

摘要

Abstract In order to achieve complex tasks at high speed in robot manipulation, the ability to perform multi-object tracking (MOT), which recognizes the many objects in the surrounding area using camera-based real-time image data processing, is essential. To overcome traditional MOT methods slow tracking speeds challenges, we propose Speed-FairMOT, a deep-learning based real-time multi-class MOT method. We evaluate the Speed-FairMOT on MOT17 dataset and our custom synthetic dataset, achieving over $$41\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>41</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> improvement in speed and slightly decrease of $$5.9\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>5.9</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> in tracking performance as trade-off compared to the original FairMOT. We verified proposed Speed-FairMOT using a camera mounted on a robot manipulator as a hand-eye system. As the result, we were able to achieve MOT at an maximum speed over 58 fps in real-time. This real-time speed is sufficient for feedback control in robotic manipulation system.