An FPGA-Based High-Throughput Keypoint Detection Accelerator Using Convolutional Neural Network for Mobile Robot Applications

Abstract

Keypoint detection is a key procedure for Visual-Inertial Odometry (VIO). In recent years, Convolutional Neural Network (CNN) has been introduced to enhance the robustness of keypoint detection. However, the high computational complexity and memory usage make them difficult to be deployed to edge platforms for high-throughput mobile robot applications such as Unmanned Aerial Vehicles (UAVs) and Autonomous Mobile Robots (AMRs). In this work, we proposed an FPGA-based high-throughput keypoint detection accelerator using CNN with algorithm-hardware co-design, including a lightweight keypoint detection neural network and a dedicated hardware accelerator architecture. Implemented on a Xilinx ZCU104 FPGA board, the proposed accelerator is able to perform keypoint detection at 94 FPS for <tex>$640 \times 480$</tex> input image with a low ATE, outperforming the state-of-the-art designs.