FPGA-based UAV and UGV for search and rescue applications: A case study

Abstract

The trend of applying robots to search and rescue (SAR) tasks has recently been growing. In this work, we present FPGA-based robot designs , including unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), aimed at supporting a broader range of SAR operations and achieving precise location of survivors. Our proposed SAR robot design incorporates a neural engine to support edge artificial intelligence (AI) for real-time and accurate survivor detection without transmitting captured images to a centralized AI platform. We employ the popular You Only Look Once (YOLO)v3 model for object detection and further propose an image fusion method that combines RGB and thermal infrared images to support accurate image-based survivor detection. Besides image-based methods, human voices and activities are utilized in the UGVs to provide more comprehensive survivor detection capabilities. Additionally, based on detected human voices, a dynamically adaptive search path planning method is presented to enhance the efficiency of survivor detection. Furthermore, we present an FPGA-based system design method to enable easy deployment of user-customized AI models for survivor detection, demonstrating the practicability and scalability of the proposed SAR robot system design. The experiments show that our SAR robot system achieves an accuracy of 90% and an F1-score of 0.891 for image-based survivor detection. Compared to similar designs incorporating AI acceleration devices such as the Intel Neural Compute Stick, our SAR system design achieves a frame rate acceleration of up to 1.9 times.