Fire detection of Unmanned Aerial Vehicle in a Mixed Reality-based System

Abstract

This paper presents a system that combines robotic operating system (ROS) and computer vision techniques for fire detection in a mixed reality environment. We have collected video streams from a mini camera on an Unmanned Aerial Vehicle (UAV), where the navigation data relied on state-of-the-art Simultaneous Localization and Mapping (SLAM) system. The data collected onboard are communicated to the ground station and processed through the robotic operating system. A robust and efficient re-localisation SLAM was performed to recover from tracking failure and frame lost in the received data. The fire detection algorithm was developed based on the colour, movement attributes, temporal variation of fire intensity and its accumulation around a point. A mixed reality environment was used to visualise and test the proposed system. The observation and data analysis confirmed that the UAV could successfully detect fire and flame, fly towards and hover around it.