ContextBots: Real-time Context-aware Inference on Aerial Robots

Abstract

Neural Networks (NNs) have made significant strides in performing specialized problems such as navigation, playing complex games, and vision-based tasks. However, these achievements have been achieved using virtually unlimited resources or with little regard for real-time ‘actionability or environmental conditions. As a result, NN-based applications cannot often be executed in real-time on small/resource-limited drones and lack crucial context information to adapt to environmental changes. In this work, a context-aware framework is proposed for efficient and robust segmentation. Our approach uses context information to conserve energy and optimize inference in drones. We evaluate our framework using public datasets and hardware-in-the-loop emulations via the Microsoft AirSim simulator and the NVIDIA Jetson TX2 GPU.