Command and Control of a Large Scale Swarm using Natural Human Interfaces

Abstract

While swarm robotics contain varying levels of autonomy, it is still important for human operators to command and control the swarm before and during mission operations. As such, humanin-the-loop interfaces must be developed that are intuitive to the operator and able to relay accurate commands to flight and ground robots in a large-scale swarm network. Furthermore, the tremendous amount of information gathered by the swarm must be sorted and presented back to the user in a way that reduces cognitive load so that they may determine the relevancy of the information and its effect on the mission. In this paper, we present sketch-based and augmented reality interfaces that both allow for command and control (C2) of unmanned ground and flight vehicles and for processing of data that the swarm has gathered in the field. These interfaces were tested in two field experiments with multiple aerial and ground robots deployed on a mission. Finally, we discuss the results of the field experiments, lessons learned, and areas for future work.