Mixed-Reality Based Multi-Agent Robotics Framework for Artificial Swarm Intelligence Experiments

Abstract

The term “Swarm intelligence” outlines a broad scope, which is generally defined as the collective behaviour of many individuals towards a certain task, each operating autonomously in a decentralised manner. Swarms are inspired by the type of biological behaviours of animals and this technology involves decentralised local control and communication, which ensures the problem-solving process is more efficient through modification and infusion into swarm robotic technology. One such application is Mixed Reality, which links both the real and virtual environments through Augmented Reality (AR) and Augmented Virtuality (AV). Enabling a robot to sense both physical and virtual environments via augmented means allows for the ability to interact with both environments. Swarm robotics related experiments and applications are relatively expensive compared to other types of robotics experiments. The most common solution is to use computer-based virtual simulations. However, as it lacks the real-world guarantee of execution, this study introduces a hybrid solution that can simultaneously execute swarm behavioural experiments on actual robots and virtually deployed robots, which can be used to test the functionalities of swarm robots with a large number of different environmental arrangements easier than physically creating them. As identifying the necessary requirements for implementing a mixed reality environment for swarm robotics simulation was this study’s main focus, the basic interaction models were designed to conduct experiments with both physical and virtual robots in near real-time in a mixed reality environment. In addition, an open-source, distributed, and modular mixed reality simulation framework was implemented with support libraries and applications.