Perception Challenges for Mixed Robot-Human Swarm Collaboration

Abstract

This paper investigates the dynamics of cooperation between robots and humans, comparing the cognitive mechanisms in fully unmanned swarms with those in mixed robot-human swarms. Perception sharing significantly reduces cognitive uncertainty in robotic swarms through robots relying on camera processing with deep learning algorithms for object detection and localization. The study explores the consequences of integrating humans into these swarms, particularly under the constraint of limited digital communication. This examination is relevant to applications like the 2025 Robocup competition, where humans and robots will compete together in soccer under FIFA rules. In such scenarios, robots cannot effectively share and unify their perceptions, leading to reduced accuracy for each robot. To address this loss in accuracy, the paper proposes enhancing deep learning image processing algorithms, increasing camera resolution, and integrating camera data with LiDAR sensors via an extended Kalman filter. Several experiments have been conducted to analyze and compare the possibility of these approaches, providing valuable insights into improving robot perception in mixed swarms.