Poster: Towards Federated Embodied AI with FEAI

摘要

Embodied AI (EAI) transforms our daily lives by bridging intelligent agents with various sensors and actuators. Large Language Models (LLMs) further enhance EAI agents in environment comprehension, task decomposition, and action execution for robotic manipulation. However, developing a general EAI agent capable of adapting to and continuously learning from diverse operating environments is extremely challenging: 1) Robots with mobility capture environments from multiple perspectives, leading to heterogeneous semantic interpretations, particularly in large or open settings. 2) Heterogeneous environments further exacerbate the variability of decomposed tasks and corresponding actions required for robotic manipulation. To address these challenges, we propose FEAI, a novel paradigm to enhance the adaptability and self-learning capabilities of EAI agents in heterogeneous environments via federated embodied learning. Specifically, FEAI shares and constructively aggregates environment semantic maps, decomposed task templates, and action-reward rules from federated EAI agents. The aggregated information can further enhance EAI agents' local models through continuous tuning or dynamically updated knowledge databases. We believe that FEAI has significant potential to integrate more advanced technologies, further advancing performance and innovation in the field of EAI.