Say’n’Fly: An LLM-Modulo Online Planning Framework to Automate UAV Command and Control

Abstract

Command and Control (C2) for Unmanned Aerial Vehicle (UAV) missions requires the dynamic coordination of task planning, environmental perception, and navigation strategies. While Large Language Model (LLM-) modulo planning frameworks have shown promise in addressing such complexity, due to their capabilities to understand natural language and common sense, their limited robustness remains a challenge. Recent adaptions of such frameworks made online planning possible, but they do not scale well with the number of actions. This limits the applicability in robotic frameworks with large sets of parameterized actions, such as those encountered in UAV C2. We introduce Say’n’Fly, an LLM-modulo online planning framework that streamlines the action space by discarding infeasible actions using domain-specific knowledge, while leveraging online heuristic search to mitigate reward uncertainty and automate the C2 processes for UAVs. Test results for our Search and Rescue (SAR) validation scenarios show that Say’n’Fly is 70 % more efficient compared to existing frameworks while maintaining or exceeding success rates.