Centralized PPO-Based DRL for Multi-UAV-BS Positioning and Trajectory Optimization in Disaster Response Networks

摘要

Unmanned aerial vehicle-mounted base stations (UAV-BSs) constitute a flexible and effective solution for global positioning system (GPS)-free emergency and disaster scenarios, where the rapid deployment of communication infrastructure is critical for maximizing life-saving operations. In this work, we extend a centralized learning framework to a multi-UAV-BS network architecture, in which a single centralized UAV-BS -- as an intelligent agent -- coordinates the three-dimensional positioning and navigation of multiple UAV-BSs, while the remaining UAV-BSs actively serve ground user equipments (UEs) with uncertain positions. We formulate a fairness-aware sum-throughput maximization problem for UAV-BS coordination, which is inherently nonconvex due to the non-linear and interference-coupled throughput expressions. To address this challenge, we cast the problem as a Markov Decision Process (MDP) and solve it using a deep reinforcement learning (DRL) framework based on Proximal Policy Optimization (PPO). The central agent interacts with the environment and learns optimal joint positioning policies that guide the serving UAV-BSs to provide efficient, adaptive, and resilient wireless coverage. The proposed approach exploits spatial configuration and radio signal sensing capabilities to dynamically adapt to heterogeneous UE mobility patterns. Extensive simulations are conducted to evaluate the performance of the proposed method. Numerical results demonstrate that PPO shows competitive performance during both training and evaluation phases. Furthermore, comparative analysis with state-of-the-art RL algorithms, namely Deep Deterministic Policy Gradient (DDPG) and Deep QNetwork (DQN), shows that PPO consistently outperforms these methods in terms of convergence stability, mean reward, and network throughput.