Traffic Engineering in Large-scale Networks via Multi-Agent Deep Reinforcement Learning with Joint-Training

Abstract

Reinforcement learning (RL) has been successfully applied in many fields for building autonomous systems, such as robotics and telecommunications. With its self-learning ability, RL provides a framework for learning from historical experience and adapting to dynamic environments. In response to the surge in network traffic and the evolving nature of traffic behavior, RL has emerged as a crucial technique for developing intelligent and adaptive traffic engineering (TE) solutions. However, most prior studies have focused on using a centralized unit (i.e., a single agent) to construct RL-based TE systems. While the centralized approach leverages global network information for solid performance, it encounters challenges related to scalability, dynamic network topology, and high monitoring overhead for collecting network information. This paper addresses these issues by introducing a jointly trained multi-agent reinforcement learning-based traffic engineering (MATE-JT) system, which operates as a distributed TE solution. Our approach utilizes multiple agents within a network node so that each agent can make independent routing decisions for a subset of flows. We take the approach of sharing parameters among agents and introduce a joint-training technique that facilitates simultaneous learning from multiple agents’ experiences. As a result, our proposed method enhances system performance while reducing training time. We evaluate the proposed approach using various network traffic datasets and demonstrate that MATE-JT improves the performance of TE (about 6.5%) and achieves faster convergence (about 35%) in large-scale networks when compared to state-of-the-art methods.