MACU: A Multiagent Cache Updating Framework for IIoT Networks

Abstract

An important role played by industrial Internet of Things (IIoTs) networks is supporting the operations of autonomous mobile robots (AMRs) by leveraging multiple-access edge caching servers. At the same time, judicious content caching strategies are essential for minimizing content retrieval delays and the costs associated with updating caches. In this study, a novel multiagent reinforcement learning (MARL)-based cache update strategy termed multiagent cache update (MACU) is proposed. MACU leverages the multiagent deep deterministic policy gradient (MADDPG) framework and aims to optimize cache updates to reduce Age of Information (AoI), minimize events where AoI exceeds acceptable levels, and ensure that the costs associated with performing cache updates are kept low. Furthermore, to mitigate the computational complexity of training agents with MACU, the MACU with global critic (MACU-GC) variant is introduced, which diverges from traditional MADDPG by employing a singular global critic for enhanced training efficiency. Extensive numerical evaluations showcase the proposed strategies’ superiority over conventional deep reinforcement learning-based caching methods, achieving significant improvements in AoI costs, caching costs, and AoI violation costs, while effectively reducing content retrieval latency, enhancing hit rates, and optimizing AoI and link load metrics.