CR-DDPG: Cache Refreshing for MEC Networks with DDPG

Abstract

In the context of the Industrial Internet of Things (IIoT), multiple access edge computing (MEC) enables the provision of computational resources closer to users. The work presented in this paper explores a common IIoT application scenario wherein autonomous mobile robots (AMR) operate in a MEC-enabled network and depend on multimedia content stored at the MEC servers over the course of operation. The age of information (AoI) metric is used to measure the freshness of the cached content from the perspective of the AMR. At the same time, the energy cost associated with refreshing the cache files is simultaneously considered as well. This paper delves into achieving an optimal trade-off between minimizing the weighted AoI cost and the energy expended for cache refreshing. We address this problem by introducing a cache refreshing-deep deterministic policy gradient (CR-DDPG) algorithm, a model-free deep reinforcement learning method, to optimize both AoI and energy usage. Various simulation studies are conducted to evaluate the proposed CR-DDPG algorithm, and the results demonstrate that CR-DDPG consistently outperforms its baseline counterparts, rendering it a robust approach for cache-refreshing in dynamic IIoT environments.