Converging Game Theory and Reinforcement Learning For Industrial Internet of Things

Abstract

The fifth-generation (5G) wireless network provides high-rate, ultra-low latency, and high-reliability connections that can meet the Industrial Internet of Things (IIoT) requirements in factory automation, especially for robot motion control. In this paper, we address 5G service provisioning in an automated warehouse scenario, where swarm robotics is controlled by an industrial controller that provides routing and job instructions over the 5G network. Leveraging the coordinated multipoint (CoMP), we formulate a time-varying joint CoMP clustering and 5G ultra-reliable low-latency communication (URLLC) beamforming design problem to control the robots that move around the automated warehouse for goods storage with the planned reference tracks. Traditional iterative optimization approaches are impractical in such a dynamic wireless environment due to high computational time. We propose a game-theoretic CoMP clustering algorithm combined with the Proximal Policy Optimization method to obtain a stationary solution closed to that of the exhaustive search algorithm considered as the global optimal solution.