Dynamic Task Allocation for Robotic Edge System Resilience Using Deep Reinforcement Learning

摘要

Incorporating edge and cloud computing with robotics provides extended options for robots to perform real-time sensing and actuation operations in various cyber–physical systems (CPSs), including smart farms. Such systems are prone to uncertain failures triggered by mechanical disruptions. Consequently, the overall system performance degrades, primarily when location-specific tasks are already assigned to a faulty robot and require immediate recovery. Using edge and cloud computing resources is not always feasible due to communication and latency constraints. Therefore, this article exclusively focuses on harnessing the mobility of robots to support the computation tasks affected by uncertain failures of previously assigned robots and ensure faster resiliency management by relocating active robots near task sources. The proposed mobility-as-a-resilience-service (MaaRS) is formulated using a Markov decision process (MDP). Later, an edge server proximal to the robots is trained using deep reinforcement learning (DRL) to assign tasks among the robots. Specifically, a multiple deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -network (MDQN)-based dynamic task allocation mechanism is proposed to converge to a solution exploring reward uncertainties with the best exploitation. Numerical evaluation using Python and TensorFlow validates the effectiveness of the proposed approach compared to other benchmarks.