Actor-Hybrid-Attention-Critic for Multi-Logistic Robots Path Planning

Abstract

With the proliferation of express delivery services, the demand for intelligent logistic services is increasing. More and more robots are operating simultaneously, which poses a great challenge to their extraction of key information from a large number of static and dynamic environments. To address this, a Multi Actor Hybrid Attention Critic (MAHAC) algorithm for path planning of multi-logistic robots is proposed. The reward function is improved, based on collision avoidance and accurate arrival at the target location, a power shortage and sparse penalty are added to improve the efficiency of path planning. A hybrid attention mechanism module is designed, with an imbedded Multi Layer Perceptron (MLP), to calculate the similarity between key and query. Combining the hybrid attention mechanism with the Multi Actor Attention Critic (MAAC) algorithm improves the ability of multi-logistic robots to extract critical states and action information in a variety of environments. In a simulated warehouse layout based on Amazon's Kiva system, the MAHAC algorithm is compared with MAAC, Multi-Agent Deep Deterministic Policy Gradient (MADDPG), and Multi-Actor Soft Actor-Critic (MASAC) algorithms. The MAHAC has a higher reward value, so as to select the better path in the least time, with the highest accuracy and no collisions, and hence is suitable for the path planning scenario of multi-logistic robots.