Design of obstacle avoidance algorithm for mobile robots in crowd environment empowered by deep reinforcement learning

摘要

In most warehouse environments at present, the accumulation of goods is complex, and the control of goods by management personnel interacts with the trajectories of warehouse mobile robots. Traditional mobile robots are unable to provide accurate obstacle avoidance strategies in response to goods and pedestrians. To enable mobile robots to complete the obstacle avoidance task efficiently and smoothly in the warehouse environment, a mobile robot obstacle avoidance algorithm based on deep reinforcement learning is proposed in this paper, which is specifically designed for the warehouse context. The value function network in the deep reinforcement learning algorithm has insufficient learning capabilities. To address this, based on the interaction with pedestrians, the value function network is enhanced by extracting the interaction information of pedestrians via the pedestrian- angle grid and the temporal features of each pedestrian through the attention mechanism, which enables the determination of the relative importance of the current state and the historical trajectory state, as well as their combined influence on the robot’s obstacle avoidance strategy, providing a basis for the subsequent learning of multi-layer perception machines. Additionally, the reward function of reinforcement learning is designed based on the spatial behavior of pedestrians, and the robot is penalized when the angle changes excessively to meet the requirement of comfortable obstacle avoidance, so that the robot can better adapt to the complex warehouse environment and avoid obstacles in a more natural and efficient way, thereby improving the overall performance and safety of the robot in the warehouse.