DQDWA: Dynamic Weight Coefficients Based on Q-learning for Dynamic Window Approach Considering Environmental Situations

Abstract

Autonomous mobile robots are used in a wide range of industrial application. Dynamic window approach (DWA) is one of effective local path planning methods considering collision avoidance and kinematic constraints. DWA selects the optical path from path candidates from velocity space by using an evaluation function with fixed weight coefficients. These fixed weight coefficients are designed for the specific environmental situation. Therefore, if the environmental situation such as congestion, road width, and obstacles changes, the evaluation function with fixed weight coefficients may select the inefficient path or path with the collision. To address this issue, this paper proposes the dynamic weight coefficients based on Q-learning for DWA considering environmental situations (DQDWA). Q-learning is one of reinforcement learning methods. The Q-table in DQDWA consists of states of robot and environmental situations, and actions of weight coefficients in DWA evaluation function. By using the learned Q-table, DQDWA dynamically selects weight coefficients and, the optimal path considering environmental situations is generated. The effectiveness of the proposed method was confirmed through simulations.