Curriculum Reinforcement Learning for Obstacle Avoidance Postures for a Hyper-redundant Manipulator

Abstract

Redundant robots with more degrees of freedom than necessary for given tasks have attracted attention due to their flexibility, but they also increase the complexity of control. Especially for highly redundant robots, accurate motion planning and obstacle avoidance remain challenging. This research aims to develop a redundant robot arm that can perform reaching tasks while avoiding randomly appearing obstacles using reinforcement learning. We adopted the Proximal Policy Optimization (PPO) algorithm and conducted simulations in the Mujoco environment. The learning process consisted of a three-stage curriculum: reaching task, fixed obstacle avoidance, and random obstacle avoidance, gradually increasing difficulty to achieve efficient learning. Experimental results showed that the arm could adapt to complex environments and effectively reach target positions while avoiding obstacles. In particular, the system demonstrated high adaptability to randomly placed obstacles, successfully reaching within a maximum distance of approximately 0.07 m from the target position.