Using redundant fitness functions to improve optimisers for humanoid robot walking

Abstract

Walking is an essential skill for a humanoid robot. Optimisation can be applied to improve the speed, efficiency and stability of an existing walk engine. A local optimiser is often employed for this task to reduce stress on the robot, however, they are prone to getting trapped in local extrema. This paper proposes an extended Policy Gradient Reinforcement Learning algorithm that includes opposition-based learning and redundant fitness functions. The algorithm is based on a local optimiser to minimise the stress on the robot during the optimisation, however, the algorithm is able to escape from local extrema using redundant fitness functions. The improved algorithm is used to optimise two existing omni-directional walk engines for the NAO robot, one in simulation, and another in hardware. It was found that the improved algorithm performed better than the standard version in both cases. Furthermore, the walk selected in hardware is the fastest to date using the Aldebaran walk engine. The proposed fitness functions are suitable for any humanoid robot. Therefore, the use of redundant fitness functions can be incorporated into the optimisation of any humanoid robot walk.