Deep reinforcement learning enabling a BCFbot to learn various undulatory patterns

Abstract

In bio-inspired marine robots, one particular motion pattern is generally adopted to achieve benefits of that pattern. However, multiple gait patterns can be utilized together in a single biomimetic design to employ their benefits, as required. However, there is a lack of a unified control scheme that can be used to optimize and mimic undulatory patterns observed among different organisms in the body and/or caudal fin (BCF) category. Thus, central pattern generators (CPGs) were incorporated into a deep reinforcement learning (DRL) architecture to train a robot to develop various swimming gaits. The proposed framework can not only develop and optimize distinct motion patterns but also seamlessly and instantly switch between them. Oscillators integrated into a learning paradigm provide a bioinspired framework to systematically develop a variety of swimming gaits. The prototyped BCFbot has multiple joints, which makes it easy to realize more than one tail undulation patterns. Three different swimming patterns (anguilliform, sub-carangiform, and carangiform) were learned through simulation and then verified on a physical robot. Testing and comparison results show that the claimed benefits of the three benchmark motion patterns can be well realized using the developed robot and can be freely switched and optimized using the developed DRL mechanism. This should be the first attempt for achieving a multimotion pattern optimization and switching within a single BCFbot and demonstrating a successful motion generation regime similar to a real animal. • Multiple gait patterns can be employed on a single robot to exploit their benefits as needed. • Central pattern generators are incorporated into reinforcement learning to develop various swimming gaits. • Each single motion pattern can be generated and optimized simultaneously. • Three different swimming patterns are learned in simulation and then verified on a physical robot. • The first attempt for achieving a multi-motion pattern optimization and switching within a single robot.