Cross-Task Collaborative Optimization Based on Knowledge Transfer for Soft Robot Design

Abstract

The automatic design of soft robots is an intertwined process of evolving morphology and learning control. As reinforcement learning is repeatedly used to learn the control policy for each candidate robot design, the design process becomes time-consuming. So far, the common design paradigm in robotics has been based on a single task. In fact, there is control similarity between different tasks. Learning a controller with combinatorial generalization capabilities across a variety of tasks can significantly reduce the computational cost of the design process. To this end, we propose a cross-task collaborative evolutionary algorithm that constructs a universal controller capable of solving a group of tasks simultaneously. Instead of “one robot, one controller, one task" paradigm, the proposed universal controller is to learn a control policy, which can generalize to unseen morphologies. After the controller learning on easy tasks, the universal controller can be further transferred to new hard tasks. Furthermore, the knowledge transfer is incorporated in the search strategy to enhance the performance of the universal controller. The experimental results on 13 test tasks demonstrate that the proposed algorithm outperforms the SOTA design algorithms on 8 of them. Compared to these algorithms, the proposed algorithm reduces the computational cost by 55% while achieving comparable performance, particularly for unseen hard tasks.