Data-efficient Co-Adaptation of Morphology and Behaviour with Deep\n Reinforcement Learning

Abstract

Humans and animals are capable of quickly learning new behaviours to solve\nnew tasks. Yet, we often forget that they also rely on a highly specialized\nmorphology that co-adapted with motor control throughout thousands of years.\nAlthough compelling, the idea of co-adapting morphology and behaviours in\nrobots is often unfeasible because of the long manufacturing times, and the\nneed to re-design an appropriate controller for each morphology. In this paper,\nwe propose a novel approach to automatically and efficiently co-adapt a robot\nmorphology and its controller. Our approach is based on recent advances in deep\nreinforcement learning, and specifically the soft actor critic algorithm. Key\nto our approach is the possibility of leveraging previously tested morphologies\nand behaviors to estimate the performance of new candidate morphologies. As\nsuch, we can make full use of the information available for making more\ninformed decisions, with the ultimate goal of achieving a more data-efficient\nco-adaptation (i.e., reducing the number of morphologies and behaviors tested).\nSimulated experiments show that our approach requires drastically less design\nprototypes to find good morphology-behaviour combinations, making this method\nparticularly suitable for future co-adaptation of robot designs in the real\nworld.\n