Physics-informed reinforcement learning framework for kinematics optimization of self-propelled articulated swimmers

摘要

Our study presents a novel optimization framework dedicated to refining the swimming gaits of self-propelled articulated swimmers. The approach integrates a fluid–structure interaction solver for multibody systems with a single-step deep reinforcement learning optimization algorithm. To overcome the computational costs incurred by evaluations during parameter search, we introduced controlled transfer learning to improve performance and efficiency. By leveraging pre-trained policies on low-fidelity models and adapting them to high-fidelity environments, the learning procedure can be accelerated with significantly less high-fidelity evaluations. Moreover, the optimization algorithm is complemented by an intricate mapping procedure designed to enforce stringent constraints derived from prior knowledge within the expansive high-dimensional design space. Then, this framework is applied to investigate the influence of segment length and number on the optimal swimming kinematics of an articulated fish model. Findings reveal that the variable-length approach may yield more parsimonious yet comparable models with fewer segments compared to the equal-length approach. This study contributes valuable insight into the design and behavior of both natural and robotic swimmers, paving the way for future advancements in optimization algorithms and fish body models.