A Distributed Actor‐Critic Learning Approach for Affine Formation Control of Multi‐Robots With Unknown Dynamics

Abstract

ABSTRACT Formation maneuverability is particularly important for multi‐robots (MRs), especially when the robots are operating cooperatively in complex and dynamic environments. Although various methods have been developed for affine formation, it is still a difficult problem to design an affine formation controller for MRs with unknown dynamics. In this paper, a distributed actor‐critic learning approach (DACL) in a look‐ahead rollout manner is proposed for the affine formation of MRs under local communication, which improves the online learning efficiency. In the proposed approach, a distributed data‐driven online optimization mechanism is designed via the sparse kernel technique to solve the near‐optimal affine formation control issue of MRs with unknown dynamics as well as improve control performance. The unknown dynamics of MRs are learned offline based on precollected input‐output datasets, and the sparse kernel‐based approach is employed to increase the feature representation capability of the samples. Then, the proposed distributed online actor‐critic algorithm for each robot in the formation includes two neural networks, which are utilized to approximate the costate functions and the near‐optimal policies. Moreover, the convergence analysis of the proposed approach has been conducted. Finally, numerical simulation and KKSwarm‐based experiment studies are performed to verify the effectiveness of the proposed approach.