Adaptive actor-critic control of robots with integral invariant manifold

Abstract

The actor-critic scheme stands for a powerful algorithm to design controllers for linear and non-linear systems subject to changing or highly uncertain dynamics. In particular, the actor-critic scheme that has succeeded is typically based on two neural network stages in a hierarchical architecture where the critic stage approximates the reward cost function. In contrast, the dynamic of the system is estimated by another neural network in the actor stage. This paper proposes an adaptive actor-critic robot learning on a lower dimension invariant error manifold as part of the Performance Evaluator. The proposed scheme guarantees an envelope of exponential convergence of tracking errors using a modified Lyapunov function, throughout integral sliding mode enforced for all time, where this becomes fundamental to drive also the learning of Reward function. Simulations show a non-linear dynamical robot learning tracking a time-varying trajectory under this Reinforcement Learning scheme.