Reinforcement Learning of an Interpretable Fuzzy System Through a Neural Fuzzy Actor-Critic Framework for Mobile Robot Control

摘要

This paper proposes a method of reinforcement learning (RL) of an interpretable fuzzy system (IFS) based on a neural fuzzy actor-critic (RLIFS-NFAC) framework. The critic in the RLIFS-NFAC framework is implemented through a neural fuzzy critic (NFC), which is built through structure and parameter learning using temporal difference error. Structure learning employs an online clustering method to generate new rules. The actor is implemented through a neural fuzzy actor (NFA), which aims to build an IFS through structure and parameter learning. For the parameter learning of the NFA, a value function considering control performance is defined and optimized through the deterministic policy gradient algorithm. In addition, two objective functions considering model interpretability are included. Parameter learning for optimization of the objective functions together with a fuzzy set clustering and merging operation help reduce model complexity and fuzzy set transparency. The RL of the RLIFS-NFAC framework using an on-policy or off-policy method is proposed. RLIFS-NFAC is applied to the RL of a wheeled robot to learn a wall-following behavior. Simulations with comparisons with different reinforcement fuzzy systems and neural networks show the effectiveness and efficiency of the proposed RLIFS-NFAC. The learned IFS has also been successfully applied to control a real wall-following robot in unknown environments.