Episode-Fuzzy-COACH Method for Fast Robot Skill Learning

Abstract

To realize robot skill learning in the real world, reinforcement learning algorithms need to be applied in continuous problems with high sample efficiency. Hybrid intelligence is regarded as an available solution for this problem, due to the ability to speed up the learning process with human knowledge and experience. Therefore, we propose Episode-Fuzzy-COACH (COrrective Advice Communicated by Humans), to imitate human fuzzy logic and involve human intelligence in the learning process. In this framework, human knowledge and experience are involved in the learning process, which are provided by human feedback and fuzzy rules designed by human users. Moreover, it is combined with Path Integrals Policy Improvement ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$PI^{2}$</tex-math></inline-formula> ), to realize hybrid intelligence, which is used to realize fast robot skill learning. Throwing Movement Primitives proposed in this article is used to represent the policy of ball-throwing skill. According to the simulation results, the learning efficiency of our method is increased by 72% and 42.86%, respectively, compared with pure <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$PI^{2}$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$PI^{2}+$</tex-math></inline-formula> COACH. Our method validated in experiments is 46.67% more effective than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$PI^{2}+$</tex-math></inline-formula> COACH. The results also show that the performance of our method is not affected by users' knowledge level of the related field. It is proven that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$PI^{2}+$</tex-math></inline-formula> Episode-Fuzzy-COACH is available for fast robot skill learning.