Interactive Learning of Continuous Actions from Corrective Advice Communicated by Humans

Abstract

An interactive learning framework that allows non-expert humans to shape a policy through corrective advice, using a binary signal in the action domain of the robot/agent, is proposed. One of the most innovative features of COACH (COrrective Advice Communicated by Humans), the proposed framework, is a mechanism for adaptively adjusting the amount of human feedback that a given action receives, taking into consideration past feedback. The performance of COACH is compared with the one of TAMER (Teaching an Agent Manually via Evaluative Reinforcement), ACTAMER (Actor-Critic TAMER), and an autonomous agent trained using SARSA(λ) in two reinforcement learning problems: ball dribbling and Cart-Pole balancing. COACH outperforms the other learning frameworks in the reported experiments. In addition, results show that COACH is able to transfer successfully human knowledge to agents with continuous actions, being a complementary approach to TAMER, which is appropriate for teaching in discrete action domains.