A Reinforcement Learning Method for Continuous Domains Using Artificial Hydrocarbon Networks

Abstract

Reinforcement learning in continuous states and actions has been limitedly studied in ocassions given difficulties in the determination of the transition function, lack of performance in continuous-to-discrete relaxation problems, among others. For instance, real-world problems, e.g. robotics, require these methods for learning complex tasks. Thus, in this paper, we propose a method for reinforcement learning with continuous states and actions using a model-based approach learned with artificial hydrocarbon networks (AHN). The proposed method considers modeling the dynamics of the continuous task with the supervised AHN method. Initial random rollouts and posterior data collection from policy evaluation improve the training of the AHN-based dynamics model. Preliminary results over the well-known mountain car task showed that artificial hydrocarbon networks can contribute to model-based approaches in continuous RL problems in both estimation efficiency (0.0012 in root mean squared-error) and sub-optimal policy convergence (reached in 357 steps), in just 5 trials over a parameter space θ ∈ R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">86</sup> . Data from experimental results are available at: http://sites.google.com/up.edu.mx/reinforcement-learning/http://sites.google.com/up.edu.mx/reinforcement-learning/.