Neural Q-Learning controller for mobile robot

Abstract

In recent years, increasing trend in application of autonomous mobile robot worldwide has highlighted the importance of path planning controller in robotics-related fields, especially where dynamic and unknown environment is involved. Writing a good robot controller program can be a very time consuming process. It is inevitably wasting of resources and efforts if we have to rewrite the controller over and over again whenever there is emergence of changes in the environment. Reinforcement Learning (RL) algorithms and Artificial Neural Network (ANN) are used to assist autonomous mobile robot to learn in an unrecognized environment. This research study is focused on exploring integration of multi-layer neural network and Q-Learning as an online learning controller. Learning process is divided into two stages. In the initial stage the agent will map the environment through collecting state-action information according to the Q-Learning procedure. Second training process involves neural network training which will utilize the state-action information gathered in earlier phase as training samples. During final application of the controller, Q-Learning would be used as the primary navigating tool whereas the trained neural network will be employed when approximation is needed. MATLAB simulation was developed to verify the validity of the algorithm before it is real-time implemented on the real world using Team AmigoBot™ robot. The results obtained from both simulation and actual application confirmed on-spot learning ability of the controller accompanied with certain degree of flexibility and robustness.