Comparative Analysis of Reinforcement Learning Techniques in Simulated Robotics Environments

Abstract

Machine learning is a critical component of artificial intelligence and computer science that uses data and algorithms to emulate the way that humans learn and predict. Reinforcement learning, one of the many forms of machine learning, trains an agent through trial and error, with successful outcomes being rewarded and unsuccessful outcomes being penalized. The implementation of three reinforcement learning algorithms, Qlearning, Deep Q-Learning, and Double Deep Q-Learning, was tested using a series of ROS (Robot Operating System) packages in the Gazebo simulation environment. The Python openai_ros package and Gym libraries were used to input a robot into a configurable environment and test out the performance of a specific reinforcement learning algorithm. The agent’s action space was discretized into three specific actions, move forward, turn left, or turn right, that the robot could take. The reward or punishment was given after each step based on if the current action moved the robot towards the goal state or away from it. For each algorithm, the agent was first pre-trained on an obstacle free environment for 200 episodes and then was placed in an environment with two box-shaped obstacles to avoid. The algorithm/neural network parameters from the pre-training environment were loaded prior to the agent being tested in the new environment. The goal state at each episode was randomly generated and the robot’s performance was evaluated using the cumulative reward achieved per episode and the number of times it reached the goal.