Acceleration of Actor-Critic Deep Reinforcement Learning for Visual Grasping by State Representation Learning Based on a Preprocessed Input Image

Abstract

For robotic grasping tasks with diverse target objects, some deep learning-based methods have achieved state-of-the-art results using direct visual input. In contrast, actor-critic deep reinforcement learning (RL) methods typically perform very poorly when applied to grasp diverse objects, especially when learning from raw images and sparse rewards. To render these RL techniques feasible for vision-based grasping tasks, we used state representation learning (SRL), in which we encode essential information for subsequent use in RL. However, typical representation learning procedures are unsuitable for extracting pertinent information for learning grasping skills owing to the high complexity of visual inputs for representation learning, in which a robot attempts to grasp a target object. We found that the proposed preprocessed input image is the key to capturing effectively a compact representation. This enables deep RL to learn robotic grasping skills from highly varied and diverse visual inputs. Further, we demonstrate the effectiveness of the proposed approach with varying levels of preprocessing in a realistic simulated environment. We also describe how the resulting model can be transferred to a real-world robot and also demonstrate a 68% success rate on real-world grasp attempts.