Reinforcement Learning for Robotic Singulation: Policy Space Impact on Interactive Manipulation

Abstract

This paper, entitled Reinforcement Learning for Robotic Singulation: Policy Space Impact on Interactive Manipulation, investigates how the design of policy space influences reinforcement learning (RL) performance in robotic manipulation. Specifically, we compare Proximal Policy Optimization (PPO) with continuous action space against Deep Q-Learning (DQL) with discretized action space in a simulated singulation environment. The objective is to analyze how action space complexity affects convergence speed, policy stability, and manipulation success. Empirical findings show that while PPO policies capture fine-grained motions, they suffer in multimodal reward landscapes, whereas DQL demonstrates higher robustness, faster convergence, and consistent success rates in cluttered scenarios. This study highlights the critical role of action space design in RL-based robotic singulation, providing actionable insights for robust and scalable real-world deployment.