SA-DEM: Dexterous Extrinsic Robotic Manipulation of Non-Graspable Objects via Stiffness-Aware Dual-Stage Reinforcement Learning

Abstract

We propose a novel framework, SA-DEM, for extrinsic non-grasping manipulation of ungraspable objects in robotics. This approach is grounded in dual-stage reinforcement learning and decouples the overall task into two sequential phases: interactive mode decision-making and manipulation action planning. Notably, this framework innovatively incorporates the stiffness information of manipulated objects into the decision-making process, enabling the robot to autonomously perceive, decide, and plan manipulation strategies for objects with diverse physical attributes. The first phase of SA-DEM involves a high-level agent responsible for planning the grasping pose of objects and their interaction locations with the environment, based on the initial state of the objects, observations from environmental point clouds, stiffness representations, and prior knowledge of grasping regions. The second phase is executed by a low-level agent, which focuses on planning specific manipulation actions such as poking and flipping. These actions are derived from autonomous exploration during the training process, negating the need for manual customization. Both agents employ a hybrid discrete-continuous action space along with time-abstracted and spatially grounded representations centered around the point cloud, culminating in a unified actor-critic reinforcement learning framework.