A Method Based on Multi-Modal Fusion of RGB-D Images for Detecting the Grasping Pose of a Robotic Arm

Abstract

The detection of the grasping pose of the robotic arm is a key technology for robots, and it is also the key to efficient, accurate, and real-time robot motion. In complex environments, the imprecise representation of grasping areas and the insufficient accuracy of grasping poses pose challenges to robotic arm manipulation. To address this, we propose a multi-modal fusion approach based on RGB-D images for detecting grasping poses in robotic arm manipulation. The multi-modal fusion module is used to integrate the feature outputs from the depth and RGB image sub-modules, supplementing the information from each modality and resolving the issue of ambiguous detection regions. Subsequently, the integrated features are fed into an improved grasping convolutional module. By directing different convolutional branches to focus on distinct grasping tasks, our approach addresses the high-precision grasping pose prediction problem. Extensive research and experimentation validate the proposed method, achieving accuracies of 98.8% and 97.7% on the Cornell and Jacquard datasets, respectively.