Rebar grasp detection using a synthetic model generator and domain randomization

Abstract

The increasing demand for automated rebar cage assembly in the construction industry highlights the need for flexible rebar grasping solutions. This paper proposes a grasp detection method that enables robotic arms to autonomously grasp rebars from the top layer of stacks, eliminating the need for complex delivery systems. To support this, a synthetic dataset pipeline incorporating domain randomization is developed, which facilitates robust rebar instance segmentation without the need for labor-intensive real-world data collection. Within this pipeline, a fully-parameterized rebar generator is proposed to eliminate the reliance on manual modeling in data generation, allowing an infinite generation of rebar datasets with realistic and diverse appearances and shapes. Real-world experiments demonstrated a segmentation accuracy of 87.9 for rebars in the top layer and a 91.6 % grasping success rate on the first attempt, validating the proposed methods. Additionally, an ablation study highlighted the significance of rebar stacking, lighting, and camera pose variations in improving the model performance in real-world scenarios. • Autonomous grasp detection approach enables robotic arms to efficiently grasp rebars. • A parameterized generator creates limitless, realistic, diverse rebar models. • Synthetic data enables rebar segmentation, reducing real-world data reliance. • Real-world tests validate the applicability of the proposed methods. • Domain randomization helps bridge the sim-to-real gap effectively.