Autonomous Robotic Assembly and Sequence Planning Based on YOLOv8

Abstract

This study contributes to autonomous assembly by introducing a robotic building system designed to observe its workspace, plan building sequences, determine appropriate grasps and manipulator motions, and operate hardware in real-time. This study involves the creation of a wooden brick dataset and employs a YOLOv8 deep learning model for object segmentation in two implemented real-time algorithms, namely, Desired Construction Object Detection (DCOD) and Robot Current Frame Object Detection (RCFOD). The DCOD algorithm is designed for the desired structure, while the RCFOD algorithm is utilized for detecting objects within the operational environment of the robot. By comparing the outputs of these two algorithms, sequences of objects for assembly are formulated, and the appropriate grasp positions are determined. Subsequently, the robot’s motion planning for both grasping and construction tasks is executed. Due to the limitations of using a single RGB gripper-mounted camera, various issues, such as failing to recognize items with the same cross-section but varying heights, have been addressed in this study. The results of this study indicate the effective learning of the YOLOv8 model, as shown by a loss value below 0.2 and an accuracy over 90% for 17 out of the total 18 classes. Practical implementation is done by a common industrial Delta parallel robot, a 2-finger gripper, and a simple RGB camera. Overview of practical implementation system is shown in Fig. 1. The proposed robotic assembly method demonstrates its effectiveness in autonomously constructing structures utilizing a reduced and cost-efficient configuration.