GNN Topology Representation Learning for Deformable Multi-Linear Objects Dual-Arm Robotic Manipulation

Abstract

Deformable Multi-Linear Objects (DMLOs), or Branched Deformable Linear Objects (BDLOs), are flexible objects that possess a linear structure similar to DLOs but also feature branching or bifurcation points where the object’s path diverges into multiple sections. The representation of complex DMLOs, such as wiring harnesses, poses significant challenges in various applications, including robotic systems’ perception and manipulation planning. This paper proposes an approach to address the robust and efficient estimation of a topological representation for DMLOs leveraging a graph-based description of the scene obtained via graph neural networks. Starting from a binary mask of the scene, graph nodes are sampled along the objects’ estimated centerlines. Then, a data-driven pipeline is employed to learn the assignment of graph edges between nodes and to characterize the node’s type based on their local topology and orientation. Finally, by utilizing the learned information, a solver combines the predictions and generates a coherent representation of the objects in the scene. The approach is experimentally evaluated using a test set of complex real-world DMLOs. Within an offline evaluation, the proposed approach achieves a Dice score exceeding 90% in predicting graph edges. Similarly, the identification accuracy of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">branch</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intersection</i> points in the graph topology is above 90%. Additionally, the method demonstrates efficient performance, achieving a runtime of over 20 FPS. In an online assessment employing a dual-arm robotic setup, the approach is successfully applied to disentangle three automotive wiring harnesses, demonstrating the effectiveness of the proposed approach in a real-world scenario.