Single-Shot Visual Relationship Detection for the Accurate Identification of Contact-Driven Hazards in Sustainable Digitized Construction

Abstract

Deploying construction robots alongside workers presents the risk of unwanted forcible contact—a critical safety concern. To address a semantic digital twin where such contact-driven hazards can be monitored accurately, the authors present a single-shot deep neural network (DNN) model that can perform proximity and relationship detections simultaneously. Given that workers and construction robots must sometimes collaborate in close proximity, their relationship must be considered, along with proximity, before concluding an event is a hazard. To address this issue, we leveraged a unique two-in-one DNN architecture called Pixel2Graph (i.e., object + relationship detections). The potential of this DNN architecture for relationship detection was confirmed by follow-up testing using real-site images, achieving 90.63% recall@5 when object bounding boxes and classes were given. When integrated with existing proximity monitoring methods, single-shot visual relationship detection will enable the accurate identification of contact-driven hazards in a digital twin platform, an essential step in realizing sustainable and safe collaboration between workers and robots.