Developing an Advanced Perception System for Robotic Attic Insulation Using Large Vision-Language Models and LiDAR

Abstract

Attic insulation in the construction industry involves significant risks due to confined, dust-laden spaces, high temperatures, and potential hazards such as falls and respiratory issues. Addressing these challenges, this paper focuses on developing a perception mechanism for robotic attic insulation, aiming to enhance both worker safety and operational efficiency. The cornerstone of this research is the creation of an advanced perception system designed to comprehensively analyze the complex environment of attic spaces and identify the insulated area. This system integrated a Large Vision-Language Model (LVLM) and LiDAR to become an extensive segmentation system adept at understanding various aspects of the construction environment. It accurately segments insulation materials from other objects, thereby facilitating robotic perception and operation in attic insulation tasks. In this system, LiDAR-scanned depth information and chromatic information help generate prompts to assist the LVLM. To evaluate the system’s effectiveness, the authors established a simulated attic environment and gathered a diverse real-life image data set. This data was crucial in testing the perception mechanism. The results of the experiments were significant, with the system achieving an Intersection over Union (IoU) metric of over 95%, indicating a high level of precision in image segmentation. This performance not only confirms the efficiency of the perception system but also highlights its potential in enhancing the automation of attic insulation. By reducing reliance on manual labor in hazardous conditions and improving operational precision, this vision-LiDAR integrated system represents a substantial advancement in the field of construction robotics, paving the way for safer and more efficient construction practices.