Framework for LLM-Enabled Construction Robot Task Planning: Knowledge Base Preparation and Robot–LLM Dialogue for Interior Wall Painting

Abstract

Task planning for a construction robot requires systematically integrating diverse elements, such as building components, construction processes, user input, and robot software. Conventional robot programming complicates this by requiring precise entity naming, relationship definitions, unstructured language interpretation, and accurate action selection. Existing research has focused on isolated components, such as natural language processing, hardcoded data linkages, or BIM data extraction. We introduce a novel framework using an LLM as the cognitive core for autonomous construction robots, encompassing both data preparation and task planning phases. Leveraging OpenAI’s ChatGPT-4, we demonstrate how LLMs can process structured BIM data and unstructured human inputs to generate robot instructions. A prototype tested in a simulated environment with a mobile painting robot adaptively executed tasks through real-time dialogues with ChatGPT-4, reducing reliance on hardcoded logic. Results suggest that LLMs can serve as the cognitive core for construction robots, with potential for extension to more complex operations.