Large Language Model for Intuitive Control of Robots in Micro-Assembly

Abstract

In an era of rapid technological advances in microdevices and photonics, the importance of efficient automation solutions is becoming increasingly evident. In particular, the automation of assembly processes is gaining importance as a significant portion of costs is incurred in this phase. Programming robots, especially in micro-assembly, requires a high level of expertise due to the complex assembly systems and processes. With the rapid development of increasingly powerful Large Language Models (LLMs), their use for programming and controlling robots is becoming more and more prevalent. However, previous approaches have been limited to the field of service robots. In this paper, we present a framework that uses an LLM as an intuitive user interface for robot control in the field of micro-assembly. We integrate an LLM into a framework based on ROS2 (Robot Operation System 2), which enables skill-based control and programming of the micro-assembly robot. LLMs offer an intuitive access to the robot skills, thus facilitating robot control for users without knowledge of programming and robots. We demonstrate how an advanced LLM functions as an efficient interface, interpreting user instructions in context and initiating corresponding actions. Using a use case with exemplary user queries, we evaluate the performance of the implemented framework. Finally, we highlight further improvements and application possibilities.