A Smart Interactive Camera Robot Based on Large Language Models

Abstract

The emergence of large language models (LLMs) has paved the way for advancing robotics capabilities, especially in intricate tasks that demand nuanced comprehension and precision. In this context, this paper introduces a novel interactive camera robot that harnesses LLMs to enhance human-robot interaction and optimize robot control. Specifically, an innovative technique that leverages the language understanding capabilities of LLMs to plan camera movement trajectories and waypoints is presented. In this study, the geometric relationships among the objects under capture are employed to plan the control strategy. Accordingly, this approach not only empowers sophisticated camera parameter manipulation and color adjustments but also fosters a natural and efficient human-robot interaction. Lots of experiments on real robots are conducted to evaluate the effectiveness of the proposed method under various scenarios. The results reveal robust performance across crucial measures, affirming the substantial potential of LLMs in elevating camera robot control and interaction experience. Videos of our experiments are available at https://youtu.be/zP-sTZHvXe4.