From Words to Flight: Integrating OpenAI ChatGPT with PX4/Gazebo for Natural Language-Based Drone Control

Abstract

The rapid progress of large language models in recent years has opened new opportunities for human-robot interaction.In this paper, we propose a novel approach to control drones using natural language commands by integrating OpenAI ChatGPT with the PX4/Gazebo simulator.The proposed system enables users to interact with the simulator using everyday language, allowing them to control the drone's actions with ease, eliminating the need for extensive training in drone piloting.We discuss the implementation details, including the validation of the ChatGPT-generated commands and their translation into executable actions in the simulator.To the best of our knowledge, this is the first proposal of a verification and validation system for commands generated by ChatGPT and LLMs in general.Furthermore, we discuss the crafting of effective prompts and the essential criteria for doing so.Our approach demonstrates promising results in terms of both usability and reliability, paving the way for further research on natural languagebased control systems for robotics applications.