CottonSim: A vision-guided autonomous robotic system for cotton harvesting in Gazebo simulation

Abstract

Cotton is a major cash crop in the United States, with the country being a leading global producer and exporter. Nearly all U.S. cotton is grown in the Cotton Belt, spanning 17 states in the southern region. Harvesting remains a critical yet challenging stage, impacted by the use of costly, environmentally harmful defoliants and heavy, expensive cotton pickers. These factors contribute to yield loss, reduced fiber quality, and soil compaction, which collectively threaten long-term sustainability. To address these issues, this study proposes a lightweight, small-scale, vision-guided autonomous robotic cotton picker as an alternative. An autonomous system, built on Clearpath’s Husky platform and integrated with the Cotton-Eye perception system, was developed and tested in the Gazebo simulation environment. A virtual cotton field was designed to facilitate autonomous navigation testing. The navigation system used Global Positioning System (GPS) and map-based guidance, assisted by an RGB-depth camera and a YOLOv8n-seg instance segmentation model. The model achieved a mean Average Precision ( m A P ) of 85.2%, recall of 88.9%, and precision of 93.0%. The GPS-based approach reached a 100% completion rate ( C R ) within a (5e-6)° threshold, while the map-based method achieved a 96.7% C R within a 0.25 m threshold. The developed Robot Operating System (ROS) packages enable robust simulation of autonomous cotton picking, offering a scalable baseline for future agricultural robotics. CottonSim code and datasets are publicly available on GitHub: https://github.com/imtheva/CottonSim . • A Gazebo-based cotton-picking simulator was developed for virtual field testing. • A vision-guided robot was implemented for autonomous navigation and cotton-picking. • CottonSim’s GPS-based vision control achieved 100% accuracy in cotton field traversal. • The system achieved high-precision robot positioning close to the planned trajectory.