Robust Visual Servoing and CNN-Based Thermal Imaging for Sugarcane Row Following With a Skid-Steering Mobile Robot

Abstract

The escalating demand for accuracy and efficiency in agriculture—driven by population growth and climate change—has increased the use of mobile robots in precision agriculture. Sugarcane production is particularly significant, providing both sugar yield and renewable energy sources such as bioplastic and biofuel from cane bagasse. Like other grass-type crops (e.g., wheat, corn, rice, barley, and bamboo), sugarcane fields pose unique challenges due to vast plantation areas and dense vegetation forming hard-to-access crop row tunnels. These conditions hinder both manual labor and autonomous robot operation, requiring higher investments in robot swarms with robust navigation systems to handle noisy imaging and low Global Navigation Satellite System coverage. This paper introduces a visual navigation system for TIBA, a cost-effective, compact, rugged skid-steering mobile robot equipped with customized sensors and designed for swarm-based operations in sugarcane fields. We propose a combination of infrared (thermal) imaging with robust control to address the locomotion challenge in sugarcane fields. Thermal images reveal temperature differences between ground and cane stalks, enabling clearer visualization of the row corridor despite occluding leaves. Consequently, computationally efficient algorithms, such as a simple convolutional neural network, can generate reliable trajectories for the robot to follow along the crop row, avoiding lateral collisions with the side plants. A sliding mode control approach ensures robust tracking under uncertainties and disturbances in both robot and camera models. We also introduce IRNavSim, a Matlab-based simulation environment tailored for crop row following tasks using thermal imaging, which helps us to evaluate the system’s performance and behavior.