Development and implementation of a hybrid visual prediction algorithm for robotic smart tomato harvesting

Abstract

This paper introduces new algorithms designed specifically for smart tomato harvesting, which combines predictions from two independently trained YOLOv8 (You Only Look Once version 8) models, each specialized on different datasets to detect tomatoes and classify their ripeness stages—ripe, unripe, and semi-ripe—while also computing their center points. To enhance detection accuracy under varying field conditions, multiple fusion strategies were developed, including a hybrid algorithm that integrates union and confidence-weighted summation methods. The Hybrid Algorithm achieved the best performance, surpassing the original models and other fusion techniques. It attained F1 scores of 0.697 and 0.694 at confidence thresholds of 0.5 and 0.9, respectively, compared to the original models' F1 scores of 0.670 and 0.648 at confidence 0.5, and 0.481 and 0.497 at confidence 0.9. This corresponds to an improvement of 4.0 % and 7.6 % at confidence 0.5, and 44.3 % and 39.6 % at confidence 0.9, demonstrating the hybrid algorithm's stability and superiority, particularly at higher thresholds. Furthermore, the system utilizes a high-resolution RGB (Red, Green, Blue) camera for real-time image capture, enhancing model performance in complex agricultural environments. This study validates the effectiveness of confidence-based fusion in developing robust and accurate vision systems for precision agriculture.