Tomato Ripeness Evaluation and Localization Using Mask R-CNN and DBSCAN Clustering

Abstract

This paper presents a deep learning approach using Mask R-CNN for detecting and classifying the ripeness of tomatoes in greenhouse imagery with the goal of enabling automated robotic harvesting. The Mask R-CNN model is trained on a small dataset of 62 annotated tomato images captured under challenging real-world conditions. Despite the limited data, the network achieves promising accuracy in classifying individual tomatoes into unripe, semi-ripe, and ripe categories. To move beyond standalone ripeness classification to robotic picking, density-based clustering with DBSCAN is applied to group tomatoes into harvestable formations based on their spatial and ripeness characteristics. The dual method of Mask R-CNN detection and DBSCAN clustering provides a complete pipeline from ripeness evaluation to tomato localization within dense clusters. Results demonstrate the feasibility of the approach for agricultural applications like selective robotic harvesting based on computer vision and deep learning.