Robotic Arm-Based Spectral Sensing for Strawberry Positioning and Non-Destructive Sweetness Measurement

Abstract

Accurate assessment of sweetness is essential for quality control in agriculture, yet conventional methods rely on destructive sampling and are difficult to scale. This thesis presents a robotic arm-based spectral sensing system for strawberry detection, localization, approach, and non-destructive sweetness estimation. The system integrates perception, calibration, and robotic control in a closed-loop pipeline. A YOLOv11s detector is adopted for real-time strawberry detection, while RGB-ToF calibration and mask-to-depth alignment are used to obtain geometrically consistent target localization. A custom eye-in-hand hand-eye calibration workflow is developed to estimate the rigid transform between gripper_link and cam_front, enabling reliable transformation of fruit targets into the robot base frame. Based on these estimates, the robot executes a waypoint-based search and an incremental closed-loop approach strategy to position the sensor at optimal working distance for sweetness sensing. Experimental results show strong end-to-end performance (88.10% success over 42 trials), with robust detection (95.24%) and successful approach execution once a target is detected (100% conditional success). Hand-eye calibration comparisons indicate that although Andreff yields the smallest translation norm in single-run results, the Park method provides better cross-sample consistency and therefore more stable downstream robot behavior. The residual failures are concentrated in the sensing stage, especially valid-region extraction for sweetness estimation under difficult depth/reflectance conditions. Overall, this work demonstrates the feasibility of integrating RGB-ToF perception, robotic manipulation, and non-destructive sensing for practical strawberry quality assessment, and provides a scalable baseline for future integration of learning-based policies such as Vision-Language-Action models.