Robotic system with tactile-enabled leaf tracking for high-resolution hyperspectral imaging device for autonomous corn leaf phenotyping in controlled environments

摘要

Hyperspectral imaging of individual corn leaves provides valuable data for analyzing nutrient content and diagnosing diseases. However, existing leaf-level imaging techniques face challenges such as low spatial resolution and labor-intensive processes. To address these limitations, this study developed a robotic system integrated with a high-resolution line-scanning hyperspectral imaging device to autonomously scan a corn leaf. The hyperspectral imaging device used a vision-based tactile sensor for active leaf tracking throughout the scanning process, ensuring high image quality. Additionally, the device incorporated an in-hand leaf manipulation mechanism that ensured the leaf was properly positioned on the tactile sensing area at the start of every scanning. The scanning process was executed by a robotic arm equipped with an RGB-D camera and integrated with the Segment Anything Model (SAM), enabling autonomous leaf detection, localization, grasping, and scanning. The system was tested on V10-stage corn plants and the success rate was 91.4 % with an average 4.8 s for leaf detection and localization and an average leaf scanning time of 38.3 s.