Rubber Tapping Position and Harvesting Cup Detection Using Faster-RCNN with MobileNetV2

Abstract

This paper presents the detection of rubber tree (Hevea brasiliensis) tapping position (tapping-path) and trunk-mounted harvesting cup on RGB-D images, which is the machine vision part of automatic rubber farming system. RGB-D images are collected in real tapping environment. Camera is placed about 1.00-meter distance to the trunk which is feasible for robot platform. Faster-RCNN (Region-based Convolutional Neural Network) object detector is proposed with ImageNet pretrained Mobilenet-v2 as the feature extraction layers to detect multiple object's class - tapping path and harvesting cup. Our experiments show that cup detection achieve higher average precision on grayscale depth image detector than only RGB image detector. We also propose 3-channel combinations from 4 grayscales (RGB-D) then, put into specific detectors. Detector's performances are computed using 0.5 and 0.75 intersection over union thresholds (IoU). The results show that Faster-RCNN with Mobilenet-v2 tapping position and cup detection can reach highest 0.95 [email protected] IoU estimated with k-fold cross validation (k=5). Our main contribution is that detection of harvesting cup using preprocessed grayscale depth with color images are more well-localized than only color image, observed at 0.75 IoU but tapping position detection is highly depended on color images and this can be impacted to practical application since cup detection are not required lighting.