Procedural Generation of Synthetic Dataset for Robotic Applications in Sweet Pepper Cultivation

Abstract

In this paper, a deep learning based object detection model is developed for robotic applications in structured agricultural cultivation of sweet peppers (Capsicum annuum). We propose a realistic simulation based method for procedural generation of a labeled sweet pepper dataset. Realistic 3D models of fruit, leaves, and other relevant plant body parts are generated through photogrammetry, enabling generation of a large and relevant dataset. The dataset is labeled for both object detection and semantic segmentation by scene rendering in multiple layers and using artificial depth information. Smart augmentation techniques are deployed in procedural generation, as well as label filtering techniques for more realistic dataset labels. A MobileNet-SSD based deep learning model is trained through transfer learning on the generated dataset. The trained model is tested both in simulated environment, as well as on a relevant real-world setup.