A Comparative Analysis of Cross-Validation Techniques for a Smart and Lean Pick-and-Place Solution with Deep Learning

Abstract

As one of the core applications of computer vision, object detection has become more important in scenarios requiring high accuracy but with limited computational resources such as robotics and autonomous vehicles. Object detection using machine learning running on embedded device such as Raspberry Pi provides the high possibility to detect any custom objects without the recalibration of camera. In this work, we developed a smart and lean object detection model for shipping containers by using the state-of-the-art deep learning TensorFlow model and deployed it to a Raspberry Pi. Using EfficientDet-Lite2, we explored the different cross-validation strategies (Hold-out and K-Fold). The experimental results show that compared with the baseline EfficientDet-Lite2 algorithm, our model improved the mean average precision (mAP) by 44.73% for the Hold-out dataset and 6.26% for K-Fold cross-validation. We achieved Average Precision (AP) of more than 80% and best detection scores of more than 93% for the Hold-out dataset. For the 5-Fold lean dataset, the results show the Average Precision across the three lightweight models are generally high as the models achieved more than 50% average precision, with YOLOv4 Tiny performing better than EfficientDet-Lite2 and Single Shot Detector (SSD) MobileNet V2 Feature Pyramid Network (FPN) 320 as a lightweight model.