Development of an intelligence vision for a robot system to pick and place objects

Abstract

This paper presents an automated pick-and-place robotic system utilizing stereo vision technology for object detection and localization in 3D space. Stereo vision is an optimal choice for short-range industrial applications due to its capability of providing accurate depth measurements at a reasonable cost, outperforming alternatives such as LiDAR or Time-of-Flight (ToF) cameras in similar settings. The proposed system is designed to operate reliably under natural lighting conditions, making it well-suited for deployment in factory production lines. An Intel RealSense D435 camera is employed to capture both RGB and depth images from the environment. Object detection is performed using a YOLOv11-based model, achieving high detection accuracy with a mean average precision (mAP50) of 98.5% across all object classes. The system processes depth information to identify the topmost object, estimates its 3D coordinates with minimal errors (average positional errors below 5.3 mm), and transmits the data to a robotic manipulator for execution of the pick-and-place task. Experimental results demonstrate the system's high precision and reliability in object detection and 3D localization.