Deep Reinforcement Learning-Based Autonomous Docking with Multi-Sensor Perception in Sim-to-Real Transfer

摘要

Autonomous docking is a critical capability for enabling fully automated operations in industrial and logistics environments using Autonomous Mobile Robots (AMRs). Traditional rule-based docking approaches often struggle with generalization and robustness in complex, dynamic scenarios. This paper presents a deep reinforcement learning-based autonomous docking framework that integrates Proximal Policy Optimization (PPO) with multi-sensor fusion. It includes YOLO-based vision detection, depth estimation, and LiDAR-based orientation correction. A concise 4D state vector, comprising relative position and angle indicators, is used to guide a continuous control policy. The outputs are linear and angular velocity commands for smooth and accurate docking. The training is conducted in a Gym-compatible Gazebo simulation, acting as a digital twin of the real-world system, and incorporates realistic variations in lighting, obstacle placement, and marker visibility. A designed reward function encourages alignment accuracy, progress, and safety. The final policy is deployed on a real robot via a sim-to-real transfer pipeline, supported by a ROS-based transfer node. Experimental results demonstrate that the proposed method achieves robust and precise docking behavior under diverse real-world conditions, validating the effectiveness of PPO-based learning and sensor fusion for practical autonomous docking applications.