Practical Mixed Palletizing Manipulator System: Incorporating Practical Reinforcement Learning and Configuration-Space Motion Planning

Abstract

Palletizing, also known as the 3D bin packing problem, is important for optimizing space utilization and automating packing processes, especially in the logistics industry. In practice, handling mixed palletizing scenarios, where a variety of boxes of different sizes are received in real time, is considerably challenging. Existing methods for solving the mixed palletizing problem often overlook practical constraints encountered in real-world applications, such as those pertaining to stability and robustness. In this paper, we propose a practical mixed palletizing manipulator system designed for structured real-world warehouse environments. Our manipulator system has two main components: a practical mixed palletizing model based on reinforcement learning (PMP-RL), which can facilitate stable and efficient box placing, and a configuration-space motion planning network (CMPNet), which can help achieve robust and efficient collision-free robot movement. The PMP-RL model is designed to maximize the pallet volume utilization while incorporating practical reward functions that enhance stability. CMPNet is used to directly predict motion trajectories in a 3D configuration space, and it facilitates real-time motion generation by effectively imitating expert-level paths. Overall, the manipulator system, comprising an automated conveyor belt, a camera-based recognition system, the PMP-RL model, and CMPNet, provide a robust and practical framework for mixed palletizing. Experiments conducted via simulations and in real-world environments have shown that the manipulator system can handle complex palletizing tasks with high efficiency and high stability.