A Survey on Diffusion Policy for Robotic Manipulation: Taxonomy, Analysis, and Future Directions

摘要

Robotic manipulation tasks face significant challenges in learning effective control policies in complex, high-dimensional action spaces and uncertain environments. Recently, diffusion models-originally developed for image generation-have emerged as a promising approach for robotic policy learning due to their exceptional ability to model complex multimodal distributions and generate diverse, high quality outputs. Since 2022, numerous research studies have explored diffusion policies for robotic manipulation, demonstrating superior performance compared to traditional methods in various tasks. However, there is a lack of comprehensive survey papers that thoroughly analyze this rapidly evolving field. To bridge this gap, we provide a comprehensive review of diffusion policies for robotic manipulation in a taxonomy of three key dimensions: 1) robot data representations, including 2D representations (robot trajectories and human videos), 3D representations (single-view and multi-view point clouds), and heterogeneous data (input data and robot proprioception heterogeneity); 2) model architectures, examining various combinations such as Large Language Models with diffusion head (general data pre-training and robot data pre-training), small-size CNN or transformer models with diffusion (for direct action generation, subgoal generation, robot data generation, and MoE architectures), and VAE/VQ-VAE integrated with diffusion; 3) diffusion strategies, covering approaches that incorporate reinforcement learning, equivariance, accelerated sampling or denoising strategies, employing classifier (free) guidance, and integration with self-supervised learning. Furthermore, we provide a comprehensive analysis of their technical characteristics, advantages, and limitations. Based on our comprehensive organization and analysis, we offer valuable suggestions for selecting appropriate diffusion policy methods for different robotic manipulation scenarios and outline promising future research directions. To facilitate further research in this field, we maintain a publicly accessible repository at https://github.com/HITSZ-Robotics/DiffusionPolicy-Robotics, which serves as an exhaustive resource of diffusion policies for robotic manipulation papers and corresponding open-source codes, and we have meticulously collected and organized the configurations of simulation platforms and real-world robots. We will keep updating this repository to include the latest advancements in the field.