Adaptive-Optimized Semantic Point Cloud Mapping and Path Planning with YOLO11-seg

Abstract

Traditional SLAM algorithms primarily emphasize geometric reconstruction, which constrains a robot’s ability to interpret and interact with its environment at a semantic level. To address this limitation, we present an integrated framework for semantic map construction and navigation validation. The framework begins by capturing RGB and depth images, followed by instance segmentation using the YOLO11-seg model to obtain object masks and detection results. ORB-SLAM3 is employed to estimate the camera pose. By combining pose and depth information, RGB-D pixels within segmented regions are projected into the world coordinate system to form a semantic point cloud. To enhance processing efficiency and map quality, the framework incorporates a semantically guided adaptive voxel downsampling strategy that dynamically adjusts voxel size based on local point cloud density. In addition, an adaptive clustering-based deduplication module leverages object size, semantic labels, and inter-frame overlap to identify and merge redundant instances. Finally, a visualization interface is developed, integrating multiple path planning and obstacle avoidance algorithms on top of the generated semantic map. This framework demonstrates the practicality of semantic mapping for navigation tasks and provides a foundation for deployment in real-world robotic applications.