RoboCleaner: Robotic Tabletop Cleaning via VLM-Powered Multi-Agent Collaboration

Abstract

Robotic tabletop cleaning is applicable in various environments ranging from domestic to industrial settings, yet it still faces challenges in the cluttered scenario where wastes of diverse types exist. Moreover, to deal with different waste, such as liquid spills or fine crumbs, different cleaning tools might be required, which further complicates the task. Inspired by significant advancements in vision language models (VLMs), in this paper, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RoboCleaner</i>, a VLM-powered multi-agent framework for tabletop cleaning that leverages the collaborative intelligence of multiple agents. Specifically, the proposed framework consists of three VLM-powered agents: a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Planning Agent</i> for decision-making, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Execution Agent</i> for precise operation and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Reflection Agent</i> for outcome evaluation and providing feedback for iterative improvement. Through the collaboration of these agents, our framework is capable of handling a wide range of challenging cleaning tasks. Extensive experiments are conducted demonstrating that the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RoboCleaner</i> can achieve high task success rates and operational efficiency within cluttered environments. Also, we have observed the emergent problem-solving capabilities of the proposed framework, which further validates the robustness and adaptability of the framework.