Reinforcement Learning of Whole-Body Control Strategies to Balance a Dynamically Stable Mobile Manipulator

Abstract

Mobile manipulators consist of a ground robot base and a mounted robotic arm, with the two components typically controlled as separate subsystems. This is enabled by the fact that most mobile bases with three or four-wheeled designs are inherently stable, though lacking in maneuverability. In contrast, dynamically stable mobile bases offer greater agility and safety in crowded human interaction scenarios, though requiring active balancing. In this work, we consider the balancing problem for a Two-Wheeled Inverted Pendulum Mobile Manipulator (TWIP-MM), designed for retail shelf inspection. Using deep reinforcement learning methods (PPO and SAC), we can generate whole-body control strategies that leverage the motion of the robotic arm for in-place stabilization of the base, through a completely model-free approach. In contrast, tuning a standard PID controller requires a model of the robot, and is considered here as a baseline. Compared to PID control in simulation, the RL-based controllers are found to be more robust against changes in initial conditions, variations in inertial parameters, and disturbances applied to the robot.