Learning fine-grained control for mapless navigation

摘要

We consider the problem of learning a control policy that allows an autonomous mobile robot to navigate safely to target positions in an environment, without access to an obstacle map. The policy can operate in environments of arbitrary size and may be deployed in resource-constrained settings where storing and maintaining an accurate map are infeasible or prohibitively expensive. The learned policy, trained end-to-end using deep reinforcement learning, outputs continuous control commands to the actuators of a simulated two-wheel differential drive robot. A new reward function is proposed to encourage the robotic agent to learn local recovery and exploration behaviours, which greatly improves the ability of the agent to solve challenging navigation tasks in new environments. The performance of the learned policy is compared to an agent equipped with full knowledge of the obstacle map. Even though the learned policy may solve many navigation tasks, we conclude that some tasks still require the use of a global path planner. However, coupled with a high-level path planner to provide intermediate target beacons to the goal, the learned policy may be employed as an effective low-level component with reactive collision avoidance behaviours and local navigation skills in static or dynamic environments.