Approximate Dynamic Programming in the Sensor-Based Navigation of the Wheeled Mobile Robot

Abstract

The article presents a new approach to the sensor-based navigation of wheeled mobile robot Pioneer 2-DX in the unknown 2-D environment with static obstacles. The navigation task has been developed using a discrete hierarchical control system with a path planning layer and a tracking control layer designed using approximate dynamic programming algorithms. The navigator realises a behavioural control approach to the path planning process using the adaptive coordination of two simple behaviours: “goal-seeking” and “obstacle avoiding”. The main part of the navigator is the Action-Dependant Heuristic Dynamic Programming structure realised in a form of the actor and critic neural networks. To avoid the time consuming trial and error learning, additional proportional controllers generating signals that prompt the direction of the sub-optimal control law seeking process at the beginning of the NNs adaptation process are arranged in the navigator. The tracking control layer is composed of a PD controller, the Dual Heuristic Dynamic Programming algorithm and a supervisory term. It generates control signal for DC motors of the robot. The performance of the proposed discrete control system was verified by a series of experiments conducted using wheeled mobile robot Pioneer 2-DX equipped with one laser and eight ultrasonic range finders that provide object detection.