Robot learning based on Partial Observable Markov Decision Process in unstructured environment

Abstract

Robot teaching is necessary for the current industrial robot applications. Because work stations have to be stopped to perform teaching processes, the manufacturing efficiency is decreased. In this paper we propose to utilize an uncalibrated vision system mounted on a mobile robot (“Adult” robot) with learning capability to supervise a group of fixed robots (“Child” robots) to accomplish a robot teaching task automatically without stopping work stations. To increase the system flexibility, hand-eye calibration and calibration between the robots are eliminated. A Partial Observable Markov Decision Process(POMDP) is formulated and solved using the Successive Approximation of the Reachable Space under Optimal Policies (SARSOP) algorithm to enable the teaching process using image features with uncertainties. The proposed algorithm was tested using the “adult” robot to teach a “child” robot to perform a high accuracy peg-in-hole assembly process. The experimental results verify the effectiveness of the proposed approach. The proposed method can also be used in other areas to enable robot teaching.