EXTRACTING ACTIONS FROM INSTRUCTION MANUAL AND TESTING THEIR EXECUTION IN A ROBOTIC SIMULATION

Abstract

Service robots are expected to execute household manipulation tasks on daily basis. These tasks are often related to the operation of home appliances and can be decomposed into specific manipulation actions such as pick up a cup, place a cup, press a button, turn a knob, open or close a cover, etc. This research proposes to use task instructions collected from manuals of household appliances to extract a series of actions that a robot needs to perform in order to accomplish the task. We describe the extraction method of action and object from these task instructions including syntax parsing of sentences and searching the pairs of action and object in each parse tree based on part-of-speech. The extracted actions then will be executed by a robot. We apply a simulation environment with ROS, Gazebo simulator and a virtual robot PR2 for testing their execution. In experiments, we implement the extraction method and evaluate the extracted results of actions and object. In addition, we test the execution of actions pick and place in the proposed simulation environment.