Learning Sequential Skills for Robot Manipulation Tasks

Abstract

Most people's imagination about robots has been shaped by Hollywood movies or novels, resulting in the dream of having robots as assistants or household helpers in our homes. However, there is still a large gap between this dream and the actual capabilities of robots. One underlying reason is that every home is unique and largely unstructured, making it impossible to pre-program a robot for all the challenges it might face in such an environment. For instance, floor plans and furniture differ from home to home. Humans and pets walk around, potentially getting in the robot's way and making the environment non-static. Hence, a pre-programmed robot deployed in such an environment will undoubtedly face problems that it cannot solve with its existing knowledge. In order to cope with this issue, researchers started to equip robots with learning capabilities. Ideally, such capabilities allow a robot to adapt skills to new or changing situations or even to learn completely new tasks. Also humans learn new skills over time and are able to adapt them if needed. Therefore, such learning capabilities seem natural to us. If we are not able to master a specific task, we usually would ask another person to demonstrate it or to give instructions on how to perform it. In robotics research, the field of "Learning from Demonstration" tries to mimic this behavior by learning new skills from demonstrations of a task. By applying machine learning techniques, the data perceived from a single or multiple demonstrations are exploited to learn a mapping from perception to the action of a robot. In this thesis, we concentrate on important Learning from Demonstration aspects that have not gotten so much attention in the research community so far. In particular, we focus on learning methods for robot manipulation tasks. These tasks have two important characteristics. First, they can be naturally decomposed into a set of subtasks and, therefore, can be mastered by performing the individual subtasks in the correct sequential order. Second, they involve physical contact between the robot and objects in its environment. One aim of this thesis is developing methods which allow for learning skills for robot manipulation tasks that generalize well to unknown situations. For instance, a learned skill should also be applicable if positions and orientations of objects differ from those seen in a demonstration. In the first part of the thesis, we focus on the "sequential" aspect of manipulation tasks. Many approaches assume that subtasks are executed in a purely sequential manner or that the human always demonstrates the same sequence of subtasks. We propose an approach that does not have this assumption. Based on the demonstrations, a graph is generated which connects the subtasks with each other. Each subtask is associated with a movement primitive, a basic elementary movement necessary to perform the subtask. Depending on the environmental conditions, different sequences of movement primitives are executed, allowing the robot to perform tasks which for instance require an arbitrary number of repetitions (e.g., unscrewing a light bulb). As we concentrate on the sequential aspects of a task in the first part of the thesis, we assume the demonstrations are labeled with the correct movement primitives over time. Additionally, the movement primitives are predefined. In the second part of the thesis, these two assumptions are relaxed. We first present an approach which decomposes the demonstrations into a set of meaningful movement primitives by inferring the underlying sequential structure of the task. The decomposition is based on a probability distribution we call Directional Normal Distribution. By utilizing the distribution, our method infers if a movement should be performed relative to an object in the scene and if a force should be applied in certain directions or not. Forces are especially important when interacting with the environment, for example if the rob