What am i doing? Robotic self-action recognition

Abstract

When a robot executes a task, it will be helpful if the robot understands its behavior and can tell people what it is doing. Inspired by previous works of detecting activities in first-person views, we present a novel idea of robotic self-action recognition. To achieve our goal, we not only use the robot's view as visual input, but also adopt the joint information as inner state input, so that we do not have to track the hands positions in the images like other first-person view action detecting methods did, and we are not worried about the hands moving out of view. We introduce a new dataset which consists of some daily tasks, then we evaluate the dataset with our deep learning model, which is based on Long-term Recurrent Convolutional Network(LRCN). Finally, we apply the model to an online system with our robot platform.