A sensor fusion approach for PbD

Abstract

Since programming by demonstration (PbD) approaches have reached prime importance in interactive robot programming, sensor technology for tracking user actions and user behavior have become more and more important. However, traditional methods based on single sensor system input are already at their limits, since PbD is an application area where real-time requirement do play an important role. Thus, a sensor fusion approach is proposed which serves input data for a finite state automaton. The input sources are on the one hand a data-glove which classifies different grips and on the other hand a movable camera head which tracks the movements of the data glove as well as estimates object positions. Both sensor sources use time efficient algorithms, since sensory data must be processed in real-time. The efficiency of this approach is proven in a PbD environment were flexible infusion bags are handled and respective actions and object positions are recognized.