Confidence fusion based emotion recognition of multiple persons for human-robot interaction

Abstract

Emotional interaction with human beings is desirable for robots. In this study, we propose an integrated system which has ability to track multiple people at the same time, to recognize their facial expressions, and to identify social atmosphere. Consequently, robots can easily recognize facial expression, emotion variations of different people, and can respond properly. In our facial expression recognition scheme, we fuse Feature Vectors based Approach (FVA) and Differential-Active Appearance Model Features based Approach (DAFA) to obtain not only apposite positions of feature points, but also more information about texture and appearance. With the obtained useful information, FVA can classify the emotions according to comparison with the distances and ratios of feature points, and DAFA can distinguish emotions from classical machine learning on a low dimensional manifold space. Furthermore, emotion recognition of multiple people at the same time is extended. Based on the proposed algorithm, multiple person emotion analysis and social atmosphere identification can be achieved, which makes the relationship between people and robots much closer. Experimental results demonstrate that the proposed algorithms can recognize facial expressions accurately and robustly. The ambient atmosphere identification system is implemented with a young Einstein robot head in our laboratory.