Nonverbal communication with a multimodal agent via facial expression recognition

Abstract

Based on our previous work in the development of a multimodal animated avatar oriented to human-machine interaction and social robotics, a facial expression recognition system is presented in this paper. Both the avatar's architecture and facial recognition system have been oriented to FACS Action Units recognition and generation, so they can be easily integrated. The resulting animated system is able to maintain nonverbal and verbal bidirectional communication, enhancing the interaction with the avatar and making it more natural for the users.