Proceedings of SLPAT 2015: 6th Workshop on Speech and Language Processing for Assistive Technologies

Abstract

Over the span of human existence our brains have evolved into sophisticated multimodal social signal processing machines. We are experts at detecting and decoding information from a variety of sources and interpreting this information in a social context. The human face is one of the most important social channels that plays a key role in the human communication chain. Today, with computer animated characters becoming ubiquitous in games and media, and social robots starting to bring human-like social interaction capabilities into the physical world, it is possible to build applications that leverage the unique human capability for social communication new ways to assist our lives and support us in a variety of domains. This talk will cover a series of experiments attempting to quantise the effect of several traits of computer generated characters/robots such as visual speech movements, non-verbal signals, physical embodiment and manual signing. It is shown that a number of human functions ranging from low-level speech perception to learning can benefit from the presence of such characters when compared to unimodal (e.g. audio-only) settings. Two examples are given of applications where these effects are exploited in order to provide support for people with special needs -a virtual lipreading support application for hard of hearing and a signing avatar game for children with communicative disorders.