Interactive training of speech articulation for hearing impaired using a talking robot

Abstract

This paper introduces a speech training system for auditory impaired people employing a talking robot. The talking robot consists of mechanically-designed vocal organs such as a vocal tract, a nasal cavity, artificial vocal cords, an air pump and a sound analyzer with a microphone system, and the mechanical parts are controlled by 10 servomotors in total for generating human-like voices. The robot autonomously learns the relation between motor control parameters and the generated vocal sounds by an auditory feedback control, in which a Selforganizing Neural Network (SONN) is employed for the adaptive learning. By employing the robot and its properties, we have constructed an interactive training system. The training is divided into two approaches; one is to use the talking robot for showing the shape and the motion of the vocal organs, and the other is to use a topological map for presenting the difference of phonetic features of a trainee’s voices. While referring to the vocal tract motions and the phonetic characteristics, a trainee is able to interactively practice vocalization for acquiring clear speech with an appropriate speech articulation. To assess the validity of the training system, a practical experiment was conducted in a school for the deaf children. 19 subjects took part in the interactive training with the robotic system, and significant results were obtained. The talking robot is expected to intensively teach an auditory impaired the vocalization skill by directing the difference between clear speech and the speech with low clarity.