Autonomous Robot Control Using Facial Expressions

Abstract

In this paper, a novel real time robot control system using human facial expressions is presented. The proposed system mainly consists of three modules: face detection, facial expression recognition and robot control. The first module aims to find the user's face from the image captured from a live video through a series of steps including skin color classification, edge detection and mathematical morphology, while the second module analyses the detected face to recognize the facial expressions like happiness, sadness, surprise, anger and neutral using Principal Component Analysis (PCA) and Euclidian distance calculation. Finally, the detected facial expressions are used as controlling commands for the robot. Experiments were conducted for respective modules. 150 images under different lightening conditions and complex background were employed. In the experiment on detecting the facial region, the rate of detection was 99.3% and on recognizing the facial expression, the recognition rate was 97.3%. The robot was then controlled autonomously with an accuracy of 100%. Combining the above three rates, the overall rate of success of our system was 97.3%.