L2CS-Net : Fine-Grained Gaze Estimation in Unconstrained Environments

Abstract

Human gaze is a crucial cue used in various applications such as human-robot interaction and virtual reality. Recently, convolution neural network (CNN) approaches have made notable progress in predicting gaze direction. The performance of existing appearance-based gaze methods remains unsatisfactory due to the uniqueness of eye appearance, lightning conditions, and the diversity of head pose and gaze directions. In this paper, we propose a novel multi-loss two-branch CNN architecture (L2CS-Net) to explicitly learn the discriminative features for each gaze angle by predicting each gaze angle using a separate fully connected layer and loss function. In addition, we introduce a new multi-loss gaze function that consists of combined classification and regression losses to further enhance the model performance. We perform gaze classification utilizing a softmax layer along with cross-entropy loss. To obtain fine-grained predictions, we calculate the expectation of the gaze-class probabilities followed by a Mean Squared Error (MSE) loss. We evaluated our model with two popular datasets collected with unconstrained settings. Our proposed model achieves state-of-the-art performance on the MPIIGaze and Gaze360 datasets, respectively. We make our code open source at https://github.com/Ahmednull/L2CS-Net.