Human visual augmentation using wearable glasses with multiple cameras and information fusion of human eye tracking and scene understanding

摘要

A smart wearable robot glasses system is proposed to assist human visual augmentation in daily life, providing a refined visual recognition result to users from multiple input images of the proposed system. It consists of a glasses-type wearable device with a front-view camera, eye-view camera, mounted display, earphone, and computing unit for signal processing. The scene-understanding process on the input image from the front-view camera can be computationally accelerated with the support of the eye-view camera that monitors the eye position of the user. For efficient information processing, the eye view camera catches the user's visual intention and attention in a given situation. It is correlated to the eye viewing direction estimated from the eye-position monitoring results of eye-view camera. The proposed device can be used to augment the human visual capability in various daily life applications.