Incremental 3D Body Reconstruction Framework for Robotic Telepresence Applications

Abstract

This research proposes an on-line incremental 3D reconstruction framework that can be used on telepresence robots or human robot interaction (HRI). The aim is a low cost solution that enables users to communicate and interact remotely experiencing the benefits of a face-to-face meeting. By exploring computers graphics techniques and spatial audio we intend to induce sensations of being physical in the presence of other people. There is a wide variety of research opportunities including high performance imaging, multi-view video, virtual view synthesis, etc. One fundamental challenge in geometry reconstruction from traditional cameras array is the lack of accuracy in low-texture or repeated pattern region. Our approach explores virtual view synthesis through motion body estimation and hybrid sensors composed by video cameras and a depth camera based on structured-light or time-of-flight. We present a full 3D body reconstruction system that combines visual features and shape-based alignment. Modeling is based on meshes computed from dense depth maps in order lower the data to be processed and create a 3D mesh representation that is independent of view-point. induce sensations of being physical in the presence of other people useful on several domains like elderly loneness minimization problem[20], tele-rehabilitation[18][32], companion robot, education, socialization, 3DTV, entertainment, etc. Phones and internet chat/audio/video conferencing programs (ex: VOIP, NetMeeting, Skype) have been used for socialization, nevertheless they are not able to create the remote person presence feeling. Means of communications that enable eye contact, gestures reconnaissance, body language and facial expressions are required.