Kinect and RGBD Images: Challenges and Applications

Abstract

Kinect is a device introduced in November 2010 as an accessory of Xbox 360. The acquired data has different and complementary natures, combining geometry with visual attributes. For this reason, Kinect is a flexible tool that can be used in applications from several areas such as: Computer Graphics, Image Processing, Computer Vision and Human-Machine Interaction. In this way, the Kinect is a widely used device in industry (games, robotics, theater performers, natural interfaces, etc.) and in research. We will initially present some concepts about the device: the architecture and the sensor. We then will discuss about the data acquisition process: capturing, representation and filtering. Capturing process consists of obtaining a colored image (RGB) and performing a depth measurement (D), with structured light technique. This data is represented by a structure called RGBD Image. We will also talk about the main tools available for developing applications on various platforms. Furthermore, we will discuss some recent projects based on RGBD Images. In particular, those related to Object Recognition, 3D Reconstruction, Augmented Reality, Image Processing, Robotic, and Interaction. In this survey, we will show some research developed by the academic community and some projects developed for the industry. We intend to show the basic principles to begin developing applications using Kinect, and present some projects developed at the VISGRAF Lab. And finally, we intend to discuss the new possibilities, challenges and trends raised by Kinect.