Novel First Person View for Human 3D Pose Estimation in Robotic Applications Using Fisheye Cameras

Abstract

A novel 3D pose estimation system is presented, using a single fisheye camera. In order to support close human-robot interaction, this camera is directly mounted on the robot and provides a robot-specific field-of-view and viewpoint for the pose recognition system. In this mobile camera setup, 3D pose estimation is realized with a processing pipeline based on YOLOv7, MeTRAbs. Distortion of images captured by the fisheye camera is corrected with Omnicalib. Two such processing pipelines have been implemented, where the first pipeline is used for publishing 3D poses, and the second with an additional component for classifyng gestures in order to allow a human to convey move commands to a robot. With these two pipelines, we provide a novel way to allow the use of a single camera and enabling the human to interact with a robot.