HBOD: A Novel Dataset with Synchronized Hand, Body, and Object Manipulation Data for Human-Robot Interaction

Abstract

Estimating hand and body posture is crucial for enabling human-robot collaboration, preventing occupational diseases, and training humanoid robots. Although advances in wearable motion sensors, such as Inertial Measurement Units (IMUs), have resulted in public datasets in industrial and occupational settings, these datasets rarely include movements with subjects holding and manipulating objects. However, it is crucial to have data on how humans move and interact with different objects so that we can better understand human motion intention and movement strategies in specific scenarios. We thus propose the HBOD dataset (hand-body-object dataset), which encompasses synchronized human pose data from an IMU sensor network, hand posture data from a smart data glove, and object position and attitude information obtained from a motion capture system, while subjects move and interact with a screwdriver, hammer, spanner, electric drill, and a rectangular workpiece. This paper provides an overview of the hardware setup, experimental protocol, data format, and data visualization results. This dataset provides crucial object information absent from existing datasets, thus offering highly valuable manipulation data for occupational diseases research, human-robot interaction, and robot skill acquisition.