A Benchmark Toolkit for Collaborative Human-Robot Interaction

Abstract

Novel human-robot collaboration (HRC) methods need careful validation. This is often achieved with user studies in laboratory environments, which mostly rely on highly individual, complex prototype setups in early design stages. Lately, the lack of replicability imposed by such experiments has vitally been discussed. In this paper, we contribute a benchmark toolkit to compose scalable, synthetic tasks as a unified basis for future efforts towards more replicable research. To this end, we pro-pose modular task boards which cover different domains and HRC scenarios. The design of benchmark tasks with these task boards is supported by a software application which generates structural task models and materials for benchmark problem reproduction by 3D printing. Our experiments show that these tasks can be carried out robustly by robots, hence preventing unintended robot failure and providing a controllable, reliably reproducible setting for user studies.