A novel concept of Human-Robot competition for evaluating a robot's reasoning capabilities in HRI

Abstract

For intelligent Human-Robot Interaction (HRI), a robot should be equipped with some core reasoning capabilities such as perspective taking, effort analysis, and affordance analysis. This paper starts to explore how a robot equipped with such reasoning abilities could be evaluated. To this end, inspired by the Turing test, we design a game involving a human-robot competition scenario. Interestingly, the participants' subjective feedback, which tended to compare the robot's abilities with their own, points toward potential criteria for developing benchmark scenarios and evaluation matrices.