Situated reference in a hybrid human-robot interaction system

Abstract

We present the situated reference generation module of a hybrid human-robot interaction system that collaborates with a human user in assembling target objects from a wooden toy construction set. The system contains a sub-symbolic goal inference system which is able to detect the goals and errors of humans by analysing their verbal and non-verbal behaviour. The dialogue manager and reference generation components then use situated references to explain the errors to the human users and provide solution strategies. We describe a user study comparing the results from subjects who heard constant references to those who heard references generated by an adaptive process. There was no difference in the objective results across the two groups, but the subjects in the adaptive condition gave higher subjective ratings to the robot’s abilities as a conversational partner. An analysis of the objective and subjective results found that the main predictors of subjective user satisfaction were the user’s performance at the assembly task and the number of times they had to ask for instructions to be repeated. 1