Analysis of category estimation for cloud based chat robot

Abstract

We have been developing and researching chat robot for elderly people. This paper proposed category estimation for voice based chat robot system. Proposed method estimates a category of the user's utterance sentence. Category or topic estimation have been studied by many researchers, however voice base chat system needs a quick response. Therefore our system is simple method just using keyword match and previous dialogue history. This paper analyzed the validity of category estimation compares with manual annotation. Category estimation is a difficult task even if human, then each sentence were categorized by three annotator to 18 categories. In this research, we collected 13727 dialogue sentences from an experiment with ten elderly participants. As the result, 12783 sentences can be categorized (2 or 3 annotator categorized for the same category). We compare the categorized sentences and robot's estimated category. The robot categorized as same as human around 50%. This coincident ratio is depending on the category. “Society” category is difficult to categorize, but “go-out” and “music” can be categorized as around 60% ratio. In the future works, we have to analyze the similarity around 18 categories and how to control the number of categories.