Timed Petri Nets for Multimodal Interaction Modeling

Abstract

Humans naturally use the multiple modalities of speech, ges-ture, and gaze when they communicate. They also engage in turn-taking to manage the seizing and yielding of the speak-ing floor, a process that controls the execution of turns com-prising discrete speech, gesture, and gaze events. We are interested in modeling such interaction processes for a so-cial robot to use that can be transferred between different domains. Timed Petri nets (TPNs) are currently uncom-mon in human-robot interaction (HRI) but offer an attrac-tive representation for modeling concurrency and synchro-nization when controlling behavior for multiple modalities. Their representation also permits the intuitive and modular combination of rule-based behavior expression with statisti-cal timing models. We describe their utility in application to multimodal interaction and compare them to finite state machines (FSMs) and Markov models, two more commonly used methods for control. 1.