Inferring Place-Object Relationships by Integrating Probabilistic Logic and Multimodal Spatial Concepts

Abstract

We propose a novel method that integrates probabilistic logic and multimodal spatial concepts to enable a robot to acquire the relationships between places and objects in a new environment with a few learning times. Using predicate logic with probability values (i.e., probabilistic logic) to represent commonsense knowledge of place-object relationships, we combine logical inference using probabilistic logic with the cross-modal inference that can calculate the conditional probabilities of other modalities given one modality. This allows the robot to infer the place of the object to find even when it does not know the likely place of the object in the home environment. We conducted experiments in which a robot searched for daily objects, including objects with undefined places, in a simulated home environment using four approaches: 1) multimodal spatial concepts only, 2) commonsense knowledge only, 3) commonsense knowledge and multimodal spatial concepts, and 4) probabilistic logic and multimodal spatial concepts (proposed). We confirmed the effectiveness of the proposed method by comparing the number of place visits it took for the robot to find all the objects. We also observed that our proposed approach reduces the on-site learning cost by a factor of 1.6 over the three baseline methods when the robot performs the task of finding objects with undefined places in a new home environment.