Grounding Words in Perception and Action: Insights from Computational Models

Abstract

We use words to communicate about things and kinds of things, their properties, relations, and actions. Mirroring human abilities, researchers are creating robotic and simulated systems that ground language in machine perception and action. A new kind of computational model is emerging from this work that bridges the symbolic realm of language with the physical realm of real-world referents. It explains aspects of contextdependent shifts of word meaning that cannot easily be explained by purely symbolic models. An exciting implication for cognitive modeling is the use of grounded systems to “step into the shoes” of humans by directly processing first-person perspective sensory data, providing a new methodology for testing various hypotheses of situated communication and learning. Words about the Physical World Over the past few decades computational models of language processing have focused on symbolic explanation of linguistic meaning [1-5]. Such models define word meanings in terms of other symbols, producing circular definitions much like those found in a dictionary [6,7]. Humans are less hindered by circular definitions since we ground many words in physical experience in the world. Researchers dissatisfied with purely symbolic models of word meaning have recently sought to build perceptual and robotic systems that ground the meaning of words in terms of their real-world referents. Thus the meaning of “round” is grounded in visual features of exemplars, “push” in motor control structures, “heavy” in haptic features, and so on. These systems provide computational explanations of how words acquire meaning through their connections with perception and action. Although the embodied nature of language has received significant recent attention [810], computational hypotheses formulated in terms of specific representations and