Symbol Interpretation in Neural Networks: an investigation on representations in communication

Abstract

Symbol Interpretation in Neural Networks: an investigation on representations in communication Emerson Oliveira (emersonso@dcc.ufba.br) Angelo Loula (angelocl@ecomp.uefs.br) Cognitive and Intelligent Systems Lab (LASIC) State University of Feira de Santana (UEFS) Feira de Santana, BA, Brazil Abstract Computer simulations have been used to study various aspects about the emergence of communication. But there have been only a few works on the underlying representation processes occurring during the interpretation by an agent of a representation produced by another agent. Here we present a study on representation processes in the emergence of communication occurring in a frequently used cognitive architecture in such experiments, artificial neural networks. We investigate the neural network’s activations during the emergence of communication in search for representational and referential processes. Results show that it is possible to evaluate such processes along the evolution of communication and analyze interpretation accordingly. Keywords: Representation; Communication; Neural Networks; Artificial Intelligence; Computer Simulation. Introduction Computer experiments involving the simulation of interactions between agents have been used to study various aspects of communication and language (for a review of works, see Nolfi and Mirolli, 2010, Christiansen and Kirby, 2003, Wagner et al. 2003). In these experiments, communication and linguistic processes are simulated in a social context, involving multiple agents. The process in focus is not pre-defined, but it rather emerges during and by means of agents’ interactions. As the main form of interaction between agents, in most of these synthetic experiments, communication has, particularly, been a significant research subject. As communication involves the production and interpretation of representations, to understand the underlying representation processes is an important issue. Even so, in computational studies on the emergence of communication, little or nothing is discussed on the representational processes taking place, therefore it remains still a rather open research trend. To study representation processes, it is necessary to examine the interpretation process occurring in the artificial agent and thus to inspect its cognitive dynamics. A frequently used cognitive architecture to control agents during the emergence of communication is neural networks. Here we propose to investigate the activation patterns of neural networks to evaluate representational processes during the emergence of communication. As a theoretical framework to define representation, its model, constituents and varieties, we apply theoretical principles from C.S.Peirce’s pragmatic theory of signs. We reproduce the experiment on the emergence of communication as proposed by Mirolli and Parisi (2008), in which the agents are controlled by a feed-forward neural network, receiving visual and auditory inputs and producing motor actions and auditory outputs. The main objective is to compare the middle layer’s activations from visual input and from auditory input and verify if an auditory input can act as a representation of an object perceived by a visual input, and determine the type of representation occurring. In the next section, we review related work on simulation of the emergence of communication using neural networks. Next, we briefly describe the theoretical principles from Peirce theory of signs. We then describe our computational experiment to study representation and interpretation processes in communication events. Finally, we outline our results and conclusions and point out perspectives on the study of representations in the emergence of sign processes. Related Work There have been several works on computational experiments related to the emergence of communication in a community of artificial agents (Nolfi and Mirolli, 2010, Christiansen and Kirby, 2003, Wagner et al. 2003). However, discussions on the u