An Intelligent Agent Architecture In Which to Pursue Robot Learning

Abstract

This paper describes a multi-layered, intelligent agent software architecture, developed for mobile and undersea robot applications in the defense sector, and to provide tele-autonomy to space-based manipulator robots. The architecture has a deliberative layer which uses a state-based planner, a middle layer for sequencing partially ordered plans using robot skills, and a lower layer repertoire of continuous robot skills. The system has been shown to provide a higher level of human supervision that preserves safety while allowing for task level direction, reaction to out-ofnorm parameters, and human intervention at all levels of control. For this workshop, we hypothesize that the architecture is a useful framework in which to explore learning techniques. In particular, we outline techniques appropriate to learning within a given layer, techniques for migrating competences from higher to lower layers, and overall system adaptation from its interaction with the environment. Examples are reinforcement learning for tuning individual skills, case-based techniques to improve the re-planning capability of the deliberative layer, and chunking or explanation-based learning to migrate new strategies created by the planner into standard procedures for the sequencing level. Background and Motivation Since the late eighties we have investigated ways to combine deliberation and reactivity in robot control architectures [Sanborn et al 1989, Bonasso 91, & Bonasso et al 92], in order to program robots to carry out tasks robustly in field environments. Field environments are those in which events for which the robot has a response can occur unpredictably, and wherein the locations of objects and other agents is usually not known with certainty until the robot is carrying out the required task. A robot control software architecture, developed at MITRE is an outgrowth of several lines of situated reasoning research in robot intelligence [Firby 89, Gat 91, Connell