Handling dimensionality and nonlinearity in connectionist learning

Abstract

Most connectionist approaches, while promising in solving small-sized problems, do not scale up well with the problem size. However, many real-world learning problems are of high dimensionality and the mappings to be learned are also highly nonlinear. Learning may prove to be intractable if the methods are scaled up naively. This thesis proposes several techniques for handling dimensionality and nonlinearity in supervised learning and in reinforcement learning. The underlying philosophy is based on two principles: divide-and-conquer and locality. We propose a class of supervised learning networks called context-sensitive networks. The basic idea is to decompose a function into a parameterized family of functions, each of which is lower in dimensionality and hence easier to learn than the original one. A context-sensitive network, composed of a context network and a function network, has two semantically different levels of abstraction. With the use of complex hidden units in a context network, sparsely distributed internal representations will emerge and the problem of high nonlinearity can be handled better. Each hidden unity is only sensitive to a localized basis region in the input space. This helps reduce the interference between different patterns represented in a distributed fashion. The a priori knowledge of forming convex basis regions is utilized in grouping simple hidden units into complex ones. Thus the network does not start from scratch but with simple internal representations already existing. We then propose a general framework for associative reinforcement learning, which includes a supervised learning network as one of its building components. With this framework, the techniques for handling dimensionality and nonlinearity is supervised learning can be transferred to reinforcement learning problems. We also propose a game-theoretic network architecture for associative reinforcement learning. Each subproblem is taken care of by one subnetwork called an associative learning automaton. The game-theoretic interactions among the associative learning automata lead to the emergence of the solution for the entire problem. Extensive simulations have been run for several control-type problems to test and illustrate the ideas. Among them, the robot arm control problem and the adaptive load balancing problem are the most extensively studied ones. (Copies available exclusively from Micrographics Department, Doheny Library, USC, Los Angeles, CA 90089-0182.)