Self-organization and nonparametric regression

Abstract

We describe a framework for self-organization which relates the formation of topologic maps to minimization of a free energy function. In the zero noise limit the resulting on-line learning rule is similar to the Kohonen learning rule. We derive a fast EM-algorithm for finite training sets. Choosing different noise parameters for input and output variables we obtain an algorithm for knot placement in nonparametric regression. This algorithm naturally fits into projection pursuit regression when we treat the noise parameter as a projection vector. 1 Self-organizing maps 1.1 Introduction Self-organizing maps have been used for many applications, including signal compression, combinatorial optimization, robot control, regression analysis and so on. Most of these methods rely on the well-known Kohonen learning rule [1]. In this paper we will describe a sligthly different version of the Kohonen learning rule, which has the advantage that it can be derived from a (free) energy function (se...