Tuning neural networks with stochastic optimization

Abstract

This paper describes a method for automated tuning of hyper-parameters of supervised learning systems. It emerges from stochastic aproximation, uses memory-based learning principles, follows certain ideas of experimental design and employs a particular approach to resampling called stochastic validation. Potential usefulness of the proposed approach is illustrated with the fuzzy-ARTMAP neural network application to learning a qualitative positioning of an indoor mobile robot equipped with ultrasonic range sensors. Automatically selected setpoints allow the system to reach a similar or better performance in comparison to that achieved by human experts in all studied cases. The presented method may serve as a design tool in practical applications of supervised learning algorithms.