Modular expansion of the hidden layer in Single Layer Feedforward neural Networks

Abstract

We present a neural network architecture and a training algorithm designed to enable very rapid training, and that requires low computational processing power, memory and time. The algorithm is based on a modular architecture, which expands the output weights layer constructively, so that the final network can be visualised as a Single Layer Feedforward Network (SLFN) with a large hidden-layer. The method does not use backpropagation, and consequently offers very fast training and very few trainable parameters in each module. It is therefore potentially a useful method for applications which require frequent retraining, or which rely on reduced hardware capability, such as mobile robots or Internet of Things (IoT). We demonstrate the efficacy of the method in two benchmark image classification datasets, MNIST and CIFAR-10. The network produces very favourable results for a SLFN on these benchmarks, with an average of 99.07% correct classification rate on MNIST and nearly 82% on CIFAR-10 when applied to convolutional features. Code for the method has been made available online.