A comparison study of three single-solution based metaheuristic optimisation for stacked auto encoder

Abstract

Deep learning has been effectively used in a variety of application such as audio processing, phonetic recognition, robotic, information retrieval and even analysis of molecules. However to train deep learning is interesting yet challenging. A layer wise pre-training, drop-connect, Hessian-free optimization, and Krylov suspense descent are amongst the successful technique or methods proposed in training it. Recently, some of metaheuristic algorithms have been used to optimize Deep learning, especially Convolutional neural network using Genetic algorithm, Particle swarm optimization, Harmony search, and Simulated annealing. In this paper, three type of Single-solution metaheuristic have been proposed, i.e. Simulated annealing, Macrocanonical annealing or Threshold accepting method to optimize Stacked autoencoder, one of the famous Deep learning. The result of experiment conducted to MNIST dataset show that the proposed method can improve accuracy at the rank of 0.07% up to 12.13% for the first time epoch, although there is an increase in computation time.