Image Inpainting and Classification Agent Training Based on Reinforcement Learning and Generative Models with Attention Mechanism

Abstract

What distinguishes the field of artificial intelligence (AI) from others is to develop fully independent agents that learn optimal behavior, change, and evolve solely through the communication of trial and error with the surrounding environment. Reinforcement learning (RL) can be seen in multiple aspects of Machine Learning (ML), provided the environment, reward, actions, the state will be defined. Agent training in previous years is seen to only relate to robotics, games, and self-driving cars. While trying to divert the focus of researchers from the view of self-driving cars, games, robots, etc. Here, we investigated using reinforcement learning in the aspect of task completion. We deployed our architecture in an inpainting task where the agent generates the distorted or missing image content into an eminent fidelity completed the image by using reinforcement learning to influence the generative model utilized. The Generative Adversary Network (GAN) problem of not being steady and challenging to train was overwhelmed by utilizing latent space representation. The dimension is reduced compared to the distorted or corrupted image in training the GAN. Then reinforcement learning was deployed to pick the correct GAN input to get the image’s latent space representation that is most suitable for the current input of the missing or distorted image region. In this paper, we also learned that the trained agent enhances the accuracy in a classification task of images with missing data. We successfully examined the classification enhancement on images missing 30%, 50%, and 70%.