Continual Learning: Tackling Catastrophic Forgetting in Deep Neural\n Networks with Replay Processes

Abstract

Humans learn all their life long. They accumulate knowledge from a sequence\nof learning experiences and remember the essential concepts without forgetting\nwhat they have learned previously. Artificial neural networks struggle to learn\nsimilarly. They often rely on data rigorously preprocessed to learn solutions\nto specific problems such as classification or regression. In particular, they\nforget their past learning experiences if trained on new ones. Therefore,\nartificial neural networks are often inept to deal with real-life settings such\nas an autonomous-robot that has to learn on-line to adapt to new situations and\novercome new problems without forgetting its past learning-experiences.\nContinual learning (CL) is a branch of machine learning addressing this type of\nproblem. Continual algorithms are designed to accumulate and improve knowledge\nin a curriculum of learning-experiences without forgetting. In this thesis, we\npropose to explore continual algorithms with replay processes. Replay processes\ngather together rehearsal methods and generative replay methods. Generative\nReplay consists of regenerating past learning experiences with a generative\nmodel to remember them. Rehearsal consists of saving a core-set of samples from\npast learning experiences to rehearse them later. The replay processes make\npossible a compromise between optimizing the current learning objective and the\npast ones enabling learning without forgetting in sequences of tasks settings.\nWe show that they are very promising methods for continual learning. Notably,\nthey enable the re-evaluation of past data with new knowledge and the\nconfrontation of data from different learning-experiences. We demonstrate their\nability to learn continually through unsupervised learning, supervised learning\nand reinforcement learning tasks.\n