Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights

Abstract

Over the past decade, remarkable progress has been made in adopting deep neural networks to enhance the performance of conventional reinforcement learning. A notable milestone was the development of Deep Q-Networks (DQN), which achieved human-level performance across a range of Atari games, demonstrating the potential of deep learning to stabilise and scale reinforcement learning. Subsequently, extensions to continuous control algorithms paved the way for a new paradigm in control, one that has attracted broader attention than any classical control approach in recent literature. These developments also demonstrated strong potential for advancing data-driven, model-free algorithms for control and for achieving higher levels of autonomy. However, the application of these methods has remained largely confined to simulated and gaming environments, with ongoing efforts to extend them to real-world applications. Before such deployment can be realised, a solid and quantitative understanding of their performance on applied control problems is necessary. This paper conducts a comparative analysis of these approaches on four diverse benchmark problems with implementation results. This analysis offers a scrutinising and systematic evaluation to shed light on the real-world capabilities and limitations of deep reinforcement learning methods in applied control settings.