Adaptive Policy-Switching Reinforcement Learning for Resource-Constrained Autonomous Systems

Abstract

Incorporating Reinforcement Learning (RL) in autonomous systems has shown significant progress in the areas of adaptability in decision making. Nevertheless, standard reinforcement learning (RL) models often ignore the dynamic and resource constrained environment that autonomous systems further more operate in like computation power, energy, and memory and communication bandwidth. We present a new framework for Resource-Aware reinforcement learning (RARL), which balances the need to maximize learning of a policy, with the limitation of resource availability. The resulting model features a dual-objective reward that both maximizes task performance while minimizing resource usage metrics, permitting the system to make efficient and context-sensitive decisions in real time. We demonstrate the efficacy of our framework across various autonomous domains such as mobile robotics through UAVs, and edge-based AI systems. Experimental results show that not only does RARL achieve high task performance, but it also greatly decreases energy consumption and latency by dynamically adapting its policy execution in relation to resources available. Furthermore, the adoption of resource monitoring agents improves the model’s robustness and quick responsiveness when facing stochastic environmental conditions. This study applies new methodologies of managed scalable and sustainable RL for autonomous systems while linking intelligent solutions to existing constraints in real life context regarding operational issues. This has important implications for the potential use of RL in low-power embedded environments as well as mission critical applications where autonomy is required, but efficiency must also be guaranteed. In future work, we will investigate federated extensions as well as meta-learning techniques to further improve adaptability in heterogeneous and distributed environments.