Hardware Realization of Reinforcement Learning Algorithms for Edge Devices

摘要

With the rapid emergence of the Internet of Things (IoT) connected to existing or emerging networks and with the increasing need and reliance to execute artificial intelligence (AI)-based applications on IoT devices at the edge of the network to reduce communication overheads, there is an increasing trend towards designing specialized hardware using novel architectures, to carry out the extremely large data computations needed by underlying AI and machine learning (ML) algorithms to realize these applications. Additional constraints on such hardware accelerators residing on the IoT devices at the edge is the availability of limited memory resources and meeting some strict real-time constraints imposed by the applications. One such application that arises in the context of autonomous robots is that of the simple reinforcement learning (SRL) algorithm. In this chapter, a novel hardware architecture is proposed for edge devices based on the SRL algorithm. We also compare its hardware implementation with that of the Q-learning algorithm—another widely popular reinforment learning (RL) algorithm. An in-depth discussion related to the applications of RL algorithms for both single-agent and multi-agent behavior is also provided. To set the context of our contribution, we additionally discuss several hardware solutions reported in the literature for accelerating ML algorithms at the edge. Application of RL algorithms implementated as a hardware accelerator on an embedded field-programmable gate arrays (FPGA) board to the functioning of an autonomous robot that detects plant diseases and concurrently treats such diseased plants in a greenhouse environment is further presented as part of future work.