Enabling Deep Learning Inferencing in Edge Devices

Abstract

Today, Deep Learning is consistently gaining widespread popularity, for its accuracy and flexibility. Deep Learning has vast applications, enabled by ever growing compute power in compute farms, cloud and so on. And there are furthermore exciting applications of Deep Learning inferencing, when brought down from cloud to tiny edge devices. The innumerable number of exciting application areas include consumer, energy and utilities, oil and gas, manufacturing, industrial, surveillance, drones, automotive, robotics, medical, wearables and so on. There are many challenges in bringing Deep Learning inferencing to edge devices though, as there is shortage of compute and battery power at the edge, typically. A Deep Learning inferencing solution for edge devices has been developed, which is presented in this paper. The Deep Learning inferencing solution developed has various components including offline training system; real-time Deep Learning inferencing hardware engine and Deep Learning software stack for the edge applications.