Towards a Task-agnostic Distillation Methodology for Creating Edge Foundation Models

Abstract

In recent years, AI has undergone significant changes. Firstly, there is a growing recognition of the need to deploy inference models based on Deep Neural Networks (DNNs) on edge devices. Secondly, there is an increasing demand for low-energy inferencing and continuous online learning, particularly in dynamic environments. Thirdly, foundation models, trained on broad datasets for diverse applications, are gaining prominence. In closed-loop systems like robotics, there is a need to use foundation models at the edge due to practical constraints in training new models for every environment or data type. This article addresses issues in current edge computing scenarios and proposes Edge Foundation models as a solution. We introduce a task-agnostic distillation method for generating compact yet generalized models and present preliminary proof-of-concept results, demonstrating the potential of Edge Foundation models to accelerate Edge AI adoption.