Model Diet: A Simple yet Effective Model Compression for Vision Tasks

Abstract

Computer vision coupled with machine learning algorithms has greatly helped mobile robotic platforms become more intelligent and capable of performing in the real world. Specifically, Convolutional Neural Networks (CNNs) have achieved a high accuracy on a range of visual perception tasks (e.g., object detection, classification, segmentation, and similar others). One of the bottlenecks in CNNs is their high computational requirement. This makes most of them not easily deployable on robotic platforms, since their on-board computational power is limited. Recently, Involution successfully reduced the number of parameters of CNNs by replacing all the 3 × 3 convolution kernels with involution kernels, which use 1 × 1 convolution for the kernel generation. Filter pruning methods have also successively reduced the number of parameters in CNNs. Notably, however, Involution has reshaping layers and the kernel size is unknown when loading the pre-trained model. In this paper, we propose a pruning method named Model Diet that can be applied to Involution and other CNNs. We present experimental results showing that it has better results compared with randomly initialized weights.