Training a Lightweight CNN Model for Fine-Grained Sewer Pipe Cracks Classification Based on Knowledge Distillation

Abstract

Current sewage pipeline inspection researches focus on locating defects, with only a few studies addressing the crack fine-grained classification task. However, in engineering applications, such as Pipeline Assessment Certification Program (PACP), subcategory coding of cacks is required. Therefore, this paper studies the fine-grained classification of pipeline cracks. Algorithms based on deep learning have been widely used in the field of sewage pipeline inspection, but these models have a large number of parameters and are computationally time-consuming. Edge or embedded devices with weak GPUs cannot accept the time and storage costs of these large models. To solve the above problems, this paper introduces knowledge distillation (KD) which is one of the mainstream model compression methods. We select the large-scale network Resnet50 as the teacher model and design a lightweight network Combined MobileNetV2 & Inception (CMI) for crack detection as the student model. High-temperature distillation allows the student model to obtain dark knowledge from the pre-trained, well-performing teacher model, which is combined with ground truth (GT) for training. Experiments show that KD is effective. By adjusting the KD hyperparameters, the classification accuracy of the student model is improved by 1.41%, and the F1-score is improved by 1.12%. This demonstrates the robustness of knowledge distillation to improve the performance of small models, laying the foundation for future deployment on multi-sensor automatic detection robots.