Lightweight Super-Resolution for Chinese Scene Images Incorporating Textual Semantic Priors

Abstract

In the fields of autonomous driving and robotics, text image super-resolution can improve the resolution of the image obtained by the device, and help the system capture text details and long-distance text in the scene to improve the perception and decision-making ability. Significant progress has been made in prior-based text image super-resolution in recent years. However, the fusion of complex languages text prior information, such as Chinese, leads to a sharp increase in network parameters and computational costs, limiting the application of resource-constrained mobile robots in corresponding scenarios. In response to this issue, we propose a lightweight scene text image super-resolution method that incorporates semantic priors (MPT-TISR). Firstly, image convolutional layer and text recognizer are employed to extract low-resolution image features and text sequences, respectively. Then, an efficient Text Semantic Feature Fusion Block (FPCAT) based on Transformers and Principal Component Analysis (PCA) is constructed to associate essential text information with image features. Simultaneously, an improved MobileViTv3-based Sequential Residual Block (SRBMVT3+) is designed to learn high-dimensional deep features and enhance feature representation capabilities. Additionally, a binary gradient loss function is utilized to filter noise and guide the reconstruction process towards focusing on text details. Experimental results on a Chinese scene dataset demonstrate that MPT-TISR outperforms existing methods in terms of reconstructed image quality and significantly improves the recognition accuracy in the downstream text recognition task.