A Large Language Model Based on the Retrieval-Augmented Generation and Prompt-Tuning Framework

Abstract

In recent years, large language models (LLMs) have made remarkable progress in the field of natural language processing, and have been widely used in many industries such as finance, healthcare, and education. However, despite the expansion of its application scenarios, LLMs continues to underperform in verticals such as technical expertise and industry-specific knowledge, especially in understanding and responding to user commands. Especially in the field of music robots, models are often difficult to provide accurate answers according to the needs of different user groups (such as professionals and non-professionals), and there are limitations in the accumulation of professional knowledge. To solve these problems, this paper puts forward a framework based on Retrieval-Augmented Generation (RAG) and Prompt-Tuning. Under this framework, we effectively construct two different answer modes, professional mode and non-professional mode, to meet the needs of different user groups. Improve the performance of the model in a specific domain. The experimental results show that the combination of external knowledge retrieval and LoRA fine-tuning method can effectively improve the performance of the model in understanding user instructions and generating tasks, and provide an innovative solution for intelligent dialogue applications in vertical fields.