Campus question-answering system based on intent recognition and retrieval-augmented generation

TANG Bowen; MA Mingxuan; ZHANG Yining; LI Hourun; WEN Feifan; WANG Dabin; YANG Jia; MA Hao

doi:10.11959/j.issn.1000-436x.2024245

您当前的位置：

首页 >

文章列表页 >

Campus question-answering system based on intent recognition and retrieval-augmented generation

Application of Artificial Intelligent | 更新时间：2024-12-31

- Campus question-answering system based on intent recognition and retrieval-augmented generation
- Journal on Communications Vol. 45, Issue Z2, Pages: 255-261(2024)
- 作者机构：
  
  1.北京大学计算机学院，北京 100871
  2.北京大学信息科学技术学院，北京 100871
  3.北京大学计算中心，北京 100871
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2024245
  CLC： TP182
- Received：22 October 2024，
  
  Published：30 November 2024
- 稿件说明：
移动端阅览
汤博文,马名轩,张以宁等.基于意图识别与检索增强生成的校园问答系统[J].通信学报,2024,45(Z2):255-261.

TANG Bowen,MA Mingxuan,ZHANG Yining,et al.Campus question-answering system based on intent recognition and retrieval-augmented generation[J].Journal on Communications,2024,45(Z2):255-261.
汤博文,马名轩,张以宁等.基于意图识别与检索增强生成的校园问答系统[J].通信学报,2024,45(Z2):255-261. DOI： 10.11959/j.issn.1000-436x.2024245.

TANG Bowen,MA Mingxuan,ZHANG Yining,et al.Campus question-answering system based on intent recognition and retrieval-augmented generation[J].Journal on Communications,2024,45(Z2):255-261. DOI： 10.11959/j.issn.1000-436x.2024245.

摘要

为解决传统校园问答系统信息整合能力不足、泛化能力差等问题，设计了基于大语言模型的校园问答系统。使用微调后的大语言模型对用户问题进行意图识别，为不同意图的问题提供有针对性的处理方法，提升用户体验。同时，针对大语言模型生成时的幻觉问题，利用多种校园数据建立了校园知识库，通过检索增强生成方法为模型提供事实依据。实验结果表明，经过指令微调的开源大语言模型可以达到接近甚至超越闭源大语言模型的意图识别准确率。

Abstract

To address the issues of poor information integration and generalization in traditional campus question-answering systems

a campus question-answering system based on a large language model was designed. The fine-tuned model identified user intents and provided targeted solutions for various types of questions

enhancing the user experience. To mitigate the hallucination problem during language model generation

a knowledge base using diverse campus data was constructed and a retrieval-augmented generation method was employed to ensure factual accuracy. Experimental results indicate that the open-source large language model

after instruction tuning

achieves intent recognition accuracy that is comparable to or even surpasses that of closed-source models.

关键词

Keywords

references

李月 , 周江 . 一种基于文本相似计算的校园智能问答系统设计 [J ] . 现代信息科技 , 2019 , 3 ( 22 ): 9 - 12, 17 .

LI Y , ZHOU J . Design of a campus intelligent question answering system based on text similarity computing [J ] . Modern Information Technology , 2019 , 3 ( 22 ): 9 - 12, 17 .

龙新征 , 郑建宁 , 欧阳荣彬 , 等 . 基于多层策略的校园智能问答系统 [J ] . 华中科技大学学报(自然科学版) , 2016 , 44 ( 11 ): 117 - 122 .

LONG X Z , ZHENG J N , OUYANG R B , et al . University intelligent question answering system based on multi-layer strategy [J ] . Journal of Huazhong University of Science and Technology (Natural Science Edition) , 2016 , 44 ( 11 ): 117 - 122 .

KAPLAN J , MCCANDLISH S , HENIGHAN T , et al . Scaling laws for neural language models [J ] . arXiv Preprint , arXiv: 2001 . 08361 v 1 ， 2020 .

ACHIAM J , ADLER S , AGARWAL S , et al . GPT-4 technical report [J ] . arXiv Preprint arXiv: 2303.08774 , 2023 .

ZHANG Y , LI Y F , CUI L Y , et al . Siren’s song in the AI ocean: a survey on hallucination in large language models [J ] . arXiv Preprint , arXiv: 2309 . 01219 v 2 , 2023 .

LEWIS P , PEREZ E , PIKTUS A , et al . Retrieval-augmented generation for knowledge-intensive NLP tasks [J ] . Advances in Neural Information Processing Systems , 2020 , 33 : 9459 - 9474 .

ARSLAN M , CRUZ C . Business-RAG: information extraction for business insights [C ] // Proceedings of the 21st International Conference on Smart Business Technologies . Setubal : SciTePress , 2024 : 88 - 94 .

GAO L Y , MA X G , LIN J , et al . Precise zero-shot dense retrieval without relevance labels [J ] . arXiv Preprint , arXiv: 2212 . 10496 v 1 , 2022 .

AGARWAL D , FABBRI A R , RISHER B , et al . Prompt Leakage effect and defense strategies for multi-turn LLM interactions [J ] . arXiv Preprint , arXiv: 2404 . 16251 v 3 , 2024 .

CHAN C M , XU C P , YUAN R B , et al . RQ-RAG: learning to refine queries for retrieval augmented generation [J ] . arXiv Preprint , arXiv: 2404 . 00610 v 1 , 2024 .

WELD H , HUANG X , LONG S , et al . A survey of joint intent detection and slot-filling models in natural language understanding [J ] . arXiv Preprint , arXiv: 2101.08091 , 2021 .

LOUVAN S , MAGNINI B . Recent neural methods on slot filling and intent classification for task-oriented dialogue systems: a survey [J ] . arXiv Preprint , arXiv: 2011 . 00564 v 1 , 2020 .

汤博文 , 杨加 , 秦辉东 , 等 . 基于大语言模型的校园网问题解决系统 [C ] // 中国计算机用户协会网络应用分会2023年第二十七届网络新技术与应用年会论文集 . 2023 : 325 - 329 .

TANG B , YANG J , QIN H , et al . Campus Network Problem-Solving System Based on Large Language Models [C ] // Proceedings of the 27th Annual Conference on New Network Technologies and Applications , Network Applications Branch of China Computer Users Association . 2023 : 325 - 329 .

ELASTICSEARCH B V . Elasticsearch [J ] . Software , Version, 2018 , 6 ( 1 ) : 1 .

WEI J , WANG X , SCHUURMANS D , et al . Chain-of-thought prompting elicits reasoning in large language models [J ] . Advances in neural information processing systems , 2022 , 35 : 24824 - 24837 .

YAO S , ZHAO J , YU D , et al . React: Synergizing reasoning and acting in language models [J ] . arXiv Preprint , arXiv: 2210.03629 , 2022 .

ROBERTSON S E , WALKER S , BEAULIEU M M , et al . Okapi at TREC-4 [R ] . NIST , 1996 .

CHEN J , XIAO S , ZHANG P , ET AL . BGE M3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation [J ] . arXiv Preprint , arXiv: 2402.03216 , 2024 .

ZHENG Y , ZHANG R , ZHANG J , et al . Llamafactory: Unified efficient fine-tuning of 100+ language models [J ] . arXiv Preprint , arXiv: 2403.13372 , 2024 .

HU E J , SHEN Y , WALLIS P , et al . Lora: low-rank adaptation of large language models [J ] . arXiv Preprint , arXiv: 2106.09685 , 2021 .

Views

1957

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

AI问答

⁰