基于预训练机制的自修正复杂语义分析方法

李青; 钟将; 李立力; 李琪

doi:10.11959/j.issn.1000-436x.2019195

您当前的位置：

首页 >

文章列表页 >

基于预训练机制的自修正复杂语义分析方法

学术论文 | 更新时间：2024-06-05

- 基于预训练机制的自修正复杂语义分析方法
- Self-correcting complex semantic analysis method based on pre-training mechanism
- 通信学报 2019年40卷第12期页码：41-50
- 作者机构：
  
  1. 重庆大学计算机学院，重庆 400044
  2. 重庆大学土木工程学院，重庆 400044
  3. 绍兴文理学院计算机科学与工程系，浙江绍兴 312000
- 作者简介：
  
  [ "李青（1989- ），女，陕西西安人，重庆大学博士生，主要研究方向为自然语言处理、复杂事件检测、医学信息学" ]
  [ "钟将（1974- ），男，重庆人，博士，重庆大学教授，主要研究方向为自然语言处理、数据挖掘" ]
  [ "李立力（1989- ），男，陕西铜川人，博士，重庆大学博士生，主要研究方向为桥梁健康监测、数据挖掘" ]
  [ "李琪（1987- ），男，江苏盱眙人，博士，绍兴文理学院讲师，主要研究方向为图计算、数据挖掘" ]
- 基金信息：
  
  中央高校基本科研业务费基金资助项目(2018CDYJSY0055);重庆市研究生科研创新基金资助项目(2017YFB1402400);重庆市研究生科研创新基金资助项目(CYB18058);重庆市技术创新与应用示范基金资助项目(cstc2018jszx-cyzdX0086)
- DOI：10.11959/j.issn.1000-436x.2019195
  中图分类号： TP302.1
- 网络出版日期：2019-12，
  
  纸质出版日期：2019-12-25
- 稿件说明：
移动端阅览
李青, 钟将, 李立力, 等. 基于预训练机制的自修正复杂语义分析方法[J]. 通信学报, 2019,40(12):41-50.

Qing LI, Jiang ZHONG, Lili LI, et al. Self-correcting complex semantic analysis method based on pre-training mechanism[J]. Journal on communications, 2019, 40(12): 41-50.
李青, 钟将, 李立力, 等. 基于预训练机制的自修正复杂语义分析方法[J]. 通信学报, 2019,40(12):41-50. DOI： 10.11959/j.issn.1000-436x.2019195.

Qing LI, Jiang ZHONG, Lili LI, et al. Self-correcting complex semantic analysis method based on pre-training mechanism[J]. Journal on communications, 2019, 40(12): 41-50. DOI： 10.11959/j.issn.1000-436x.2019195.

摘要

面向知识服务过程中内容资源的智能化、知识化、精细化和重组化的碎片性管理需求。深层分析并挖掘语义隐层知识、技术、经验与信息，突破已有传统文本到结构化查询语言（SQL）的语义分析技术瓶颈，提出基于预训练机制的自修正复杂语义分析方法PT-Sem2SQL。设计结合Kullback-Leibler差异技术的MT-DNN预训练机制，以加强上下文语义理解深度；设计专有增强模块，捕获句内上下文语义信息的位置；并通过自修正方法优化生成模型的执行过程，以解决解码过程中的错误输出。实验结果表明，PT-Sem2SQL 能够有效提高复杂语义的解析性能，准确度优于相关工作。

Abstract

In the process of knowledge service

in order to meet the fragmentation management needs of intellectualization

knowledge ability

refinement and reorganization content resources.Through deep analysis and mining of semantic hidden knowledge

technology

experience

and information

it broke through the existing bottleneck of traditional semantic parsing technology from Text-to-SQL.The PT-Sem2SQL based on the pre-training mechanism was proposed.The MT-DNN pre-training model mechanism combining Kullback-Leibler technology was designed to enhance the depth of context semantic understanding.A proprietary enhancement module was designed that captured the location of contextual semantic information within the sentence.Optimize the execution process of the generated model by the self-correcting method to solve the error output during decoding.The experimental results show that PT-Sem2SQL can effectively improve the parsing performance of complex semantics

and its accuracy is better than related work.

关键词

Keywords

references

CHOI E , HE H , IYYER M , et al . QuAC:question answering in context [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 2174 - 2184 .

SUHR A , IYER S , ARTZI Y . Learning to map context-dependent sentences to executable formal queries [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long Papers) . 2018 : 2238 - 2249 .

李青 , 钟将 , 李立力 , 等 . 领域专业知识富关联关系提取方法研究 [J ] . 控制与决策 , 2019 : 1 - 9 .

LI Q , ZHONG J , LI L L , et al . Research on the extraction method of multiple semantic relations in domain knowledge [J ] . Control and De-cision , 2019 : 1 - 9 .

孙加东 , 赵铁军 , 梁华参 . 基于结构对齐的统计机器翻译模型 [J ] . 通信学报 , 2009 , 30 ( 7 ): 124 - 129 .

SUN J D , ZHAO T J , LIANG H S . Statistical machine translation model based on structure alignment [J ] . Journal on Communications , 2009 , 30 ( 7 ): 124 - 129 .

LI Q , ZHONG J , TAO Y , et al . Research of the processing technology for time complex event based on LSTM [J ] . Cluster Computing , 2019 , 22 : 9571 - 9579 .

GUO Y , SHANG X , LI Z . Identification of cancer subtypes by integrating multiple types of transcriptomics data with deep learning in breast cancer [J ] . Neurocomputing , 2019 , 324 : 20 - 30 .

YU T , ZHANG R , YANG K , et al . Spider:a large-scale human-labeled dataset for complex and cross-domain semantic parsing and Text-to-SQL task [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 3911 - 3921 .

YU T , ZHANG R , YASUNAGA M , et al . SParC:cross-domain semantic parsing in context [J ] . arXiv Preprint,arXiv:1906.02285 , 2019 .

ZHONG V , XIONG C , SOCHER R . Seq2sql:generating structured queries from natural language using reinforcement learning [J ] . arXiv Preprint,arXiv:1709.00103 , 2017 .

United States.Defense advanced research projects agency . speech and natural language:proceedings of a workshop held at Hidden Valley,Pennsylvania,June 24-27,1990 [M ] . Morgan Kaufmann Pub , 1990 .

DAHL D A , BATES M , BROWN M , et al . Expanding the scope of the ATIS task:The ATIS-3 corpus [C ] // The Workshop on Human Language Technology . Association for Computational Linguistics , 1994 : 43 - 48 .

FINEGAN-DOLLAK C , KUMMERFELD J K , ZHANG L , et al . Improving Text-to-SQL Evaluation Methodology [C ] // The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) . 2018 : 351 - 360 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [C ] // The 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers) . 2019 : 4171 - 4186 .

LIU X , HE P , CHEN W , et al . Multi-task deep neural networks for natural language understanding [J ] . arXiv Preprint,arXiv:1901.11504 , 2019 .

SOLOMON J , CHARETTE F . Hierarchical multi-task deep neural network architecture for end-to-end driving [J ] . arXiv Preprint,arXiv:1902.03466 , 2019 .

KULLBACK S . Letter to the editor:the Kullback-Leibler distance [J ] . American Statistician , 1987 , 41 ( 4 ): 340 - 341 .

WARREN D H D , PEREIRA F C N . An efficient easily adaptable system for interpreting natural language queries [J ] . Computational Linguistics , 1982 , 8 ( 3-4 ): 110 - 122 .

WONG Y W , MOONEY R . Learning synchronous grammars for semantic parsing with lambda calculus [C ] // The 45th Annual Meeting of the Association of Computational Linguistics . 2007 : 960 - 967 .

ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming [C ] // The National Conference on Artificial Intelligence . 1996 : 1050 - 1055 .

ZETTLEMOYER L S , COLLINS M . Learning to map sentences to logical form:structured classification with probabilistic categorial grammars [C ] // The Twenty-First Conference on Uncertainty in Artificial Intelligence . AUAI Press , 2005 : 658 - 666 .

ZHAO K , HUANG L . Type-driven incremental semantic parsing with polymorphism [C ] // The 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . 2015 : 1416 - 1421 .

HERZIG J , BERANT J . Decoupling structure and lexicon for zero-shot semantic parsing [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 1619 - 1629 .

SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks [C ] // Advances in Neural Information Processing Systems . 2014 : 3104 - 3112 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [C ] // International Conference on Learning Representations,abs/1409 . 0473 , 2014 .

YU T , LI Z , ZHANG Z , et al . TypeSQL:knowledge-based type-aware neural Text-to-SQL generation [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2 (Short Papers) . 2018 : 588 - 594 .

DONG L , LAPATA M . Coarse-to-fine decoding for neural semantic parsing [C ] // The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) . 2018 : 731 - 742 .

WANG C , BROCKSCHMIDT M , SINGH R . Pointing out SQL queries from text [J ] . 2018 :

PENNINGTON J , SOCHER R , MANNING C . Glove:global vectors for word representation [C ] // The 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 2014 : 1532 - 1543 .

HWANG W , YIM J , PARK S , et al . A comprehensive exploration on WikiSQL with table-aware word contextualization [J ] . arXiv Preprint,arXiv:1902.01069 , 2019 .

RADFORD A , NARASIMHAN K , SALIMANS T , et al . Improving language understanding by generative pre-training [J ] . American Statistician , 2018 .

YU T , YASUNAGA M , YANG K , et al . SyntaxSQLNet:Syntax tree networks for complex and cross-domain Text-to-SQL task [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 1653 - 1663 .

TANG L R , MOONEY R J . Using multiple clause constructors in inductive logic programming for semantic parsing [C ] // European Conference on Machine Learning . Springer , 2001 : 466 - 477 .

POPESCU A M , ETZIONI O , KAUTZ H . Towards a theory of natural language interfaces to databases [C ] // The 8th International Conference on Intelligent User Interfaces . ACM , 2003 : 149 - 157 .

ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming [C ] // The National Conference on Artificial Intelligence . 1996 : 1050 - 1055 .

IYER S , KONSTAS I , CHEUNG A , et al . Learning a neural semantic parser from user feedback [J ] . arXiv Preprint,arXiv:1704.08760 , 2017 .

LI F , JAGADISH H V . Constructing an interactive natural language interface for relational databases [J ] . Proceedings of the VLDB Endowment , 2014 , 8 ( 1 ): 73 - 84 .

YAGHMAZADEH N , WANG Y , DILLIG I , et al . SQLizer:query synthesis from natural language [J ] . Proceedings of the ACM on Programming Languages , 2017 , 1(OOPSLA) :63.

SMITH S L , KINDERMANS P J , YING C , et al . Don’t decay the learning rate,increase the batch size [J ] . arXiv Preprint,arXiv:1711.00489 , 2017 , 41 ( 4 ): 340 - 341 .

浏览量

577

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于溯源图和注意力机制的APT攻击检测模型构建

基于深度学习的ABAC访问控制策略自动化生成技术

基于远程监督的多因子人物关系抽取模型

主动式复杂事件处理方法的研究

基于S4框架的并行复杂事件处理系统