Self-correcting complex semantic analysis method based on pre-training mechanism

Qing LI; Jiang ZHONG; Lili LI; Qi LI

doi:10.11959/j.issn.1000-436x.2019195

您当前的位置：

首页 >

文章列表页 >

Self-correcting complex semantic analysis method based on pre-training mechanism

Papers | 更新时间：2024-06-05

- Self-correcting complex semantic analysis method based on pre-training mechanism
- Journal on Communications Vol. 40, Issue 12, Pages: 41-50(2019)
- 作者机构：
  
  1. 重庆大学计算机学院，重庆 400044
  2. 重庆大学土木工程学院，重庆 400044
  3. 绍兴文理学院计算机科学与工程系，浙江绍兴 312000
- 作者简介：
- 基金信息：
  
  Fundamental Research Funds for the Central Universities(2018CDYJSY0055);The National Key Research and Development Program of China(2017YFB1402400);The National Key Research and Development Program of China(CYB18058);Chongqing Technological Innovation and Application Demonstration Project(cstc2018jszx-cyzdX0086)
- DOI：10.11959/j.issn.1000-436x.2019195
  CLC： TP302.1
- Online First：2019-12，
  
  Published：25 December 2019
- 稿件说明：
移动端阅览
Qing LI, Jiang ZHONG, Lili LI, et al. Self-correcting complex semantic analysis method based on pre-training mechanism[J]. Journal on Communications, 2019, 40(12): 41-50.
DOI：

Qing LI, Jiang ZHONG, Lili LI, et al. Self-correcting complex semantic analysis method based on pre-training mechanism[J]. Journal on Communications, 2019, 40(12): 41-50. DOI： 10.11959/j.issn.1000-436x.2019195.

摘要

面向知识服务过程中内容资源的智能化、知识化、精细化和重组化的碎片性管理需求。深层分析并挖掘语义隐层知识、技术、经验与信息，突破已有传统文本到结构化查询语言（SQL）的语义分析技术瓶颈，提出基于预训练机制的自修正复杂语义分析方法PT-Sem2SQL。设计结合Kullback-Leibler差异技术的MT-DNN预训练机制，以加强上下文语义理解深度；设计专有增强模块，捕获句内上下文语义信息的位置；并通过自修正方法优化生成模型的执行过程，以解决解码过程中的错误输出。实验结果表明，PT-Sem2SQL 能够有效提高复杂语义的解析性能，准确度优于相关工作。

Abstract

In the process of knowledge service

in order to meet the fragmentation management needs of intellectualization

knowledge ability

refinement and reorganization content resources.Through deep analysis and mining of semantic hidden knowledge

technology

experience

and information

it broke through the existing bottleneck of traditional semantic parsing technology from Text-to-SQL.The PT-Sem2SQL based on the pre-training mechanism was proposed.The MT-DNN pre-training model mechanism combining Kullback-Leibler technology was designed to enhance the depth of context semantic understanding.A proprietary enhancement module was designed that captured the location of contextual semantic information within the sentence.Optimize the execution process of the generated model by the self-correcting method to solve the error output during decoding.The experimental results show that PT-Sem2SQL can effectively improve the parsing performance of complex semantics

and its accuracy is better than related work.

关键词

Keywords

references

CHOI E , HE H , IYYER M , et al . QuAC:question answering in context [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 2174 - 2184 .

SUHR A , IYER S , ARTZI Y . Learning to map context-dependent sentences to executable formal queries [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long Papers) . 2018 : 2238 - 2249 .

李青 , 钟将 , 李立力 , 等 . 领域专业知识富关联关系提取方法研究 [J ] . 控制与决策 , 2019 : 1 - 9 .

LI Q , ZHONG J , LI L L , et al . Research on the extraction method of multiple semantic relations in domain knowledge [J ] . Control and De-cision , 2019 : 1 - 9 .

孙加东 , 赵铁军 , 梁华参 . 基于结构对齐的统计机器翻译模型 [J ] . 通信学报 , 2009 , 30 ( 7 ): 124 - 129 .

SUN J D , ZHAO T J , LIANG H S . Statistical machine translation model based on structure alignment [J ] . Journal on Communications , 2009 , 30 ( 7 ): 124 - 129 .

LI Q , ZHONG J , TAO Y , et al . Research of the processing technology for time complex event based on LSTM [J ] . Cluster Computing , 2019 , 22 : 9571 - 9579 .

GUO Y , SHANG X , LI Z . Identification of cancer subtypes by integrating multiple types of transcriptomics data with deep learning in breast cancer [J ] . Neurocomputing , 2019 , 324 : 20 - 30 .

YU T , ZHANG R , YANG K , et al . Spider:a large-scale human-labeled dataset for complex and cross-domain semantic parsing and Text-to-SQL task [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 3911 - 3921 .

YU T , ZHANG R , YASUNAGA M , et al . SParC:cross-domain semantic parsing in context [J ] . arXiv Preprint,arXiv:1906.02285 , 2019 .

ZHONG V , XIONG C , SOCHER R . Seq2sql:generating structured queries from natural language using reinforcement learning [J ] . arXiv Preprint,arXiv:1709.00103 , 2017 .

United States.Defense advanced research projects agency . speech and natural language:proceedings of a workshop held at Hidden Valley,Pennsylvania,June 24-27,1990 [M ] . Morgan Kaufmann Pub , 1990 .

DAHL D A , BATES M , BROWN M , et al . Expanding the scope of the ATIS task:The ATIS-3 corpus [C ] // The Workshop on Human Language Technology . Association for Computational Linguistics , 1994 : 43 - 48 .

FINEGAN-DOLLAK C , KUMMERFELD J K , ZHANG L , et al . Improving Text-to-SQL Evaluation Methodology [C ] // The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) . 2018 : 351 - 360 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [C ] // The 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers) . 2019 : 4171 - 4186 .

LIU X , HE P , CHEN W , et al . Multi-task deep neural networks for natural language understanding [J ] . arXiv Preprint,arXiv:1901.11504 , 2019 .

SOLOMON J , CHARETTE F . Hierarchical multi-task deep neural network architecture for end-to-end driving [J ] . arXiv Preprint,arXiv:1902.03466 , 2019 .

KULLBACK S . Letter to the editor:the Kullback-Leibler distance [J ] . American Statistician , 1987 , 41 ( 4 ): 340 - 341 .

WARREN D H D , PEREIRA F C N . An efficient easily adaptable system for interpreting natural language queries [J ] . Computational Linguistics , 1982 , 8 ( 3-4 ): 110 - 122 .

WONG Y W , MOONEY R . Learning synchronous grammars for semantic parsing with lambda calculus [C ] // The 45th Annual Meeting of the Association of Computational Linguistics . 2007 : 960 - 967 .

ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming [C ] // The National Conference on Artificial Intelligence . 1996 : 1050 - 1055 .

ZETTLEMOYER L S , COLLINS M . Learning to map sentences to logical form:structured classification with probabilistic categorial grammars [C ] // The Twenty-First Conference on Uncertainty in Artificial Intelligence . AUAI Press , 2005 : 658 - 666 .

ZHAO K , HUANG L . Type-driven incremental semantic parsing with polymorphism [C ] // The 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . 2015 : 1416 - 1421 .

HERZIG J , BERANT J . Decoupling structure and lexicon for zero-shot semantic parsing [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 1619 - 1629 .

SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks [C ] // Advances in Neural Information Processing Systems . 2014 : 3104 - 3112 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [C ] // International Conference on Learning Representations,abs/1409 . 0473 , 2014 .

YU T , LI Z , ZHANG Z , et al . TypeSQL:knowledge-based type-aware neural Text-to-SQL generation [C ] // The 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2 (Short Papers) . 2018 : 588 - 594 .

DONG L , LAPATA M . Coarse-to-fine decoding for neural semantic parsing [C ] // The 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers) . 2018 : 731 - 742 .

WANG C , BROCKSCHMIDT M , SINGH R . Pointing out SQL queries from text [J ] . 2018 :

PENNINGTON J , SOCHER R , MANNING C . Glove:global vectors for word representation [C ] // The 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 2014 : 1532 - 1543 .

HWANG W , YIM J , PARK S , et al . A comprehensive exploration on WikiSQL with table-aware word contextualization [J ] . arXiv Preprint,arXiv:1902.01069 , 2019 .

RADFORD A , NARASIMHAN K , SALIMANS T , et al . Improving language understanding by generative pre-training [J ] . American Statistician , 2018 .

YU T , YASUNAGA M , YANG K , et al . SyntaxSQLNet:Syntax tree networks for complex and cross-domain Text-to-SQL task [C ] // The 2018 Conference on Empirical Methods in Natural Language Processing . 2018 : 1653 - 1663 .

TANG L R , MOONEY R J . Using multiple clause constructors in inductive logic programming for semantic parsing [C ] // European Conference on Machine Learning . Springer , 2001 : 466 - 477 .

POPESCU A M , ETZIONI O , KAUTZ H . Towards a theory of natural language interfaces to databases [C ] // The 8th International Conference on Intelligent User Interfaces . ACM , 2003 : 149 - 157 .

ZELLE J M , MOONEY R J . Learning to parse database queries using inductive logic programming [C ] // The National Conference on Artificial Intelligence . 1996 : 1050 - 1055 .

IYER S , KONSTAS I , CHEUNG A , et al . Learning a neural semantic parser from user feedback [J ] . arXiv Preprint,arXiv:1704.08760 , 2017 .

LI F , JAGADISH H V . Constructing an interactive natural language interface for relational databases [J ] . Proceedings of the VLDB Endowment , 2014 , 8 ( 1 ): 73 - 84 .

YAGHMAZADEH N , WANG Y , DILLIG I , et al . SQLizer:query synthesis from natural language [J ] . Proceedings of the ACM on Programming Languages , 2017 , 1(OOPSLA) :63.

SMITH S L , KINDERMANS P J , YING C , et al . Don’t decay the learning rate,increase the batch size [J ] . arXiv Preprint,arXiv:1711.00489 , 2017 , 41 ( 4 ): 340 - 341 .

Views

1182

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Construction of advanced persistent threat attack detection model based on provenance graph and attention mechanism

ABAC access control policy generation technique based on deep learning

Multi-factor person entity relation extraction model based on distant supervision

Research of proactive complex event processing method

Parallel complex event processing system based on S4 framework

Related Author

Hao LUO

Xinyu WANG

Jiexuan YUAN

Yuancheng LI

Rui QIAO

Na WANG

Xuehui DU

Aodi LIU

Related Institution

School of Control and Computer Engineering, North China Electric Power University

Information Engineering University

He’nan Province Key Laboratory of Information Security

Zhoukou Normal University

College of Computer,National University of Defense Technology

AI问答

⁰