结合全局向量特征的神经网络依存句法分析模型

王衡军; 司念文; 宋玉龙; 单义栋

doi:10.11959/j.issn.1000-436x.2018024

您当前的位置：

首页 >

文章列表页 >

结合全局向量特征的神经网络依存句法分析模型

学术论文 | 更新时间：2024-06-05

- 结合全局向量特征的神经网络依存句法分析模型
- Neural network model for dependency parsing incorporating global vector feature
- 通信学报 2018年39卷第2期页码：53-64
- 作者机构：
  
  1. 解放军信息工程大学三院，河南郑州 450001
  2. 73671部队，安徽六安 237000
- 作者简介：
  
  [ "王衡军（1973-），男，湖南衡阳人，解放军信息工程大学副教授、硕士生导师，主要研究方向为机器学习、自然语言处理和信息安全。" ]
  [ "司念文（1992-），男，湖北襄阳人，解放军信息工程大学硕士生，主要研究方向为机器学习、自然语言处理。" ]
  [ "宋玉龙（1995-），男，安徽阜阳人，73671部队助理工程师，主要研究方向为网络与信息安全。" ]
  [ "单义栋（1988-），男，山东乳山人，解放军信息工程大学硕士生，主要研究方向为自然语言处理。" ]
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2018024
  中图分类号： TN912.3
- 网络出版日期：2018-02，
  
  纸质出版日期：2018-02-25
- 稿件说明：
移动端阅览
王衡军, 司念文, 宋玉龙, 等. 结合全局向量特征的神经网络依存句法分析模型[J]. 通信学报, 2018,39(2):53-64.

Hengjun WANG, Nianwen SI, Yulong SONG, et al. Neural network model for dependency parsing incorporating global vector feature[J]. Journal on communications, 2018, 39(2): 53-64.
王衡军, 司念文, 宋玉龙, 等. 结合全局向量特征的神经网络依存句法分析模型[J]. 通信学报, 2018,39(2):53-64. DOI： 10.11959/j.issn.1000-436x.2018024.

Hengjun WANG, Nianwen SI, Yulong SONG, et al. Neural network model for dependency parsing incorporating global vector feature[J]. Journal on communications, 2018, 39(2): 53-64. DOI： 10.11959/j.issn.1000-436x.2018024.

摘要

利用时序型长短时记忆（LSTM

long short term memory）网络和分片池化的卷积神经网络（CNN

convolutional neural network），分别提取词向量特征和全局向量特征，将2类特征结合输入前馈网络中进行训练；模型训练中，采用基于概率的训练方法。与改进前的模型相比，该模型能够更多地关注句子的全局特征；相较于最大化间隔训练算法，所提训练方法更充分地利用所有可能的依存句法树进行参数更新。为了验证该模型的性能，在宾州中文树库（CTB5

Chinese Penn Treebank 5）上进行实验，结果表明，与已有的仅使用LSTM或CNN的句法分析模型相比，该模型在保证一定效率的同时，能够有效提升依存分析准确率。

Abstract

LSTM and piecewise CNN were utilized to extract word vector features and global vector features

respectively.Then the two features were input to feed forward network for training.In model training

the probabilistic training method was adopted.Compared with the original dependency paring model

the proposed model focused more on global features

and used all potential dependency trees to update model parameters.Experiments on Chinese Penn Treebank 5 (CTB5) dataset show that

compared with the parsing model using LSTM or CNN only

the proposed model not only remains the relatively low model complexity

but also achieves higher accuracies.

关键词

Keywords

references

MCDONALD R , CRAMMER K , PEREIRA F . Online large-margin training of dependency parsers [C ] // The 43rd Annual Meeting on Association for Computational Linguistics . 2005 : 91 - 98 .

EISNER J M . Three new probabilistic models for dependency parsing:an exploration [J ] . Computer Science , 1997 : 340 - 345 .

MCDONALD R T , PEREIRA F C N . Online learning of approximate dependency parsing algorithms [C ] // The 11th Conference of the European Chapter of the Association for Computational Linguistics . 2006 : 81 - 88 .

CARRERAS X , . Experiments with a higher-order projective dependency parser [C ] // The 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning . 2007 : 957 - 961 .

KOO T , COLLINS M . Efficient third-order dependency parsers [C ] // The 48th Annual Meeting of the Association for Computational Linguistics . 2010 : 1 - 11 .

马学喆 . 依存句法分析的若干关键问题的研究 [D ] . 上海:上海交通大学 , 2013 .

MA X Z . Research on key issues of dependency parsing [D ] . Shanghai:Shanghai Jiaotong University , 2013 .

PEI W Z , GE T , CHANG B B . An effective neural network model for graph-based dependency parsing [C ] // The 53rd Annual Meeting of the Association for Computational Linguistics . 2015 : 313 - 322 .

WANG W H , CHANG B B . Graph-based dependency parsing with bidirectional LSTM [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 2306 - 2315 .

KIPERWASSER E , GOLDBERG Y . Simple and accurate dependency parsing using bidirectional LSTM feature representations [J ] . Transactions of the Association for Computational Linguistics , 2016 ( 4 ): 313 - 327 .

ZHANG Z S , ZHAO H , QIN L H . Probabilistic graph-based dependency parsing with convolutional neural network [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 1382 - 1392 .

MA X Z , HOVY E . Neural probabilistic model for non-projective MST parsing [J ] .arXiv:arXiv,2017. arXiv:arXiv :1701.00874 2017 .

ZHANG J J , ZHANG D K , HAO J . Local translation prediction with global sentence representation [C ] // The 24th International Joint Conference on Artificial Intelligence . 2015 : 1398 - 1404 .

KALCHBRENNER N , GREFENSTETTE E , BLUNSOM P . A convolutional neural network for modelling sentences [J ] . Eprint Arxiv , 2014 ( 1 ).

COLLOBERT R , WESTON J , BOTTOU L , et al . Natural language processing (almost) from scratch [J ] . Journal of Machine Learning Research , 2011 ( 12 ): 2493 - 2537 .

JASON C , ERIC N . Named entity recognition with bidirectional lstm-cnns [J ] . Transactions of the Association for Computational Linguistics , 2016 ( 4 ): 357 - 370 .

曾谁飞 , 张笑燕 , 杜晓峰 , 等 . 基于神经网络的文本表示模型新方法 [J ] . 通信学报 , 2017 , 38 ( 4 ): 86 - 98 .

ZENG S F , ZHANG X Y , DU X F , et al . New method of text representation model based on neural network [J ] . Journal on Communications , 2017 , 38 ( 4 ): 86 - 98 .

CHEN D Q , MANNING C . A fast and accurate dependency parser using neural networks [C ] // Conference on Empirical Methods in Natural Language Processing . 2014 : 740 - 750 .

WEISS D , ALBERTI C , COLLINS M , et al . Structured training for neural network transition-based parsing [C ] // The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing . 2015 : 323 - 333 .

ANDOR D , ALBERTI C , WEISS D , et al . Globally normalized transition-based neural networks [C ] // The 54rd Annual Meeting of the Association for Computational Linguistics . 2016 : 2442 - 2452 .

DYER C , BALLESTEROS M , WANG L , et al . Transition-based dependency parsing with stack long short-term memory [J ] . ComputerScience , 2015 , 37 ( 2 ): 321 - 332 .

CHENG H , FANG H , HE X D , et al . Bi-directional attention with agreement for dependency parsing [C ] // Conference on Empirical Methods in Natural Language Processing . 2016 : 2204 - 2214 .

DOZAT T , MANNING C D . Deep biaffine attention for neural dependency parsing [J ] .arXiv:arXiv 1611.01734,2016. arXiv:arXiv 1611 ,01734, 2016 .

NEUBIG G , DYER C , GOLDBERG Y , et al . DyNet:the dynamic neural network toolkit [J ] . arXiv:arXiv 1701 .03980, 2017 .

ZHANG Y , NIVRE J . Transition-based dependency parsing with rich non-local features [C ] // The 49th Annual Meeting of the Association for Computational Linguistics . 2011 : 188 - 193 .

ZHANG H , MCDONALD R . Enforcing structural diversity in cube-pruned dependency parsing [C ] // The 52nd Annual Meeting of the Association for Computational Linguistics . 2014 : 656 - 666 .

浏览量

1371

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于左右轮廓特征的打印体邮政编码数字快速识别

综合利用颜色和纹理特征的图像检索