Neural network model for dependency parsing incorporating global vector feature

Hengjun WANG; Nianwen SI; Yulong SONG; Yidong SHAN

doi:10.11959/j.issn.1000-436x.2018024

您当前的位置：

首页 >

文章列表页 >

Neural network model for dependency parsing incorporating global vector feature

Papers | 更新时间：2024-06-05

- Neural network model for dependency parsing incorporating global vector feature
- Journal on Communications Vol. 39, Issue 2, Pages: 53-64(2018)
- 作者机构：
  
  1. 解放军信息工程大学三院，河南郑州 450001
  2. 73671部队，安徽六安 237000
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2018024
  CLC： TN912.3
- Online First：2018-02，
  
  Published：25 February 2018
- 稿件说明：
移动端阅览
Hengjun WANG, Nianwen SI, Yulong SONG, et al. Neural network model for dependency parsing incorporating global vector feature[J]. Journal on Communications, 2018, 39(2): 53-64.
DOI：

Hengjun WANG, Nianwen SI, Yulong SONG, et al. Neural network model for dependency parsing incorporating global vector feature[J]. Journal on Communications, 2018, 39(2): 53-64. DOI： 10.11959/j.issn.1000-436x.2018024.

摘要

利用时序型长短时记忆（LSTM

long short term memory）网络和分片池化的卷积神经网络（CNN

convolutional neural network），分别提取词向量特征和全局向量特征，将2类特征结合输入前馈网络中进行训练；模型训练中，采用基于概率的训练方法。与改进前的模型相比，该模型能够更多地关注句子的全局特征；相较于最大化间隔训练算法，所提训练方法更充分地利用所有可能的依存句法树进行参数更新。为了验证该模型的性能，在宾州中文树库（CTB5

Chinese Penn Treebank 5）上进行实验，结果表明，与已有的仅使用LSTM或CNN的句法分析模型相比，该模型在保证一定效率的同时，能够有效提升依存分析准确率。

Abstract

LSTM and piecewise CNN were utilized to extract word vector features and global vector features

respectively.Then the two features were input to feed forward network for training.In model training

the probabilistic training method was adopted.Compared with the original dependency paring model

the proposed model focused more on global features

and used all potential dependency trees to update model parameters.Experiments on Chinese Penn Treebank 5 (CTB5) dataset show that

compared with the parsing model using LSTM or CNN only

the proposed model not only remains the relatively low model complexity

but also achieves higher accuracies.

关键词

Keywords

references

MCDONALD R , CRAMMER K , PEREIRA F . Online large-margin training of dependency parsers [C ] // The 43rd Annual Meeting on Association for Computational Linguistics . 2005 : 91 - 98 .

EISNER J M . Three new probabilistic models for dependency parsing:an exploration [J ] . Computer Science , 1997 : 340 - 345 .

MCDONALD R T , PEREIRA F C N . Online learning of approximate dependency parsing algorithms [C ] // The 11th Conference of the European Chapter of the Association for Computational Linguistics . 2006 : 81 - 88 .

CARRERAS X , . Experiments with a higher-order projective dependency parser [C ] // The 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning . 2007 : 957 - 961 .

KOO T , COLLINS M . Efficient third-order dependency parsers [C ] // The 48th Annual Meeting of the Association for Computational Linguistics . 2010 : 1 - 11 .

马学喆 . 依存句法分析的若干关键问题的研究 [D ] . 上海:上海交通大学 , 2013 .

MA X Z . Research on key issues of dependency parsing [D ] . Shanghai:Shanghai Jiaotong University , 2013 .

PEI W Z , GE T , CHANG B B . An effective neural network model for graph-based dependency parsing [C ] // The 53rd Annual Meeting of the Association for Computational Linguistics . 2015 : 313 - 322 .

WANG W H , CHANG B B . Graph-based dependency parsing with bidirectional LSTM [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 2306 - 2315 .

KIPERWASSER E , GOLDBERG Y . Simple and accurate dependency parsing using bidirectional LSTM feature representations [J ] . Transactions of the Association for Computational Linguistics , 2016 ( 4 ): 313 - 327 .

ZHANG Z S , ZHAO H , QIN L H . Probabilistic graph-based dependency parsing with convolutional neural network [C ] // The 54th Annual Meeting of the Association for Computational Linguistics . 2016 : 1382 - 1392 .

MA X Z , HOVY E . Neural probabilistic model for non-projective MST parsing [J ] .arXiv:arXiv,2017. arXiv:arXiv :1701.00874 2017 .

ZHANG J J , ZHANG D K , HAO J . Local translation prediction with global sentence representation [C ] // The 24th International Joint Conference on Artificial Intelligence . 2015 : 1398 - 1404 .

KALCHBRENNER N , GREFENSTETTE E , BLUNSOM P . A convolutional neural network for modelling sentences [J ] . Eprint Arxiv , 2014 ( 1 ).

COLLOBERT R , WESTON J , BOTTOU L , et al . Natural language processing (almost) from scratch [J ] . Journal of Machine Learning Research , 2011 ( 12 ): 2493 - 2537 .

JASON C , ERIC N . Named entity recognition with bidirectional lstm-cnns [J ] . Transactions of the Association for Computational Linguistics , 2016 ( 4 ): 357 - 370 .

曾谁飞 , 张笑燕 , 杜晓峰 , 等 . 基于神经网络的文本表示模型新方法 [J ] . 通信学报 , 2017 , 38 ( 4 ): 86 - 98 .

ZENG S F , ZHANG X Y , DU X F , et al . New method of text representation model based on neural network [J ] . Journal on Communications , 2017 , 38 ( 4 ): 86 - 98 .

CHEN D Q , MANNING C . A fast and accurate dependency parser using neural networks [C ] // Conference on Empirical Methods in Natural Language Processing . 2014 : 740 - 750 .

WEISS D , ALBERTI C , COLLINS M , et al . Structured training for neural network transition-based parsing [C ] // The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing . 2015 : 323 - 333 .

ANDOR D , ALBERTI C , WEISS D , et al . Globally normalized transition-based neural networks [C ] // The 54rd Annual Meeting of the Association for Computational Linguistics . 2016 : 2442 - 2452 .

DYER C , BALLESTEROS M , WANG L , et al . Transition-based dependency parsing with stack long short-term memory [J ] . ComputerScience , 2015 , 37 ( 2 ): 321 - 332 .

CHENG H , FANG H , HE X D , et al . Bi-directional attention with agreement for dependency parsing [C ] // Conference on Empirical Methods in Natural Language Processing . 2016 : 2204 - 2214 .

DOZAT T , MANNING C D . Deep biaffine attention for neural dependency parsing [J ] .arXiv:arXiv 1611.01734,2016. arXiv:arXiv 1611 ,01734, 2016 .

NEUBIG G , DYER C , GOLDBERG Y , et al . DyNet:the dynamic neural network toolkit [J ] . arXiv:arXiv 1701 .03980, 2017 .

ZHANG Y , NIVRE J . Transition-based dependency parsing with rich non-local features [C ] // The 49th Annual Meeting of the Association for Computational Linguistics . 2011 : 188 - 193 .

ZHANG H , MCDONALD R . Enforcing structural diversity in cube-pruned dependency parsing [C ] // The 52nd Annual Meeting of the Association for Computational Linguistics . 2014 : 656 - 666 .

Views

1915

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

基于左右轮廓特征的打印体邮政编码数字快速识别

综合利用颜色和纹理特征的图像检索

Related Author

GUAN Mingxiang

WU Zhou

GAO Kuandong

DAI Yijue

WANG Le

CUI Yingjie

TANG Hui

吕岳

Related Institution

School of Information and Communication, Shenzhen University of Information Technology

School of Sino-German Robotics, Shenzhen University of Information Technology

邮电部第三研究所

清华大学电子工程系

AI问答

⁰