基于混合maxout单元的卷积神经网络性能优化

赵慧珍; 刘付显; 李龙跃; 罗畅

doi:10.11959/j.issn.1000-436x.2017145

您当前的位置：

首页 >

文章列表页 >

基于混合maxout单元的卷积神经网络性能优化

学术论文 | 更新时间：2024-06-05

- 基于混合maxout单元的卷积神经网络性能优化
- Improving deep convolutional neural networks with mixed maxout units
- 通信学报 2017年38卷第7期页码：105-114
- 作者机构：
  
  空军工程大学防空反导学院，陕西西安 710051
- 作者简介：
  
  [ "赵慧珍（1990-），女，山东单县人，空军工程大学博士生，主要研究方向为深度学习、计算机视觉。" ]
  [ "刘付显（1962-），男，山东曹县人，空军工程大学教授，主要研究方向为作战建模与仿真、指挥决策优化。" ]
  [ "李龙跃（1988-），男，河南驻马店人，空军工程大学博士生，主要研究方向为作战建模与仿真、指挥决策优化。" ]
  [ "罗畅（1988-），男，四川广安人，空军工程大学博士生，主要研究方向为深度学习、计算机视觉。" ]
- 基金信息：
  
  国家自然科学基金资助项目(61601499)
- DOI：10.11959/j.issn.1000-436x.2017145
  中图分类号： TP391.3
- 网络出版日期：2017-07，
  
  纸质出版日期：2017-07-25
- 稿件说明：
移动端阅览
赵慧珍, 刘付显, 李龙跃, 等. 基于混合maxout单元的卷积神经网络性能优化[J]. 通信学报, 2017,38(7):105-114.

Hui-zhen ZHAO, Fu-xian LIU, Long-yue LI, et al. Improving deep convolutional neural networks with mixed maxout units[J]. Journal on communications, 2017, 38(7): 105-114.
赵慧珍, 刘付显, 李龙跃, 等. 基于混合maxout单元的卷积神经网络性能优化[J]. 通信学报, 2017,38(7):105-114. DOI： 10.11959/j.issn.1000-436x.2017145.

Hui-zhen ZHAO, Fu-xian LIU, Long-yue LI, et al. Improving deep convolutional neural networks with mixed maxout units[J]. Journal on communications, 2017, 38(7): 105-114. DOI： 10.11959/j.issn.1000-436x.2017145.

摘要

针对深度卷积神经网络中 maxout 单元非最大特征无法传递、特征图像子空间池化表达能力不足的局限性，提出混合maxout (mixout

mixed maxout)单元。首先，计算相同输入在不同卷积变换下所形成的特征图像子空间的指数概率分布；其次，根据概率分布计算特征图像子空间的期望；最后，利用伯努利分布对子空间的最大值与期望值加权，均衡单元模型。分别构建基于 mixout 单元的简单模型和网中网模型进行实验，结果表明 mixout单元模型性能较好。

Abstract

The maxout units have the problem of not delivering non-max features， resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings

when they are applied in deep convolutional neural networks.The mixed maxout (mixout) units were proposed to deal with this constrain.Firstly

the exponential probability of the feature mappings getting from different linear transformations was computed.Then

the averaging of a subspace of different feature mappings by the exponential probability was computed.Finally

the output was randomly sampled from the max feature and the mean value by the Bernoulli distribution

leading to the better utilizing of model averaging ability of dropout.The simple models and network in network models was built to evaluate the performance of mixout units.The results show that mixout units based models have better performance.

关键词

Keywords

references

周昌令 , 栾兴龙 , 肖建国 . 基于深度学习的域名查询行为向量空间嵌入 [J ] . 通信学报 , 2016 , 37 ( 3 ): 165 - 174 .

ZHOU C L , LUAN X L , XIAO J G . Vector space embedding of DNS query behaviors by deep learning [J ] . Journal on Communications , 2016 , 37 ( 3 ): 165 - 174 .

杨钊 , 陶大鹏 , 张树业 , 等 . 大数据下的基于深度神经网的相似汉字识别 [J ] . 通信学报 , 2014 , 35 ( 9 ): 184 - 189 .

YANG Z , TAO D P , ZHANG S Y , et al . Similar handwritten Chinese character recognition based on deep neural networks with big data [J ] . Journal on Communications , 2014 , 35 ( 9 ): 184 - 189 .

SPRINGENBERG J T , RIEDMILLER M . Improving deep neural networks with probabilistic maxout units [J ] . arXiv preprint arXiv:1312.6116 , 2013 .

HINTON G E , SRIVASTAVA N , KRIZHEVSKY A , et al . Improving neural networks by preventing co-adaptation of feature detectors [J ] . arXiv preprint arXiv:1207.0580 , 2012 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [C ] // The 26th Annual Conference on Neural Information Processing Systems . 2012 : 1097 - 1105 .

WANG S I , MANNING C D . Fast dropout training [C ] // The 30th International Conference on Machine Learning . 2013 : 118 - 126 .

BA J , FREY B . Adaptive dropout for training deep neural networks [C ] // The Advances in Neural Information Processing Systems . 2013 : 3084 - 3092 .

TOMPSON J , GOROSHIN R , JAIN A , et al . Efficient object localization using convolutional networks [C ] // The IEEE Conference on Computer Vision and Pattern Recognition . 2015 : 648 - 656 .

WAN L , ZEILER M , ZHANG S , et al . Regularization of neural networks using dropconnect [C ] // The 30th International Conference on Machine Learning . 2013 : 1058 - 1066 .

GOODFELLOW I J , WARDE-FARLEY D , MIRZA M , et al . Maxout networks [C ] // The 30th International Conference on Machine Learning . 2013 : 1319 - 1327 .

ZEILER M D , FERGUS R . Stochastic pooling for regularization of deep convolutional neural networks [J ] . arXiv Preprint arXiv:1301.3557 , 2013 .

NAIR V , HINTON G E . Rectified linear units improve restricted Boltzmann machines [C ] // The 27th International Conference on Machine Learning . 2010 : 807 - 814 .

MAAS A L , HANNUN A Y , NG A Y . Rectifier nonlinearities improve neural network acoustic models [C ] // The 30th International Conference on Machine Learning . 2013 ,30(1).

HE K , ZHANG X , REN S , et al . Delving deep into rectifiers:surpassing human-level performance on ImageNet classification [C ] // The IEEE International Conference on Computer Vision . 2015 : 1026 - 1034 .

WANG S . General constructive representations for continuous piecewise-linear functions [J ] . IEEE Transactions on Circuits and Systems I:Regular Papers , 2004 , 51 ( 9 ): 1889 - 1896 .

LI J , WANG X , XU B . Understanding the dropout strategy and analyzing its effectiveness on LVCSR [C ] // 38th IEEE International Conference on Acoustics,Speech,and Signal Processing . 2013 : 7614 - 7618 .

YU D , WANG H , CHEN P , et al . Mixed pooling for convolutional neural networks [C ] // International Conference on Rough Sets and Knowledge Technology . 2014 : 364 - 375 .

LIN M , CHEN Q , YAN S . Network in network [J ] . Journal of Artificial Intelligence Research ， 2011 ， 41 （ 2 ）： 297 - 327 .

KRIZHEVSKY A , HINTON G E . Learning multiple layers of features from tiny images [R ] . Computer Science Department,University of Toronto,Tech . 2009 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [J ] . arxiv Prepraint arXiv:1512.03385 , 2015 .

NETZER Y , WANG T , COATES A , et al . Reading digits in natural images with unsupervised feature learning [R ] . NIPS Workshop on Deep Learning and Unsupervised Feature Learning , 2011 .

CHANG J R , CHEN Y S . Batch-normalized maxout network in network [J ] . arXiv Preprint arXiv:1511.02583 , 2015 .

JIN X , XU C , FENG J , et al . Deep learning with s-shaped rectified linear activation units [J ] . arXiv Preprint arXiv:1512.07030 , 2015 .

LIAO Z , CARNEIRO G . On the importance of normalisation layers in deep learning with piecewise linear activation units [C ] // The 2016 IEEE Winter Conference on Applications of Computer Vision . 2016 : 1 - 8 .

XU B , WANG N , CHEN T , et al . Empirical evaluation of rectified activations in convolutional network [J ] . arXiv Preprint arXiv:1505.00853 , 2015 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural networks from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 ( 1 ): 1929 - 1958 .

CLEVERT D A , UNTERTHINER T , HOCHREITER S . Fast and accurate deep network learning by exponential linear units (ELUS) [J ] . arXiv preprint arXiv:1511.07289 , 2015 .

浏览量

1046

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于改进YOLOv3-SPP算法的道路车辆检测

基于深度学习的光学遥感图像目标检测研究进展

基于上下文注意力CNN的三维点云语义分割

基于图像深度学习的无线电信号识别

基于卷积神经网络的交通场景语义分割方法研究