融合注意力胶囊的深度因子分解机模型

顾亦然; 姚朱鹏; 杨海根

doi:10.11959/j.issn.1000-436x.2021185

您当前的位置：

首页 >

文章列表页 >

融合注意力胶囊的深度因子分解机模型

学术论文 | 更新时间：2024-06-05

- 融合注意力胶囊的深度因子分解机模型
- Deep factorization machine model based on attention capsule
- 通信学报 2021年42卷第10期页码：130-139
- 作者机构：
  
  1. 南京邮电大学自动化学院、人工智能学院，江苏南京 210023
  2. 南京邮电大学智慧校园研究中心，江苏南京 210023
  3. 南京邮电大学宽带无线通信技术教育部工程研究中心，江苏南京 210003
- 作者简介：
  
  [ "顾亦然（1972- ），女，江苏南京人，博士，南京邮电大学教授、硕士生导师，主要研究方向为复杂网络、大数据处理等" ]
  [ "姚朱鹏（1997- ），男，江苏苏州人，南京邮电大学硕士生，主要研究方向为推荐算法、自然语言处理等" ]
  [ "杨海根（1983- ），男，江苏南京人，博士，南京邮电大学副教授、硕士生导师，主要研究方向为无线通信、虚拟论证、虚拟设计等" ]
- 基金信息：
  
  国防基础科研基金资助项目(JCKY2019210B005);国防基础科研基金资助项目(JCKY2018204B025);国防基础科研基金资助项目(JCKY2017204B011);国防重大工程基金资助项目(ZQ2019D20401);装备发展部仿真预研课题(41401030301)
- DOI：10.11959/j.issn.1000-436x.2021185
  中图分类号： TP181
- 网络出版日期：2021-10，
  
  纸质出版日期：2021-10-25
- 稿件说明：
移动端阅览
顾亦然, 姚朱鹏, 杨海根. 融合注意力胶囊的深度因子分解机模型[J]. 通信学报, 2021,42(10):130-139.

Yiran GU, Zhupeng YAO, Haigen YANG. Deep factorization machine model based on attention capsule[J]. Journal on communications, 2021, 42(10): 130-139.
顾亦然, 姚朱鹏, 杨海根. 融合注意力胶囊的深度因子分解机模型[J]. 通信学报, 2021,42(10):130-139. DOI： 10.11959/j.issn.1000-436x.2021185.

Yiran GU, Zhupeng YAO, Haigen YANG. Deep factorization machine model based on attention capsule[J]. Journal on communications, 2021, 42(10): 130-139. DOI： 10.11959/j.issn.1000-436x.2021185.

摘要

针对深度学习中推荐模型特征组合单一、消解大量有价值特征信息以及过拟合等问题，设计了一种新型的注意力得分机制——注意力胶囊，提出了一种融合注意力胶囊的深度因子分解机模型。基于DeepFM模型，将用户历史点击行为与候选物品进行权重计算，降低了无关特征对模型的影响，充分挖掘了不同历史行为对用户兴趣的差异性影响。训练过程中加入自适应正则化式，在不影响训练速度的前提下，有效地减少了过拟合。在2个公开数据集上进行对比实验，实验结果表明，所提模型相对于其他模型在损失函数和GAUC上均有明显提升。

Abstract

Aiming at the problems of single feature combination of recommendation model

resolution of a large amount of valuable feature information

and over-fitting in deep learning

a new attentional scoring mechanism called attention capsule was designed

and a deep factorization machine model based on attention capsule was proposed.Users’ historical clicking and candidate items were processed through weight calculation based on the DeepFM model

reducing the impact of irrelevant features on the model

and the differential impact of different historical behaviors on users’ interests was fully explored.The adaptive regularization formulation was added to the training

which effectively reduced over-fitting without affecting the training speed.The comparison test on two public data sets shows that the proposed model is significantly enhanced in loss function and GAUC compared to other models.

关键词

Keywords

references

SEDHAIN S , MENON A K , SANNER S , et al . AutoRec:autoencoders meet collaborative filtering [C ] // Proceedings of the 24th International Conference on World Wide Web . New York:ACM Press , 2015 : 111 - 112 .

HE X N , LIAO L Z , ZHANG H W , et al . Neural collaborative filtering [C ] // Proceedings of the 26th International Conference on World Wide Web . New York:ACM Press , 2017 : 173 - 182 .

SHAN Y , HOENS T R , JIAO J , et al . Deep crossing:Web-scale modeling without manually crafted combinatorial features [C ] // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York:ACM Press , 2016 : 255 - 262 .

QU Y R , CAI H , REN K , et al . Product-based neural networks for user response prediction [C ] // Proceedings of 2016 IEEE 16th International Conference on Data Mining (ICDM) . Piscataway:IEEE Press , 2016 : 1149 - 1154 .

WANG R X , FU B , FU G , et al . Deep ＆ cross network for ad click predictions [C ] // Proceedings of the 23nd ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2017 : 1 - 7 .

GUO H , TANG R , YE Y , et al . DeepFM:a factorization-machine based neural network for CTR prediction [C ] // Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence . New York:ACM Press , 2017 : 1725 - 1731 .

ZHU H , LI X , ZHANG P Y , et al . Learning tree-based deep model for recommender systems [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1079 - 1088 .

CHEN Q W , ZHAO H , LI W , et al . Behavior sequence transformer for e-commerce recommendation in Alibaba [C ] // Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data . New York:ACM Press , 2019 : 1 - 4 .

RENDLE S , . Factorization machines [C ] // Proceedings of 2010 IEEE International Conference on Data Mining . Piscataway:IEEE Press , 2010 : 995 - 1000 .

YUAN W H , WANG H , YU X M , et al . Attention-based context-aware sequential recommendation model [J ] . Information Sciences , 2020 , 510 : 122 - 134 .

GHASEMIAN A , HOSSEINMARDI H , CLAUSET A . Evaluating overfit and underfit in models of network community structure [J ] . IEEE Transactions on Knowledge and Data Engineering , 2020 , 32 ( 9 ): 1722 - 1735 .

WU D R , YUAN Y , HUANG J , et al . Optimize TSK fuzzy systems for regression problems:minibatch gradient descent with regularization,DropRule,and AdaBound (MBGD-RDA) [J ] . IEEE Transactions on Fuzzy Systems , 2020 , 28 ( 5 ): 1003 - 1015 .

ZHU H , JIN J Q , TAN C , et al . Optimized cost per click in Taobao display advertising [C ] // Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York:ACM Press , 2017 : 2191 - 2200 .

QUEVEDO J R , MONTAÑÉS E , RANILLA J , et al . Ranked tag recommendation systems based on logistic regression [M ] . Berlin : Springer , 2010 .

XIAO J , YE H , HE X N , et al . Attentional factorization machines:learning the weight of feature interactions via attention networks [C ] // Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence . New York:ACM Press , 2017 : 3119 - 3125 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Advances in Neural Information Processing Systems . Massachusetts:MIT Press , 2017 : 5998 - 6008 .

LIAN J X , ZHOU X H , ZHANG F Z , et al . xDeepFM:combining explicit and implicit feature interactions for recommender systems [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1754 - 1763 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural networks from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 ( 1 ): 1929 - 1958 .

浏览量

420

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

超大规模太赫兹系统深度学习信道估计算法

基于机器学习的加密流量分类研究综述

基于深度学习的SDN异常流量分布式检测方法

基于Ngram-TFIDF的深度恶意代码可视化分类方法

基于后门攻击的恶意流量逃逸方法