Deep factorization machine model based on attention capsule

Yiran GU; Zhupeng YAO; Haigen YANG

doi:10.11959/j.issn.1000-436x.2021185

您当前的位置：

首页 >

文章列表页 >

Deep factorization machine model based on attention capsule

Papers | 更新时间：2024-06-05

- Deep factorization machine model based on attention capsule
- Journal on Communications Vol. 42, Issue 10, Pages: 130-139(2021)
- 作者机构：
  
  1. 南京邮电大学自动化学院、人工智能学院，江苏南京 210023
  2. 南京邮电大学智慧校园研究中心，江苏南京 210023
  3. 南京邮电大学宽带无线通信技术教育部工程研究中心，江苏南京 210003
- 作者简介：
- 基金信息：
  
  The National Defense Basic Scientific Research Program of China(JCKY2019210B005);The National Defense Basic Scientific Research Program of China(JCKY2018204B025);The National Defense Basic Scientific Research Program of China(JCKY2017204B011);The Major National Defense Projects of China(ZQ2019D20401);The Simulation Pre Research Project of Equipment Development Department of China(41401030301)
- DOI：10.11959/j.issn.1000-436x.2021185
  CLC： TP181
- Online First：2021-10，
  
  Published：25 October 2021
- 稿件说明：
移动端阅览
Yiran GU, Zhupeng YAO, Haigen YANG. Deep factorization machine model based on attention capsule[J]. Journal on Communications, 2021, 42(10): 130-139.
DOI：

Yiran GU, Zhupeng YAO, Haigen YANG. Deep factorization machine model based on attention capsule[J]. Journal on Communications, 2021, 42(10): 130-139. DOI： 10.11959/j.issn.1000-436x.2021185.

摘要

针对深度学习中推荐模型特征组合单一、消解大量有价值特征信息以及过拟合等问题，设计了一种新型的注意力得分机制——注意力胶囊，提出了一种融合注意力胶囊的深度因子分解机模型。基于DeepFM模型，将用户历史点击行为与候选物品进行权重计算，降低了无关特征对模型的影响，充分挖掘了不同历史行为对用户兴趣的差异性影响。训练过程中加入自适应正则化式，在不影响训练速度的前提下，有效地减少了过拟合。在2个公开数据集上进行对比实验，实验结果表明，所提模型相对于其他模型在损失函数和GAUC上均有明显提升。

Abstract

Aiming at the problems of single feature combination of recommendation model

resolution of a large amount of valuable feature information

and over-fitting in deep learning

a new attentional scoring mechanism called attention capsule was designed

and a deep factorization machine model based on attention capsule was proposed.Users’ historical clicking and candidate items were processed through weight calculation based on the DeepFM model

reducing the impact of irrelevant features on the model

and the differential impact of different historical behaviors on users’ interests was fully explored.The adaptive regularization formulation was added to the training

which effectively reduced over-fitting without affecting the training speed.The comparison test on two public data sets shows that the proposed model is significantly enhanced in loss function and GAUC compared to other models.

关键词

Keywords

references

SEDHAIN S , MENON A K , SANNER S , et al . AutoRec:autoencoders meet collaborative filtering [C ] // Proceedings of the 24th International Conference on World Wide Web . New York:ACM Press , 2015 : 111 - 112 .

HE X N , LIAO L Z , ZHANG H W , et al . Neural collaborative filtering [C ] // Proceedings of the 26th International Conference on World Wide Web . New York:ACM Press , 2017 : 173 - 182 .

SHAN Y , HOENS T R , JIAO J , et al . Deep crossing:Web-scale modeling without manually crafted combinatorial features [C ] // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York:ACM Press , 2016 : 255 - 262 .

QU Y R , CAI H , REN K , et al . Product-based neural networks for user response prediction [C ] // Proceedings of 2016 IEEE 16th International Conference on Data Mining (ICDM) . Piscataway:IEEE Press , 2016 : 1149 - 1154 .

WANG R X , FU B , FU G , et al . Deep ＆ cross network for ad click predictions [C ] // Proceedings of the 23nd ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2017 : 1 - 7 .

GUO H , TANG R , YE Y , et al . DeepFM:a factorization-machine based neural network for CTR prediction [C ] // Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence . New York:ACM Press , 2017 : 1725 - 1731 .

ZHU H , LI X , ZHANG P Y , et al . Learning tree-based deep model for recommender systems [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1079 - 1088 .

CHEN Q W , ZHAO H , LI W , et al . Behavior sequence transformer for e-commerce recommendation in Alibaba [C ] // Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data . New York:ACM Press , 2019 : 1 - 4 .

RENDLE S , . Factorization machines [C ] // Proceedings of 2010 IEEE International Conference on Data Mining . Piscataway:IEEE Press , 2010 : 995 - 1000 .

YUAN W H , WANG H , YU X M , et al . Attention-based context-aware sequential recommendation model [J ] . Information Sciences , 2020 , 510 : 122 - 134 .

GHASEMIAN A , HOSSEINMARDI H , CLAUSET A . Evaluating overfit and underfit in models of network community structure [J ] . IEEE Transactions on Knowledge and Data Engineering , 2020 , 32 ( 9 ): 1722 - 1735 .

WU D R , YUAN Y , HUANG J , et al . Optimize TSK fuzzy systems for regression problems:minibatch gradient descent with regularization,DropRule,and AdaBound (MBGD-RDA) [J ] . IEEE Transactions on Fuzzy Systems , 2020 , 28 ( 5 ): 1003 - 1015 .

ZHU H , JIN J Q , TAN C , et al . Optimized cost per click in Taobao display advertising [C ] // Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York:ACM Press , 2017 : 2191 - 2200 .

QUEVEDO J R , MONTAÑÉS E , RANILLA J , et al . Ranked tag recommendation systems based on logistic regression [M ] . Berlin : Springer , 2010 .

XIAO J , YE H , HE X N , et al . Attentional factorization machines:learning the weight of feature interactions via attention networks [C ] // Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence . New York:ACM Press , 2017 : 3119 - 3125 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Advances in Neural Information Processing Systems . Massachusetts:MIT Press , 2017 : 5998 - 6008 .

LIAN J X , ZHOU X H , ZHANG F Z , et al . xDeepFM:combining explicit and implicit feature interactions for recommender systems [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM Press , 2018 : 1754 - 1763 .

SRIVASTAVA N , HINTON G , KRIZHEVSKY A , et al . Dropout:a simple way to prevent neural networks from overfitting [J ] . The Journal of Machine Learning Research , 2014 , 15 ( 1 ): 1929 - 1958 .

Views

1088

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Review of unsupervised anomaly detection techniques for medical imaging

Target visibility prediction method based on cost-sensitive CNN-BiLSTM network

Multi-modal semantics aided mmWave beam prediction method in industrial haze environment

Robust RF fingerprint identification under temporal domain shifts via confidence discrepancy and entropy minimization

Malware detection method based on API temporal relationship graph

Related Author

Zhao Yingcheng

Song Xiaogang

Shi Zhenghao

You Zhenzhen

Hei Xinhong

Liao Xi

Lin Ruijia

Zheng Xiangquan

Related Institution

School of Computer Science and Engineering, Xi’an University of Technology

Shaanxi Key Laboratory for Network Computing and Security Technology

Human Machine Integration Intelligent Robot Shaanxi Provincial University Engineering Research Center

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications

PLA Unit

AI问答

⁰