Research on Q-learning based rate control approach for HTTP adaptive streaming

Li-rong XIONG; Jing-zhi LEI; Xin JIN

doi:10.11959/j.issn.1000-436x.2017178

您当前的位置：

首页 >

文章列表页 >

Research on Q-learning based rate control approach for HTTP adaptive streaming

Papers | 更新时间：2024-06-05

- Research on Q-learning based rate control approach for HTTP adaptive streaming
- Journal on Communications Vol. 38, Issue 9, Pages: 18-24(2017)
- 作者机构：
  
  浙江工业大学计算机科学与技术学院，浙江杭州 310023
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.1000-436x.2017178
  CLC： TN919.85
- Online First：2017-09，
  
  Published：25 September 2017
- 稿件说明：
移动端阅览
Li-rong XIONG, Jing-zhi LEI, Xin JIN. Research on Q-learning based rate control approach for HTTP adaptive streaming[J]. Journal on Communications, 2017, 38(9): 18-24.
DOI：

Li-rong XIONG, Jing-zhi LEI, Xin JIN. Research on Q-learning based rate control approach for HTTP adaptive streaming[J]. Journal on Communications, 2017, 38(9): 18-24. DOI： 10.11959/j.issn.1000-436x.2017178.

摘要

基于HTTP的自适应流HAS已经成为自适应视频流服务的标准。在HAS客户端网络状态多变的情况下，硬编码形式的码率决策方法灵活性偏低，对用户体验考虑不足。为了优化用户体验质量（QoE），提出一种基于Q-Learning 的码率控制算法，结合 HTTP 自适应视频流客户端环境进行建模并定义状态转移规则；量化与用户QoE相关的参数，构建新的回报函数；实验表明引入Q-Learning进行码率调整的自适应算法在码率切换的稳定性方面表现较好。

Abstract

HTTP adaptive streaming (HAS) has become the standard for adaptive video streaming service.In changing network environments

current hardcoded-based rate adaptation algorithm was less flexible

and it is insufficient to consider the quality of experience (QoE).To optimize the QoE of users

a rate control approach based on Q-learning strategy was proposed.the client environments of HTTP adaptive video streaming was modeled and the state transition rule was defined.Three parameters related to QoE were quantified and a novel reward function was constructed.The experiments were employed by the Q-learning rate control approach in two typical HAS algorithms.The experiments show the rate control approach can enhance the stability of rate switching in HAS clients.

关键词

Keywords

references

LIU C , BOUAZIZI I , GABBOUJ M . Rate adaptation for adaptive HTTP streaming [C ] // The Second Annual ACM conference on Multimedia systems . 2011 : 169 - 174 .

THANG T C , HO Q D , KANG J W , et al . Adaptive streaming of audiovisual content using MPEG DASH [J ] . IEEE Transactions on Consumer Electronics , 2012 , 58 ( 1 ): 78 - 85 .

ROMERO L R . A dynamic adaptive http streaming video service for google android [J ] . Communication Systems Cos , 2011 : 33 - 51 .

LIU C , BOUAZIZI I , HANNUKSELA M M , et al . Rate adaptation for dynamic adaptive streaming over HTTP in content distribution network [J ] . Signal Processing Image Communication , 2012 , 27 ( 4 ): 288 - 311 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design of a Q-learningbased client quality selection algorithm for HTTP adaptive video streaming [C ] // Adaptive and Learning Agents Workshop . 2013 : 30 - 37 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design and evaluation of a self-learning HTTP adaptive video streaming client [J ] . IEEE Communications Letters , 2014 , 18 ( 4 ): 716 - 719 .

MART V , GARC N . Q-learning based control algorithm for HTTP adaptive streaming [C ] // 2015 Visual Communications and Image Processing (VCIP) . 2015 : 1 - 4 .

MOK R K P , CHAN E W W , CHANG R K C . Measuring the quality of experience of HTTP video streaming [C ] // The 2011 IFIP/IEEE International Symposium on Integrated Network Management . 2011 : 485 - 492 .

LI Z , ZHU X , GAHM J , et al . Probe and adapt:rate adaptation for http video streaming at scale [J ] . IEEE Journal on Selected Areas in Communications , 2014 , 32 ( 4 ): 719 - 733 .

HESSE S , . Design of scheduling and rate-adaptation algorithms for adaptive HTTP streaming [C ] // Applications of Digital Image Processing . 2013 : 219 - 224 .

GARCIA S , CABRERA J , GARCIA N . Quality-control algorithm for adaptive streaming services over wireless channels [J ] . IEEE Journal of Selected Topics in Signal Processing , 2015 , 9 ( 1 ): 50 - 59 .

MARTIN V , CABRERA J , GARCIA N . Evaluation of Q-Learning approach for HTTP adaptive streaming [C ] // IEEE International Conference on Consumer Electronics . 2016 : 293 - 294 .

ZHOU C , LIN C W , GUO Z . mDASH:a Markov decision-based rate adaptation approach for dynamic HTTP streaming [J ] . IEEE Transactions on Multimedia , 2016 , 18 ( 4 ): 738 - 751 .

ISTEPANIAN R S H , PHILIP N Y , MARTINI M G . Medical QoS provision based on reinforcement learning in ultrasound streaming over 3.5 G wireless systems [J ] . IEEE Journal on Selected Areas in Communications , 2009 , 27 ( 4 ): 566 - 574 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design and optimisation of a (FA) Q-learning-based HTTP adaptive streaming client [J ] . Connection Science , 2014 , 26 ( 1 ): 25 - 43 .

MENKOVSKI V , LIOTTA A . Intelligent control for adaptive video streaming [C ] // International Conference on Consumer Electronics . 2013 : 127 - 128 .

CHIARIOTTI F . Reinforcement learning algorithms for DASH video streaming [D ] . Italian:University of Padova , 2015 .

CHIARIOTTI F , D'ARONCO S L , et al . Online learning adaptation strategy for DASH clients [C ] // International Conference on Multimedia Systems . 2016 :8.

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . MA : MIT Press , 1998 .

TOKIC M , . Adaptive ε-greedy exploration in reinforcement learning based on value differences [C ] // Advances in Artificial Intelligence,German Conference on Ai,Karlsruhe,Germany,September 21-24,2010.Proceedings . DBLP , 2010 : 203 - 210 .

TOKIC M , PALM G . Value-difference based exploration:adaptive control between epsilon-greedy and softmax [C ] // The 34th Annual German Conference on Advances in Artificial Intelligence . 2011 : 335 - 346 .

MUELLER C , LEDERER S , POECHER J , et al . Demo paper:libdash-an open source software library for the MPEG-DASH standard [C ] // The 2013 IEEE International Conference on Multimedia and Expo Workshops . 2013 : 1 - 2 .

Views

2505

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Vehicle platooning scheme based on two-way auction

HEVC perceptual intra-frame rate control inspired by video subjective observation experiment

Q-learning based handoff algorithm for satellite system with ancillary terrestrial component

Research on adaptive rate control algorithm for H.264 based on R-Q model

HVS-based rate-control scheme for object-based embedded image coding

Related Author

PENG Hongye

WANG Tianxi

WU Jinyu

LIU Hai

Ling WANG

Fuping WANG

Qingfan LIN

Kaifang YANG

Related Institution

Key Laboratory of Blockchain and Fintech of Department of Education of Guizhou Province, Guizhou University of Finance and Economics

Guizhou Provincial Key Laboratory of Computing and Network Convergence, Guizhou University of Finance and Economics

School of Information, Guizhou University of Finance and Economics

School of Communications and Information Engineering, Xi’an University of Posts and Telecommunications

International Joint Research Center for Wireless Communication and Information Processing Technology

AI问答

⁰