基于Q-learning的HTTP自适应流码率控制方法研究

熊丽荣; 雷静之; 金鑫

doi:10.11959/j.issn.1000-436x.2017178

您当前的位置：

首页 >

文章列表页 >

基于Q-learning的HTTP自适应流码率控制方法研究

学术论文 | 更新时间：2024-06-05

- 基于Q-learning的HTTP自适应流码率控制方法研究
- Research on Q-learning based rate control approach for HTTP adaptive streaming
- 通信学报 2017年38卷第9期页码：18-24
- 作者机构：
  
  浙江工业大学计算机科学与技术学院，浙江杭州 310023
- 作者简介：
  
  [ "熊丽荣（1973-），女，湖北崇阳人，浙江工业大学副教授，主要研究方向为服务计算和计算机网络。" ]
  [ "雷静之（1992-），女，江西抚州人，浙江工业大学硕士生，主要研究方向为服务计算、计算机网络。" ]
  [ "金鑫（1991-），男，浙江台州人，浙江工业大学硕士生，主要研究方向为服务计算。" ]
- 基金信息：
  
  浙江省重大科技专项重大工业基金资助项目;The Key Project of Science and Technology Planning Project of Zhejiang Province(2012C11026-2);杭州市重大科技创新专项基金资助项目;The Special Found for Science and Technology Innovation Program of Hangzhou(20132011A16)
- DOI：10.11959/j.issn.1000-436x.2017178
  中图分类号： TN919.85
- 网络出版日期：2017-09，
  
  纸质出版日期：2017-09-25
- 稿件说明：
移动端阅览
熊丽荣, 雷静之, 金鑫. 基于Q-learning的HTTP自适应流码率控制方法研究[J]. 通信学报, 2017,38(9):18-24.

Li-rong XIONG, Jing-zhi LEI, Xin JIN. Research on Q-learning based rate control approach for HTTP adaptive streaming[J]. Journal on communications, 2017, 38(9): 18-24.
熊丽荣, 雷静之, 金鑫. 基于Q-learning的HTTP自适应流码率控制方法研究[J]. 通信学报, 2017,38(9):18-24. DOI： 10.11959/j.issn.1000-436x.2017178.

Li-rong XIONG, Jing-zhi LEI, Xin JIN. Research on Q-learning based rate control approach for HTTP adaptive streaming[J]. Journal on communications, 2017, 38(9): 18-24. DOI： 10.11959/j.issn.1000-436x.2017178.

摘要

基于HTTP的自适应流HAS已经成为自适应视频流服务的标准。在HAS客户端网络状态多变的情况下，硬编码形式的码率决策方法灵活性偏低，对用户体验考虑不足。为了优化用户体验质量（QoE），提出一种基于Q-Learning 的码率控制算法，结合 HTTP 自适应视频流客户端环境进行建模并定义状态转移规则；量化与用户QoE相关的参数，构建新的回报函数；实验表明引入Q-Learning进行码率调整的自适应算法在码率切换的稳定性方面表现较好。

Abstract

HTTP adaptive streaming (HAS) has become the standard for adaptive video streaming service.In changing network environments

current hardcoded-based rate adaptation algorithm was less flexible

and it is insufficient to consider the quality of experience (QoE).To optimize the QoE of users

a rate control approach based on Q-learning strategy was proposed.the client environments of HTTP adaptive video streaming was modeled and the state transition rule was defined.Three parameters related to QoE were quantified and a novel reward function was constructed.The experiments were employed by the Q-learning rate control approach in two typical HAS algorithms.The experiments show the rate control approach can enhance the stability of rate switching in HAS clients.

关键词

Keywords

references

LIU C , BOUAZIZI I , GABBOUJ M . Rate adaptation for adaptive HTTP streaming [C ] // The Second Annual ACM conference on Multimedia systems . 2011 : 169 - 174 .

THANG T C , HO Q D , KANG J W , et al . Adaptive streaming of audiovisual content using MPEG DASH [J ] . IEEE Transactions on Consumer Electronics , 2012 , 58 ( 1 ): 78 - 85 .

ROMERO L R . A dynamic adaptive http streaming video service for google android [J ] . Communication Systems Cos , 2011 : 33 - 51 .

LIU C , BOUAZIZI I , HANNUKSELA M M , et al . Rate adaptation for dynamic adaptive streaming over HTTP in content distribution network [J ] . Signal Processing Image Communication , 2012 , 27 ( 4 ): 288 - 311 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design of a Q-learningbased client quality selection algorithm for HTTP adaptive video streaming [C ] // Adaptive and Learning Agents Workshop . 2013 : 30 - 37 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design and evaluation of a self-learning HTTP adaptive video streaming client [J ] . IEEE Communications Letters , 2014 , 18 ( 4 ): 716 - 719 .

MART V , GARC N . Q-learning based control algorithm for HTTP adaptive streaming [C ] // 2015 Visual Communications and Image Processing (VCIP) . 2015 : 1 - 4 .

MOK R K P , CHAN E W W , CHANG R K C . Measuring the quality of experience of HTTP video streaming [C ] // The 2011 IFIP/IEEE International Symposium on Integrated Network Management . 2011 : 485 - 492 .

LI Z , ZHU X , GAHM J , et al . Probe and adapt:rate adaptation for http video streaming at scale [J ] . IEEE Journal on Selected Areas in Communications , 2014 , 32 ( 4 ): 719 - 733 .

HESSE S , . Design of scheduling and rate-adaptation algorithms for adaptive HTTP streaming [C ] // Applications of Digital Image Processing . 2013 : 219 - 224 .

GARCIA S , CABRERA J , GARCIA N . Quality-control algorithm for adaptive streaming services over wireless channels [J ] . IEEE Journal of Selected Topics in Signal Processing , 2015 , 9 ( 1 ): 50 - 59 .

MARTIN V , CABRERA J , GARCIA N . Evaluation of Q-Learning approach for HTTP adaptive streaming [C ] // IEEE International Conference on Consumer Electronics . 2016 : 293 - 294 .

ZHOU C , LIN C W , GUO Z . mDASH:a Markov decision-based rate adaptation approach for dynamic HTTP streaming [J ] . IEEE Transactions on Multimedia , 2016 , 18 ( 4 ): 738 - 751 .

ISTEPANIAN R S H , PHILIP N Y , MARTINI M G . Medical QoS provision based on reinforcement learning in ultrasound streaming over 3.5 G wireless systems [J ] . IEEE Journal on Selected Areas in Communications , 2009 , 27 ( 4 ): 566 - 574 .

CLAEYS M , LATRÉ S , FAMAEY J , et al . Design and optimisation of a (FA) Q-learning-based HTTP adaptive streaming client [J ] . Connection Science , 2014 , 26 ( 1 ): 25 - 43 .

MENKOVSKI V , LIOTTA A . Intelligent control for adaptive video streaming [C ] // International Conference on Consumer Electronics . 2013 : 127 - 128 .

CHIARIOTTI F . Reinforcement learning algorithms for DASH video streaming [D ] . Italian:University of Padova , 2015 .

CHIARIOTTI F , D'ARONCO S L , et al . Online learning adaptation strategy for DASH clients [C ] // International Conference on Multimedia Systems . 2016 :8.

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . MA : MIT Press , 1998 .

TOKIC M , . Adaptive ε-greedy exploration in reinforcement learning based on value differences [C ] // Advances in Artificial Intelligence,German Conference on Ai,Karlsruhe,Germany,September 21-24,2010.Proceedings . DBLP , 2010 : 203 - 210 .

TOKIC M , PALM G . Value-difference based exploration:adaptive control between epsilon-greedy and softmax [C ] // The 34th Annual German Conference on Advances in Artificial Intelligence . 2011 : 335 - 346 .

MUELLER C , LEDERER S , POECHER J , et al . Demo paper:libdash-an open source software library for the MPEG-DASH standard [C ] // The 2013 IEEE International Conference on Multimedia and Expo Workshops . 2013 : 1 - 2 .

浏览量

1012

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于双向拍卖的车辆编队方案

视频主观观测实验启发的HEVC感知帧内码率控制

星地融合网络中基于Q学习的切换算法研究

H.264中一种基于R-Q模型的自适应码率控制算法研究

结合视觉感知的对象基嵌入式图像编码中的码率控制方法