Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning

Runzi LIU; Tianci MA; Weihua WU; Chenhong YAO; Qinghai YANG

doi:10.11959/j.issn.1000-436x.2023130

您当前的位置：

首页 >

文章列表页 >

Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning

Correspondences | 更新时间：2024-05-31

- Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning
- Journal on Communications Vol. 44, Issue 7, Pages: 207-217(2023)
- 作者机构：
  
  1. 西安建筑科技大学信息与控制工程学院，陕西西安 710399
  2. 陕西师范大学物理学与信息技术学院，陕西西安 710119
  3. 西安电子科技大学通信工程学院，陕西西安 710071
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(61701365);The National Natural Science Foundation of China(61801365);The National Natural Science Foundation of China(61971327);Natural Science Basic Research Program of Shaanxi(2023-JC-YB-566);Natural Science Basic Research Program of Shaanxi(2023-JC-YB-542);Key Research and Development Program of Shaanxi Province(2021GY-066);Young Talent Fund of University Association for Science and Technology in Shannxi Province(20200112);Postdoctoral Foundation of Shaanxi Province(2018BSHEDZZ47)
- DOI：10.11959/j.issn.1000-436x.2023130
  CLC： TN92
- Online First：2023-07，
  
  Published：25 July 2023
- 稿件说明：
移动端阅览
Runzi LIU, Tianci MA, Weihua WU, et al. Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning[J]. Journal on Communications, 2023, 44(7): 207-217.
DOI：

Runzi LIU, Tianci MA, Weihua WU, et al. Dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning[J]. Journal on Communications, 2023, 44(7): 207-217. DOI： 10.11959/j.issn.1000-436x.2023130.

摘要

摘要：近年来，随着各类紧急任务数量的不断增长，如何在控制对常规任务影响的同时保障系统的收益已成为中继卫星网络任务动态调度的巨大挑战。针对这一问题，以最大化紧急任务总收益和最小化常规任务破坏程度为目标，提出了一种基于分层强化学习的中继卫星网络任务动态调度方法。具体而言，为了兼顾系统的长期与短期性能，设计了由上、下级DQN实现的双层调度框架，上级DQN从长期性能出发决定临时优化目标，下级DQN根据优化目标决定当前任务的调度策略。仿真结果表明，与传统的深度学习方法以及部分处理动态调度问题的启发式方法相比，所提方法能够在降低常规任务破坏程度的同时提升紧急任务总收益。

Abstract

In recent years

with the increasing number of various emergency tasks

how to control the impact on common tasks while ensuring system revenue has become a huge challenge for the dynamic scheduling of relay satellite networks.Aiming at this problem

with the goal of maximizing the total revenue of emergency tasks and minimizing the damage to common tasks

a dynamic task scheduling method for relay satellite networks based on hierarchical reinforcement learning was proposed.Specifically

in order to take into account the long-term and short-term performance of the system at the same time

a two-layer scheduling framework implemented by upper-level and lower-level DQN was designed.The upper-level DQN was responsible for determining the temporary optimization goal based on long-term performance

and the lower-level DQN determined the scheduling strategy for current task according to the optimization goal.Simulation results show that compared with traditional deep learning methods and the heuristic methods dealing with dynamic scheduling problems

the proposed method can improve the total revenue of urgent tasks while reducing the damage to common tasks.

关键词

Keywords

references

王家胜 . 中国数据中继卫星系统及其应用拓展 [J ] . 航天器工程 , 2013 , 22 ( 1 ): 1 - 6 .

WANG J . China’s data relay satellite system and its application prospect [J ] . Spacecraft Engineering , 2013 , 22 ( 1 ): 1 - 6 .

贺川 , 朱晓敏 , 邱涤珊 . 面向应急成像观测任务的多星协同调度方法 [J ] . 系统工程与电子技术 , 2012 , 34 ( 4 ): 726 - 731 .

HE C , ZHU X M , QIU D S . Cooperative scheduling method of multi-satellites for imaging reconnaissance in emergency condition [J ] . Systems Engineering and Electronics , 2012 , 34 ( 4 ): 726 - 731 .

WU G H , MA M H , ZHU J H , et al . Multi-satellite observation integrated scheduling method oriented to emergency tasks and common tasks [J ] . Journal of Systems Engineering and Electronics , 2012 , 23 ( 5 ): 723 - 733 .

李飞龙 , 李广侠 , 李志强 , 等 . 基于多层分簇的北斗卫星导航系统拓扑结构与路由策略 [J ] . 通信学报 , 2014 , 35 ( 10 ): 31 - 41 .

LI F L , LI G X , LI Z Q , et al . Topology structure and routing policy based on multilayered clusters in Beidou satellite navigation system [J ] . Journal on Communications , 2014 , 35 ( 10 ): 31 - 41 .

HAN S M , BEAK S W , CHO K R , et al . Satellite mission scheduling using genetic algorithm [C ] // Proceedings of 2008 SICE Annual Conference . Piscataway:IEEE Press , 2008 : 1226 - 1230 .

HUI-CHCNG H , JIANG W , LI Y J . Solving on agile satellites mission planning based on tabu search-parallel genetic algorithms [C ] // Proceed ings of 2013 International Conference on Management Science and Proceedings of Engineering 20th Annual Conference Proceedings . Piscataway:IEEE Press , 2013 : 120 - 125 .

HE L J , LI J D , SHENG M , et al . Dynamic scheduling of hybrid tasks with time windows in data relay satellite networks [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 5 ): 4989 - 5004 .

DENG B Y , JIANG C X , KUANG L L , et al . Two-phase task scheduling in data relay satellite systems [J ] . IEEE Transactions on Vehicular Technology , 2018 , 67 ( 2 ): 1782 - 1793 .

DAI C Q , LI C , FU S , et al . Dynamic scheduling for emergency tasks in space data relay network [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 1 ): 795 - 807 .

ZHANG Z J , ZHANG N , FENG Z R . Multi-satellite control resource scheduling based on ant colony optimization [J ] . Expert Systems with Applications , 2014 , 41 ( 6 ): 2816 - 2823 .

刘润滋 , 盛敏 , 唐成圆 , 等 . 基于任务拆分聚合的中继卫星系统任务规划方法 [J ] . 通信学报 , 2017 , 38 ( S1 ): 110 - 117 .

LIU R Z , SHENG M , TANG C Y , et al . Tasking planning based on task splitting and merging in relay satellite network [J ] . Journal on Communications , 2017 , 38 ( S1 ): 110 - 117 .

ZHU X M , SIM K M , JIANG J Q , et al . Agent-based dynamic scheduling for earth-observing tasks on multiple airships in emergency [J ] . IEEE Systems Journal , 2016 , 10 ( 2 ): 661 - 672 .

ROJANASOONTHON S , BARD J . A GRASP for parallel machine scheduling with time windows [J ] . INFORMS Journal on Computing , 2005 , 17 ( 1 ): 32 - 51 .

ZHOU D , SHENG M , LI J D , et al . Aerospace integrated networks innovation for empowering 6G:a survey and future challenges [J ] . IEEE Communications Surveys ＆ Tutorials , 2023 , 25 ( 2 ): 975 - 1019 .

LUO S , ZHANG L X , FAN Y S . Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning [J ] . Computers ＆Industrial Engineering , 2021 ,159:107489.

Views

1090

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Resource allocation algorithm for multi-beam LEO satellite based on decision performance evaluation

Optimization of air-ground cooperative communication performance for UAV-mounted STAR-RIS

GAT-based decision mechanism for decentralized joint routing and spectrum access

Multi-cluster computing power resource scheduling algorithm based on DDPG reinforcement learning

Task offloading and resource allocation strategy for vehicular edge computing assisted by intelligent reflecting surfaces

Related Author

WANG Chaowei

PANG Mingliang

WANG Su

ZHAO Lingli

GAO Feifei

CUI Gaofeng

WANG Weidong

Han Zhengjiang

Related Institution

School of Electronic Engineering, Beijing University of Posts and Telecommunications

China Mobile Communications Corporation

Department of Automation, Tsinghua University

Electronic Engineering College, Heilongjiang University

Systems Engineering Institute, Academy of Military Science

AI问答

⁰