面向智能任务的语义通信：理论、技术和挑战

刘传宏; 郭彩丽; 杨洋; 陈九九; 朱美逸; 孙鲁楠

doi:10.11959/j.issn.1000-436x.2022117

您当前的位置：

首页 >

文章列表页 >

面向智能任务的语义通信：理论、技术和挑战

专题：面向6G的智能至简网络关键技术 | 更新时间：2024-06-06

- 面向智能任务的语义通信：理论、技术和挑战
- Intelligent task-oriented semantic communications:theory, technology and challenges
- 通信学报 2022年43卷第6期页码：41-57
- 作者机构：
  
  1. 北京邮电大学先进信息网络北京实验室，北京 100876
  2. 北京邮电大学网络体系构建与融合北京市重点实验室，北京 100876
- 作者简介：
  
  [ "刘传宏（1998- ），男，安徽池州人，北京邮电大学博士生，主要研究方向为深度学习、语义通信、资源分配等" ]
  [ "郭彩丽（1977- ），女，山西太原人，博士，北京邮电大学教授、博士生导师，主要研究方向为语义通信、无线移动通信技术、认知无线电、信号检测与估值、车联网、可见光通信、视觉智能计算、社交跨媒体数据挖掘与分析等" ]
  [ "杨洋（1991- ），男，湖南娄底人，博士，北京邮电大学讲师，主要研究方向为可见光通信、室内定位技术、车联网技术、语义通信技术等" ]
  [ "陈九九（1994- ），男，湖南平江人，北京邮电大学博士生，主要研究方向为车联网资源分配、语义通信、强化学习算法等" ]
  [ "朱美逸（1999- ），女，湖北保康人，北京邮电大学博士生，主要研究方向为语义通信、车联网通信、强化学习算法等" ]
  [ "孙鲁楠（1996- ），女，辽宁葫芦岛人，北京邮电大学博士生，主要研究方向为语义通信、图像传输、信源信道编码等" ]
- 基金信息：
  
  中央高校基本科研业务费专项资金资助项目(2021XD-A01-1);北京市自然科学基金资助项目(4202049);北京邮电大学博士生创新基金资助项目(CX2022101)
- DOI：10.11959/j.issn.1000-436x.2022117
  中图分类号： TN929.5
- 网络出版日期：2022-06，
  
  纸质出版日期：2022-06-25
- 稿件说明：
移动端阅览
刘传宏, 郭彩丽, 杨洋, 等. 面向智能任务的语义通信：理论、技术和挑战[J]. 通信学报, 2022,43(6):41-57.

Chuanhong LIU, Caili GUO, Yang YANG, et al. Intelligent task-oriented semantic communications:theory, technology and challenges[J]. Journal on communications, 2022, 43(6): 41-57.
刘传宏, 郭彩丽, 杨洋, 等. 面向智能任务的语义通信：理论、技术和挑战[J]. 通信学报, 2022,43(6):41-57. DOI： 10.11959/j.issn.1000-436x.2022117.

Chuanhong LIU, Caili GUO, Yang YANG, et al. Intelligent task-oriented semantic communications:theory, technology and challenges[J]. Journal on communications, 2022, 43(6): 41-57. DOI： 10.11959/j.issn.1000-436x.2022117.

摘要

目的：未来机-机、人-机万物智能互联对传统通信方式提出了挑战，提取信源语义信息进行传输的语义通信方法为6G通信系统提供了新的解决方法。然而如何度量语义信息、如何实现最优的语义编解码等均存在挑战，本文综述现有语义通信相关的论文，提出面向智能任务的语义通信方法和框架，为进一步推动语义通信的发展铺平道路。

方法：首先综述了语义通信的发展历程和研究现状，通过分析总结了语义通信面临的两大瓶颈问题，提出了面向智能任务的语义通信方法。针对语义熵难度量的问题，本文通过定义构成语义消息的最小基本单元为语义元，引入模糊数学理论刻画语义理解的模糊程度，给出语义信息熵的计算表达式。紧接着，本文基于信息瓶颈理论提出了语义信息编码方案和语义信道联合编码方案，分别考虑了接收端是否需要重建原始信源的场景；此外，从神经网络可解释性的角度出发，提出了基于可解释性的语义编码方法。最后，基于USRP和LabView等软硬件搭建了面向智能任务的语义通信平台，对所提算法进行性能验证。

结果：在需要重建信源的通信场景中，本文所提的语义通信方法可以大大提升信源数据的压缩比，大幅降低传输的数据量；在相同的压缩比下可以提升接收端执行后续智能任务的性能，同时提升信源重建的性能。在无需重建信源的场景中，语义通信方式可以在极大压缩比的情况下，较好地完成智能任务，这是因为语义通信传输图像的语义信息而非图像的所有数据，大大减小了其带宽需求，实现语义通信的带宽利用率超出传统通信方式的100倍。此外，语义通信方式抗噪声性能远远好于传统通信方法，这是因为语义通信方法传输的数据保留了图像的语义特征，且模型训练时考虑了信道噪声的影响，使智能任务性能更优，具有更好的鲁棒性。语义通信方式由于传输数据量大大减少，因此在带宽资源相同的情况下，传输时延显著下降；此外，由于不需要进行图像的重构，软硬件的处理负荷减小，处理时延也有所下降。因此本文所提的方案可以在保证高精度分类性能的同时，大幅减少了端到端智能任务的时延。

结论：面向智能任务的语义通信方法相较于传统通信方法具有明显优势，可以在大幅降低传输数据量的同时提升接收端智能任务的性能，因此语义通信将继续保持快速发展的趋势。然而，语义通信中仍有大量的基础概念和基础问题亟需进一步讨论和完善，如语义信息的基础理论、语义通信的统一架构和语义通信中的资源分配策略等等，对这些问题进行探讨和研究对推动6G时代的技术创新和突破具有重要意义，需要学术同仁共同推动实现。

Abstract

Objectives: In the future

intelligent interconnection of all things

such as machine-to-machine and human-to-machine

poses challenges to traditional communication methods. The semantic communication method that extracts semantic information from source information and transmits them provides a novel solution for the sixth generation (6G) communication system. However

there are challenges in how to measure semantic information and how to achieve optimal semantic codec.This paper reviews the existing works related to semantic communication

and proposes a semantic communication method and framework for intelligent tasks

paving the way for further development of semantic communication.

Methods: Firstly

the development history and research status of semantic communication are reviewed

the two bottleneck problems faced by semantic communication are analyzed and summarized

and a semantic communication method oriented to intelligent tasks is proposed. Aiming at the difficulty of semantic entropy

this paper defines the smallest basic unit of semantic message as semantic element

introduces fuzzy mathematics theory to describe the fuzzy degree of semantic understanding

and gives the calculation expression of semantic information entropy. Then

based on the information bottleneck theory

this paper proposes a semantic information coding scheme and a semantic channel joint coding scheme

respectively considering whether the receiver needs to reconstruct the original source. Furthermore

from the perspective of neural network interpretability

an interpretability-based semantic encoding method is proposed.Finally

a semantic communication platform for intelligent tasks is built based on software and hardware such as USRP and LabView

and the performance of the proposed algorithm is verified.

Results:In the communication scenario where the source needs to be reconstructed

the semantic communication method proposed in this paper can greatly improve the compression ratio of the source data and greatly reduce the amount of transmitted data.Under the same compression ratio

the performance of the receiver to perform subsequent intelligent tasks can be improved

and the performance of source reconstruction can be improved at the same time.In scenarios where there is no need to reconstruct the source

the semantic communication method can better accomplish intelligent tasks with a large compression ratio.This is because semantic communication transmits the semantic information of the image instead of all the data of the image

which greatly reduces its bandwidth requirements

and the bandwidth utilization rate of semantic communication is 100 times higher than that of traditional communication methods. In addition

the anti-noise performance of the semantic communication method is much better than that of the traditional communication method

because the data transmitted by the semantic communication method retains the semantic features of the image

and the influence of channel noise is considered during model training

which makes the performance of intelligent tasks better and makes the communication system more robust. The semantic communication method greatly reduces the amount of data transmitted

so the transmission delay is significantly reduced under the same bandwidth resources.In addition

since image reconstruction is not required

the processing load of software and hardware is reduced

and the processing delay is also reduced. Therefore

the scheme proposed in this paper can greatly reduce the delay of end-to-end intelligent tasks while ensuring high-precision classification performance.

Conclusions: Compared with traditional communication methods

the semantic communication method oriented to intelligent tasks has obvious advantages

which can greatly reduce the amount of transmitted data and improve the performance of intelligent tasks at the receiving end. Therefore

semantic communication will continue to maintain the trend of rapid development. However

there are still a lot of basic concepts and basic problems in semantic communication that need to be further discussed and improved

such as the basic theory of semantic information

the unified architecture of semantic communication

and the resource allocation strategy in semantic communication. Research is of great significance to promoting technological innovation and breakthroughs in the 6G era

and academic colleagues need to jointly promote the realization.

关键词

Keywords

references

CHEN M Z , CHALLITA U , SAAD W , et al . Artificial neural networks-based machine learning for wireless networks:a tutorial [J ] . IEEE Communications Surveys ＆ Tutorials , 2019 , 21 ( 4 ): 3039 - 3071 .

STRINATI E C , BARBAROSSA S . 6G networks:beyond Shannon towards semantic and goal-oriented communications [J ] . Computer Networks , 2021 , 190 : 107930 .

WEAVER W . Recent contributions to the mathematical theory of communication [J ] . ETC:A Review of General Semantics , 1953 , 10 ( 4 ): 261 - 281 .

XIE H Q , QIN Z J , LI G Y , et al . Deep learning enabled semantic communication systems [J ] . IEEE Transactions on Signal Processing , 2021 , 69 : 2663 - 2675 .

CARNAP R , BAR-HILLEL Y , . An outline of a theory of semantic information [R ] . 1952 .

POPPER K . Conjectures and refutations [M ] . New York : Routledge , 2002 .

ZADEH L A . Fuzzy sets [J ] . Information and Control , 1965 , 8 ( 3 ): 338 - 353 .

ZADEH L A . Probability measures of fuzzy events [J ] . Journal of Mathematical Analysis and Applications , 1968 , 23 ( 2 ): 421 - 427 .

LUCA A D , TERMINI S . Algebraic properties of fuzzy sets [J ] . Journal of Mathematical Analysis and Applications , 1972 , 40 ( 2 ): 373 - 386 .

FLORIDI L . Outline of a theory of strongly semantic information [J ] . Minds and Machines , 2004 , 14 ( 2 ): 197 - 221 .

LU C G . From Bayesian inference to logical Bayesian inference:a new mathematical frame for semantic communication and machine learning [J ] . arXiv Preprint,arXiv:1809.01577 , 2018 .

LU C G , . The third kind of Bayes’ theorem links membership functions to likelihood functions and sampling distributions [C ] // International Conference on Cognitive Systems and Signal Processing . Berlin:Springer , 2019 : 268 - 280 .

钟义信 . 信息科学原理(第 3 版) [M ] . 北京 : 北京邮电大学出版社 , 2002 .

ZHONG Y X . Principles of information science(3rd edition) [M ] . Beijing : Beijing University of Posts and Telecommunications Press , 2002 .

钟义信 . 面向智能研究的全信息理论:纪念 Shannon 信息论 50 周年 [J ] . 北京邮电大学学报 , 1998 , 21 ( 4 ): 1 - 6 .

ZHONG Y X . Intelligence oriented comprehensive information theory:in memory of the 50th anniversary of Shannon information theory [J ] . Journal of Beijing University of Posts and Telecommunications , 1998 , 21 ( 4 ): 1 - 6 .

钟义信 , 张瑞 . 信息生态学与语义信息论 [J ] . 图书情报知识 , 2017 ( 6 ): 4 - 11 .

ZHONG Y X , ZHANG R . Information ecology and semantic information theory [J ] . Documentation,Information ＆ Knowledge , 2017 ( 6 ): 4 - 11 .

石光明 , 李莹玉 , 谢雪梅 . 语义通讯:智能时代的产物 [J ] . 模式识别与人工智能 , 2018 , 31 ( 1 ): 91 - 99 .

SHI G M , LI Y Y , XIE X M . Semantic communications:outcome of the intelligence era [J ] . Pattern Recognition and Artificial Intelligence , 2018 , 31 ( 1 ): 91 - 99 .

BAO J , BASU P , DEAN M K , et al . Towards a theory of semantic communication [C ] // Proceedings of 2011 IEEE Network Science Workshop . Piscataway:IEEE Press , 2011 : 110 - 117 .

POPOVSKI P , SIMEONE O , BOCCARDI F , et al . Semantic-effectiveness filtering and control for post-5G wireless connectivity [J ] . Journal of the Indian Institute of Science , 2020 , 100 ( 2 ): 435 - 443 .

KOUNTOURIS M , PAPPAS N . Semantics-empowered communication for networked intelligent systems [J ] . IEEE Communications Magazine , 2021 , 59 ( 6 ): 96 - 102 .

YANG Y , GUO C L , LIU F F , et al . Semantic communications with AI tasks [J ] . arXiv Preprint,arXiv:2109.14170 , 2021 .

KALFA M , GOK M , ATALIK A , et al . Towards goal-oriented semantic signal processing:applications and future challenges [J ] . Digital Signal Processing , 2021 , 119 : 103134 .

牛凯 , 戴金晟 , 张平 , 等 . 面向6G的语义通信 [J ] . 移动通信 , 2021 , 45 ( 4 ): 85 - 90 .

NIU K , DAI J C , ZHANG P , et al . 6G-oriented semantic communications [J ] . Mobile Communications , 2021 , 45 ( 4 ): 85 - 90 .

ZHANG P , XU W J , GAO H , et al . Toward wisdom-evolutionary and primitive-concise 6G:a new paradigm of semantic communication networks [J ] . Engineering , 2022 , 8 : 60 - 73 .

石光明 , 肖泳 , 李莹玉 , 等 . 面向万物智联的语义通信网络 [J ] . 物联网学报 , 2021 , 5 ( 2 ): 26 - 36 .

SHI G M , XIAO Y , LI Y Y , et al . Semantic communication networking for the intelligence of everything [J ] . Chinese Journal on Internet of Things , 2021 , 5 ( 2 ): 26 - 36 .

UYSAL E , KAYA O , EPHREMIDES A , et al . Semantic communications in networked systems [J ] . arXiv Preprint,arXiv:2103.05391 , 2021 .

LAN Q , WEN D , ZHANG Z , et al . What is semantic communication? a view on conveying meaning in the era of machine intelligence [J ] . Journal of Communications and Information Networks , 2021 , 6 ( 4 ): 336 - 371 .

FARSAD N , RAO M , GOLDSMITH A . Deep learning for joint source-channel coding of text [C ] // Proceedings of 2018 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2018 : 2326 - 2330 .

XIE H Q , QIN Z J . A lite distributed semantic communication system for internet of things [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 1 ): 142 - 153 .

WENG Z Z , QIN Z J . Semantic communication systems for speech transmission [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 8 ): 2434 - 2444 .

BOURTSOULATZE E , KURKA D B , GÜNDÜZ D . Deep joint source-channel coding for wireless image transmission [J ] . IEEE Transactions on Cognitive Communications and Networking , 2019 , 5 ( 3 ): 567 - 579 .

LEE C H , LIN J W , CHEN P H , et al . Deep learning-constructed joint transmission-recognition for Internet of things [J ] . IEEE Access , 2019 , 7 : 76547 - 76561 .

JANKOWSKI M , GÜNDÜZ D , MIKOLAJCZYK K . Wireless image retrieval at the edge [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 1 ): 89 - 100 .

刘传宏 , 郭彩丽 , 杨洋 , 等 . 人工智能物联网中面向智能任务的语义通信方法 [J ] . 通信学报 , 2021 , 42 ( 11 ): 97 - 108 .

LIU C H , GUO C L , YANG Y , et al . Intelligent task-oriented semantic communication method in artificial intelligence of things [J ] . Journal on Communications , 2021 , 42 ( 11 ): 97 - 108 .

XIE H Q , QIN Z J , LI G Y . Task-oriented multi-user semantic communications for VQA [J ] . IEEE Wireless Communications Letters , 2022 , 11 ( 3 ): 553 - 557 .

XIE H , QIN Z , TAO X , et al . Task-oriented multi-user semantic communications [J ] . arXiv Preprint,arXiv:2112.10255 , 2021 .

李蕾 , 周延泉 , 钟义信 . 基于语用的自然语言处理研究与应用初探 [J ] . 智能系统学报 , 2006 , 1 ( 2 ): 1 - 6 .

LI L , ZHOU Y Q , ZHONG Y X . Pragmatic information based NLP research and application [J ] . CAAI Transactions on Intelligent Systems , 2006 , 1 ( 2 ): 1 - 6 .

LUCA A D , TERMINI S . A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory [J ] . Information and Control , 1972 , 20 ( 4 ): 301 - 312 .

吴伟陵 . 广义信息源与广义熵 [J ] . 北京邮电大学学报 , 1982 ( 1 ): 29 - 41 .

WU W L . Generalized information source and generalized entropy [J ] . Journal of Beijing University of Posts and Telecommunications , 1982 ( 1 ): 29 - 41 .

BASU P , BAO J , DEAN M , et al . Preserving quality of information by using semantic relationships [J ] . Pervasive and Mobile Computing , 2014 , 11 : 188 - 202 .

HOANG D T , LONG P M , VITTER J S . Rate-distortion optimizations for motion estimation in low-bit-rate video coding [C ] // Digital Video Compression:Algorithms and Technologies .[S.l.:s.n. ] , 1996 : 18 - 27 .

HUYNH-THU Q , GHANBARI M . Scope of validity of PSNR in image/video quality assessment [J ] . Electronics Letters , 2008 , 44 ( 13 ): 800 .

WANG Z , BOVIK A C . A universal image quality index [J ] . IEEE Signal Processing Letters , 2002 , 9 ( 3 ): 81 - 84 .

WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment:from error visibility to structural similarity [J ] . IEEE Transactions on Image Processing , 2004 , 13 ( 4 ): 600 - 612 .

KIM J , LEE S . Deep learning of human visual sensitivity in image quality assessment framework [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 1969 - 1977 .

ZHANG R , ISOLA P , EFROS A A , et al . The unreasonable effectiveness of deep features as a perceptual metric [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 586 - 595 .

HU Y Q , ZHOU W , ZHAO S X , et al . SDM:semantic distortion measurement for video encryption [C ] // Proceedings of 2018 13th IEEE International Conference on Automatic Face ＆ Gesture Recognition . Piscataway:IEEE Press , 2018 : 764 - 768 .

HUSSAIN A J , AL-FAYADH A , RADI N . Image compression techniques:a survey in lossless and lossy algorithms [J ] . Neurocomputing , 2018 , 300 : 44 - 69 .

YANG Z H , WANG Y H , XU C , et al . Discernible image compression [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York:ACM Press , 2020 : 1561 - 1569 .

韦来生 . 数理统计(第2版) [M ] . 北京 : 科学出版社 , 2015 .

WEI L S . Mathematical statistics(2nd edition) [M ] . Beijing : Science Press , 2015 .

SHWARTZ-ZIV R , TISHBY N . Opening the black box of deep neural networks via information [J ] . arXiv Preprint,arXiv:1703.00810 , 2017 .

LI Y Y , ZHAO P P , WANG D Q , et al . Learning disentangled user representation based on controllable VAE for recommendation [C ] // International Conference on Database Systems for Advanced Applications . Berlin:Springer , 2021 : 179 - 194 .

TISHBY N , PEREIRA F C , BIALEK W . The information bottleneck method [J ] . arXiv Preprint,arXiv:physics/0004057 , 2000 .

QIN Z J , TAO X M , LU J H , et al . Semantic communications:principles and challenges [J ] . arXiv Preprint,arXiv:2201.01389 , 2022 .

LIU F F , TONG W J , SUN Z F , et al . Task-oriented semantic communication with semantic reconstruction:an extended rate-distortion theory based scheme [J ] . arXiv Preprint,arXiv:2201.10929 , 2022 .

陈九九 , 冯春燕 , 郭彩丽 , 等 . 车联网中视频语义驱动的资源分配算法 [J ] . 通信学报 , 2021 , 42 ( 7 ): 1 - 11 .

CHEN J J , FENG C Y , GUO C L , et al . Video semantics-driven resource allocation algorithm in Internet of vehicles [J ] . Journal on Communications , 2021 , 42 ( 7 ): 1 - 11 .

浏览量

1782

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向6G的深度图像语义通信模型

6G知识体系构建：面向全域全场景的学术知识挖掘及其按需应用

可扩展分层次编译码的语义通信系统

高效融合全局和局部上下文特征的语义通信系统

知识增强的语义通信接收端设计