Joint vibrotactile coding for machine recognition and human perception

Ying FANG; Yiwen XU; Tiesong ZHAO

doi:10.11959/j.issn.1000-436x.2023099

您当前的位置：

首页 >

文章列表页 >

Joint vibrotactile coding for machine recognition and human perception

Topics: Multi-/Cross-Modal Semantic Communications | 更新时间：2024-06-06

- Joint vibrotactile coding for machine recognition and human perception
- Journal on Communications Vol. 44, Issue 5, Pages: 42-51(2023)
- 作者机构：
  
  福州大学福建省媒体信息智能处理与无线传输重点实验室，福建福州 350001
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62171134);The Natural Science Foundation of Fujian Province(2022J02015)
- DOI：10.11959/j.issn.1000-436x.2023099
  CLC： TP37
- Online First：2023-05，
  
  Published：25 May 2023
- 稿件说明：
移动端阅览
Ying FANG, Yiwen XU, Tiesong ZHAO. Joint vibrotactile coding for machine recognition and human perception[J]. Journal on Communications, 2023, 44(5): 42-51.
DOI：

Ying FANG, Yiwen XU, Tiesong ZHAO. Joint vibrotactile coding for machine recognition and human perception[J]. Journal on Communications, 2023, 44(5): 42-51. DOI： 10.11959/j.issn.1000-436x.2023099.

摘要

为了精确地传输信号内容含义，实现智能识别与信号重建，针对振动触觉信号，提出了一种面向机器识别-人类感知的联合编码方案。在编码端，将三维振动信号转化为一维信号，采用短时傅里叶变换提取信号的语义信息，并实现语义信息高效压缩与表征。在解码端，基于语义信息采用全卷积神经网络实现触觉的智能识别；同时，将原始信号与基于语义信息的重构信号的残差值作为语义信息的补偿，逐步提高重构信号的质量，满足人类感知需求。实验结果表明，所提方案用较低比特率的语义信息实现触觉识别，同时在满足人类感知需求情况下，触觉数据的压缩效率有所提高。

Abstract

In order to accurately transmit the content meaning of vibrotactile signals and achieve intelligent recognition and signal reconstruction

a joint vibrotactile coding scheme for machine recognition and human perception was proposed.At the encoding end

the original three-dimensional vibrotactile signals were converted into one-dimensional signals.Then the semantic information of the signals was extracted using a short-time Fourier transform before being effectively compressed and transmitted.At the decoding end

a fully convolutional neural network was used to intelligently recognize based on the semantic information.The difference between the original signals and the reconstructed signals based on semantic information was used as compensation for the semantic information

and the quality of the reconstructed signals was gradually improved to meet human perceptual needs.The experimental results show that the proposed scheme achieve tactile recognition with semantic information at a lower bit rate while improving the compression efficiency of tactile data

thus satisfying human perceptual needs.

关键词

Keywords

references

HUANG H , HSIEH M H . Tactile emotional coding:the perceptual linking of vibrotactile stimuli with basic emotions [C ] // Proceedings of 2019 IEEE 2nd International Conference on Knowledge Innovation and Invention (ICKII) . Piscataway:IEEE Press , 2020 : 134 - 137 .

ANTONAKOGLOU K , XU X , STEINBACH E , et al . Toward haptic communications over the 5G tactile Internet [J ] . IEEE Communications Surveys ＆ Tutorials , 2018 , 20 ( 4 ): 3034 - 3059 .

XU X , CIZMECI B , SCHUWERK C , et al . Haptic data reduction for time-delayed teleoperation using the time domain passivity approach [C ] // Proceedings of 2015 IEEE World Haptics Conference (WHC) . Piscataway:IEEE Press , 2015 : 512 - 518 .

STEINBACH E , STRESE M , EID M , et al . Haptic codecs for the tactile Internet [J ] . Proceedings of the IEEE , 2019 , 107 ( 2 ): 447 - 470 .

刘传宏 , 郭彩丽 , 杨洋 , 等 . 面向智能任务的语义通信:理论、技术和挑战 [J ] . 通信学报 , 2022 , 43 ( 6 ): 41 - 57 .

LIU C H , GUO C L , YANG Y , et al . Intelligent task-oriented semantic communications:theory,technology and challenges [J ] . Journal on Communications , 2022 , 43 ( 6 ): 41 - 57 .

DUAN L Y , LIU J Y , YANG W H , et al . Video coding for machines:a paradigm of collaborative compression and intelligent analytics [J ] . IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society , 2020 , 29 : 8680 - 8695 .

HU Y Y , YANG S , YANG W H , et al . Towards coding for human and machine vision:a scalable image coding approach [C ] // Proceedings of 2020 IEEE International Conference on Multimedia and Expo (ICME) . Piscataway:IEEE Press , 2020 : 1 - 6 .

LIU K , LIU D , LI L , et al . Semantics-to-signal scalable image compression with learned revertible representations [J ] . International Journal of Computer Vision , 2021 , 129 ( 9 ): 2605 - 2621 .

CHOI H , BAJIĆ I V . Scalable image coding for humans and machines [J ] . IEEE Transactions on Image Processing , 2022 , 31 : 2739 - 2754 .

HUANG Z M , JIA C M , WANG S S , et al . HMFVC:a human-machine friendly video compression scheme [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 ,doi:10.1109/TCSVT.2022.3207596.

XIA S F , LIANG K , YANG W H , et al . An emerging coding paradigm vcm:a scalable coding approach beyond feature and signal [C ] // Proceedings of 2020 IEEE International Conference on Multimedia and Expo (ICME) . Piscataway:IEEE Press , 2020 : 1 - 6 .

LIU X , DOHLER M . A data-driven approach to vibrotactile data compression [C ] // Proceedings of 2019 IEEE International Workshop on Signal Processing Systems (SiPS) . Piscataway:IEEE Press , 2020 : 341 - 346 .

CHAUDHARI R , SCHUWERK C , DANAEI M , et al . Perceptual and bitrate-scalable coding of haptic surface texture signals [J ] . IEEE Journal of Selected Topics in Signal Processing , 2015 , 9 ( 3 ): 462 - 473 .

OKAMOTO S , YAMADA Y . Perceptual properties of vibrotactile material texture:effects of amplitude changes and stimuli beneath detection thresholds [C ] // Proceedings of 2010 IEEE/SICE International Symposium on System Integration . Piscataway:IEEE Press , 2011 : 384 - 389 .

HASSEN R , GÜLECYÜZ B , STEINBACH E . PVC-SLP:perceptual vibrotactile-signal compression based-on sparse linear prediction [J ] . IEEE Transactions on Multimedia , 2021 , 23 : 4455 - 4468 .

NOLL A , NOCKENBERG L , GÜLECYÜZ B , et al . VC-PWQ:vibrotactile signal compression based on perceptual wavelet quantization [C ] // Proceedings of 2021 IEEE World Haptics Conference (WHC) . Piscataway:IEEE Press , 2021 : 427 - 432 .

KIRSCH J , NOLL A , STRESE M , et al . A low-cost acquisition,display,and evaluation setup for tactile codec development [C ] // Proceedings of 2018 IEEE International Symposium on Haptic,Audio and Visual Environments and Games (HAVE) . Piscataway:IEEE Press , 2018 : 1 - 6 .

LANDIN N , ROMANO J M , MCMAHAN W , et al . Dimensional reduction of high-frequency accelerations for haptic rendering [C ] // International Conference on Human Haptic Sensing and Touch Enabled Computer Applications . Berlin:Springer , 2010 : 79 - 86 .

李昂 , 陈建新 , 魏昕 , 等 . 面向6G的跨模态信号重建技术 [J ] . 通信学报 , 2022 , 43 ( 6 ): 28 - 40 .

LI ANG , CHEN J X , WEI X , et al . 6G-oriented cross-modal signal reconstruction technology [J ] . Journal on Communications , 2022 , 43 ( 6 ): 28 - 40 .

SAJEEVAN N , ARATHI N M , ARAVIND S R , et al . Surface material classification using acceleration signal [C ] // Proceedings of International Conference on Communication and Computational Technologies . Berlin:Springer , 2021 : 49 - 58 .

ZHENG H T , FANG L , JI M Q , et al . Deep learning for surface material classification using haptic and visual information [J ] . IEEE Transactions on Multimedia , 2016 , 18 ( 12 ): 2407 - 2416 .

XIAO J Q , ZHOU Z Y . Research progress of RNN language model [C ] // Proceedings of 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) . Piscataway:IEEE Press , 2020 : 1285 - 1288 .

WANG Z G , YAN W Z , OATES T . Time series classification from scratch with deep neural networks:a strong baseline [C ] // Proceedings of 2017 International Joint Conference on Neural Networks (IJCNN) . Piscataway:IEEE Press , 2017 : 1578 - 1585 .

HASSEN R , STEINBACH E . Subjective evaluation of the spectral temporal SIMilarity (ST-SIM) measure for vibrotactile quality assessment [J ] . IEEE Transactions on Haptics , 2020 , 13 ( 1 ): 25 - 31 .

JI M Q , FANG L , ZHENG H T , et al . Preprocessing-free surface material classification using convolutional neural networks pretrained by sparse autoencoder [C ] // Proceedings of 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP) . Piscataway:IEEE Press , 2015 : 1 - 6 .

Views

1629

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Feature dependence graph based source code loophole detection method

Related Author

Hongyu YANG

Haiyun YANG

Liang ZHANG

Xiang CHENG

Related Institution

School of Safety Science and Engineering, Civil Aviation University of China

School of Computer Science and Technology, Civil Aviation University of China

School of Information, University of Arizona, Tucson 21

School of Information Engineering, Yangzhou University

Jiangsu Engineering Research Center for Knowledge Management and Intelligent Service

AI问答

⁰