Cross-modality person re-identification based on multi-granularity fusion and cross-scale perception

CHENG Deqiang; JI Guangkai; ZHANG Haoxiang; JIANG He; KOU Qiqi

doi:10.11959/j.issn.1000-436x.2025019

您当前的位置：

首页 >

文章列表页 >

Cross-modality person re-identification based on multi-granularity fusion and cross-scale perception

Papers | 更新时间：2025-02-13

- Cross-modality person re-identification based on multi-granularity fusion and cross-scale perception
- Journal on Communications Vol. 46, Issue 1, Pages: 108-123(2025)
- 作者机构：
  
  1.中国矿业大学信息与控制工程学院，江苏徐州 221116
  2.中国矿业大学计算机科学与技术学院，江苏徐州 221116
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(52204117;52304182);The Key Research and Development Program of Jining City(2023KJHZ007)
- DOI：10.11959/j.issn.1000-436x.2025019
  CLC： TP391
- Received：02 September 2024，
  
  Revised：2024-12-11，
  
  Published：25 January 2025
- 稿件说明：
移动端阅览
程德强,姬广凯,张皓翔等.基于多粒度融合和跨尺度感知的跨模态行人重识别[J].通信学报,2025,46(01):108-123.

CHENG Deqiang,JI Guangkai,ZHANG Haoxiang,et al.Cross-modality person re-identification based on multi-granularity fusion and cross-scale perception[J].Journal on Communications,2025,46(01):108-123.
程德强,姬广凯,张皓翔等.基于多粒度融合和跨尺度感知的跨模态行人重识别[J].通信学报,2025,46(01):108-123. DOI： 10.11959/j.issn.1000-436x.2025019.

CHENG Deqiang,JI Guangkai,ZHANG Haoxiang,et al.Cross-modality person re-identification based on multi-granularity fusion and cross-scale perception[J].Journal on Communications,2025,46(01):108-123. DOI： 10.11959/j.issn.1000-436x.2025019.

摘要

提出一种基于多粒度融合和跨尺度感知的跨模态行人重识别网络，该网络能够有效提取行人图像特征并减少图像间的模态差异。首先，提出多尺度特征融合注意力机制并设计一种多粒度非局部融合框架，有效融合不同模态和不同尺度的图像特征；其次，提出一种跨尺度特征信息感知策略，该策略可有效降低因视角变化、行人背景变化等产生的无关噪声对行人判别的影响；最后，针对行人图像特征信息不足，设计并行空洞卷积残差模块，获取更为丰富的行人特征信息。将所提方法在2个标准公共数据集与当前先进的跨模态行人重识别方法比较。实验结果表明，所提方法在SYSU-MM01数据集的全搜索模式下的R-1和平均精度（mAP）分别达到75.9%和73.3%，在RegDB数据集的可见光到红外的搜索（VIS to IR）模式下的Rank-1和mAP分别达到93.7%和89.3%，优于所对比的方法，充分证实了所提方法的有效性。

Abstract

A cross-modality person re-identification network based on multi-granularity fusion and cross-scale perception was proposed

which could effectively extract person image features and reduce the modality discrepancies between images. Firstly

a multi-scale feature fusion attention mechanism was proposed

and a multi-granularity non-local fusion framework was designed to effectively integrate image features from different modalities and scales. Secondly

a cross-scale feature information perception strategy was proposed

which could effectively reduce the influence of irrelevant noise caused by the change of perspective and person background on person discrimination. Finally

in view of the lack of person image feature information

a parallel dilated convolution residual module was designed to obtain more abundant person feature information. The proposed method was compared with current state-of-the-art cross-modal person re-identification algorithms on two standard public datasets. Experimental results show that the Rank-1 and mAP of the proposed method reach 75.9% and 73.3%

respectively

in the all search mode of the SYSU-MM01 dataset

and 93.7% and 89.3% in the VIS to IR retrieval mode of the RegDB dataset

respectively

which is better than the compared methods

which fully confirms the effectiveness of the proposed method.

关键词

Keywords

references

罗浩 , 姜伟 , 范星 , 等 . 基于深度学习的行人重识别研究进展 [J ] . 自动化学报 , 2019 , 45 ( 11 ): 2032 - 2049 .

LUO H , JIANG W , FAN X , et al . A survey on deep learning based person re-identification [J ] . Acta Automatica Sinica , 2019 , 45 ( 11 ): 2032 - 2049 .

ZOU C , CHEN Z Q , CUI Z C , et al . Discrepant and multi-instance proxies for unsupervised person re-identification [C ] // Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2023 : 11024 - 11034 .

程德强 , 黄绩 , 寇旗旗 , 等 . 基于相关性得分的伪标签优化行人重识别 [J ] . 控制与决策 , 2024 , 39 ( 8 ): 2579 - 2587 .

CHENG D Q , HUANG J , KOU Q Q , et al . Person re-identification with pseudo label refinement based on correlation score [J ] . Control and Decision , 2024 , 39 ( 8 ): 2579 - 2587 .

寇旗旗 , 黄绩 , 程德强 , 等 . 基于语义融合的域内相似性分组行人重识别 [J ] . 通信学报 , 2022 , 43 ( 7 ): 153 - 162 .

KOU Q Q , HUANG J , CHENG D Q , et al . Person re-identification with intra-domain similarity grouping based on semantic fusion [J ] . Journal on Communications , 2022 , 43 ( 7 ): 153 - 162 .

GAO G W , SHAO H , WU F , et al . Leaning compact and representative features for cross-modality person re-identification [J ] . World Wide Web , 2022 , 25 ( 4 ): 1649 - 1666 .

WANG C D , ZHANG C , FENG Y J , et al . Learning visible thermal person re-identification via spatial dependence and dual-constraint loss [J ] . Entropy , 2022 , 24 ( 4 ): 443 .

HU W P , LIU B H , ZENG H T , et al . Adversarial decoupling and modality-invariant representation learning for visible-infrared person re-identification [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2022 , 32 ( 8 ): 5095 - 5109 .

YE M , LAN X Y , LI J W , et al . Hierarchical discriminative learning for visible thermal person re-identification [C ] // Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence . Palo Alto : AAAI Press , 2018 : 7501 - 7508 .

LU Y , WU Y , LIU B , et al . Cross-modality person re-identification with shared-specific feature transfer [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2020 : 4610 - 4617 .

HAO X , ZHAO S Y , YE M , et al . Cross-modality person re-identification via modality confusion and center aggregation [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 16403 - 16412 .

WU Q , DAI P Y , CHEN J , et al . Discover cross-modality nuances for visible-infrared person re-identification [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 4328 - 4337 . .

WANG G A , ZHANG T Z , CHENG J , et al . RGB-infrared cross-modality person re-identification via joint pixel and feature alignment [C ] // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2019 : 3622 - 3631 .

WANG Z X , WANG Z , ZHENG Y Q , et al . Learning to reduce dual-level discrepancy for infrared-visible person re-identification [C ] // Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2019 : 618 - 626 .

WANG G A , ZHANG T Z , YANG Y , et al . Cross-modality paired-images generation for RGB-infrared person re-identification [J ] . Neural Networks , 2020 , 34 ( 7 ): 12144 - 12151 .

YE M , SHEN J B , SHAO L . Visible-infrared person re-identification via homogeneous augmented tri-modal learning [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 16 : 728 - 739 .

FAN X , JIANG W , LUO H , et al . Modality-transfer generative adversarial network and dual-level unified latent representation for visible thermal Person re-identification [J ] . The Visual Computer , 2022 , 38 ( 1 ): 279 - 294 .

WANG Z J , LIU L , ZHANG H X . Dual-path image pair joint discrimination for visible–infrared person re-identification [J ] . Journal of Visual Communication and Image Representation , 2022 , 85 : 103512 .

ZHANG Z Y , JIANG S , HUANG C , et al . RGB-IR cross-modality person ReID based on teacher-student GAN model [J ] . Pattern Recognition Letters , 2021 , 150 : 155 - 161 .

CHOI S , LEE S M , KIM Y , et al . Hi-CMD: hierarchical cross-modality disentanglement for visible-infrared person re-identification [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2020 : 10254 - 10263 .

徐胜军 , 刘求缘 , 史亚 , 等 . 基于多样化局部注意力网络的行人重识别 [J ] . 电子与信息学报 , 2022 , 44 ( 1 ): 211 - 220 .

XU S J , LIU Q Y , SHI Y , et al . Person re-identification based on diversified local attention network [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 1 ): 211 - 220 .

WOO S , PARK J , LEE J Y , et al . CBAM: convolutional block attention module [C ] // 2018 Proceedings of the European conference on computer vision . Berlin : Springer , 2018 : 3 - 19 .

JIA M X , SUN Y F , ZHAI Y P , et al . Semi-attention partition for occluded person re-identification [C ] // Proceedings of the 37th AAAI Conference on Artificial Intelligence . Palo Alto : AAAI Press , 2023 : 998 - 1006 .

YANG K W , YANG J W , TIAN X M . Learning multi-granularity features from multi-granularity regions for person re-identification [J ] . Neurocomputing , 2021 , 432 : 206 - 215 .

王新兴 , 蓝凯 , 李伟 . 注意力空洞卷积无人机遥感多目标检测方法研究 [J ] . 测绘与空间地理信息 , 2023 , 46 ( 12 ): 136 - 139 .

WANG X X , LAN K , LI W . Research on multi-target detection method of UAV remote sensing based on attention hole convolution [J ] . Geomatics and Spatial Information Technology , 2023 , 46 ( 12 ): 136 - 139 .

陈清江 , 顾媛 . 基于多尺度深度可分离卷积的低照度图像增强算法 [J ] . 计算机工程与科学 , 2023 , 45 ( 10 ): 1830 - 1837 .

CHEN Q J , GU Y . A low-light image enhancement algorithm based on multi-scale depthwise separable convolution [J ] . Computer Engineering and Science , 2023 , 45 ( 10 ): 1830 - 1837 .

李庆 , 王宏健 , 李本银 , 等 . 基于改进的IIE-SegNet的快速图像语义分割方法 [J ] . 哈尔滨工程大学学报 , 2024 , 45 ( 2 ): 314 - 323 .

LI Q , WANG H J , LI B Y , et al . Fast image semantic segmentation method based on improved IIE-SegNet [J ] . Journal of Harbin Engineering University , 2024 , 45 ( 2 ): 314 - 323 .

WU A C , ZHENG W S , YU H X , et al . RGB-infrared cross-modality person re-identification [C ] // Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2017 : 5390 - 5399 .

NGUYEN D , HONG H , KIM K , et al . Person recognition system based on a combination of body images from visible light and thermal cameras [J ] . Sensors , 2017 , 17 ( 3 ): 605 - 634 .

WEI Z Y , YANG X , WANG N N , et al . Syncretic modality collaborative learning for visible infrared person re-identification [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 225 - 234 .

YE M , RUAN W J , DU B , et al . Channel augmented joint learning for visible-infrared recognition [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 13547 - 13556 .

LING Y G , ZHONG Z , LUO Z M , et al . Cross-modality earth mover’s distance for visible thermal person re-identification [C ] // Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence . Palo Alto : AAAI Press , 2023 : 1631 - 1639 .

FENG J W , WU A C , ZHENG W S . Shape-erased feature learning for visible-infrared person re-identification [C ] // Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2023 : 22752 - 22761 .

ZHANG Y K , WANG H Z . Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification [C ] // Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2023 : 2153 - 2162 .

PARK H , LEE S , LEE J , et al . Learning by aligning: visible-infrared person re-identification using cross-modal correspondences [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 12026 - 12035 .

CHEN Y , WAN L , LI Z H , et al . Neural feature search for RGB-infrared person re-identification [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 587 - 597 .

YANG M X , HUANG Z Y , HU P , et al . Learning with twin noisy labels for visible-infrared person re-identification [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 14288 - 14297 .

FU C Y , HU Y B , WU X , et al . CM-NAS: cross-modality neural architecture search for visible-infrared person re-identification [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 11803 - 11812 .

ZHANG Q , LAI C Z , LIU J N , et al . FMCNet: feature-level modality compensation for visible-infrared person re-identification [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 7339 - 7348 .

LI D G , WEI X , HONG X P , et al . Infrared-visible cross-modal person re-identification with an X modality [C ] // Proceedings of the 34th AAAI Conference on Artificial Intelligence . Palo Alto : AAAI Press , 2020 : 4610 - 4617 .

YE M , SHEN J B , CRANDALL D J , et al . Dynamic dual-attentive aggregation learning for visible-infrared person re-identification [C ] // Proceedings of the 16th European Conference . Berlin : Springer , 2020 : 229 - 247 .

ZHANG Y K , YAN Y , LU Y , et al . Towards a unified middle modality learning for visible-infrared person re-identification [C ] // Proceedings of the 29th ACM International Conference on Multimedia . New York : ACM Press , 2021 : 788 - 796 .

SUN H Z , LIU J , ZHANG Z Z , et al . Not all pixels are matched: dense contrastive learning for cross-modality person re-identification [C ] // Proceedings of the 30th ACM International Conference on Multimedia . New York : ACM Press , 2022 : 5333 - 5341 .

LIU J L , SUN Y F , ZHU F , et al . Learning memory-augmented unidirectional metrics for cross-modality person re-identification [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 19344 - 19353 .

ZHANG Y K , YAN Y , LI J , et al . MRCN: a novel modality restitution and compensation network for visible-infrared person re-identification [C ] // Proceedings of the 37th AAAI Conference on Artificial Intelligence . Palo Alto : AAAI Press , 2023 : 3498 - 3506 .

MOLCHANOV P , TYREE S , KARRAS T , et al . Pruning convolutional neural networks for resource efficient inference [J ] . arXiv Preprint , arXiv: 1611.06440 , 2016 .

ZHENG X T , CHEN X M , LU X Q . Visible-infrared person re-identification via partially interactive collaboration [J ] . IEEE Transactions on Image Processing , 2022 , 31 : 6951 - 6963 .

JAMBIGI C , RAWAL R , CHAKRABORTY A . Mmd-reid: a simple but effective solution for visible-thermal person rei [J ] . arXiv Preprint , arXiv: 2111.05059 , 2021 .

ZHANG H D , CHENG S L , DU A Y . Multi-stage auxiliary learning for visible-infrared person re-identification [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2024 , 34 ( 11 ): 12032 - 12047 .

SELVARAJU R R , COGSWELL M , DAS A , et al . Grad-CAM: visual explanations from deep networks via gradient-based localization [C ] // Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2017 : 618 - 626 .

Views

1993

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Encrypted traffic classification method based on parallel traffic graph and graph neural network

Person re-identification with intra-domain similarity grouping based on semantic fusion

Multi-scale aware dual path network for face detection in resource-constrained edge computing environment

Non-stationary signal combined analysis based fault diagnosis method

Related Author

An Yishuai

Yu Yihan

FU Yu

Liu Taotao

Ji HUANG

Jianying ZHANG

Yunlong LI

Deqiang CHENG

Related Institution

Department of Information Security, Naval University of Engineering

Department of Operational Operations and Planning, Naval University of Engineering

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

College of Computer and Information Engineering, Hohai University

School of Engineering,Kyushu Institute of Technology,Kyushu

AI问答

⁰