特征分离和非阴影信息引导的阴影去除网络

黄颖; 房少杰; 程彬; 姜茂; 钱鹰

doi:10.11959/j.issn.1000-436x.2024099

您当前的位置：

首页 >

文章列表页 >

特征分离和非阴影信息引导的阴影去除网络

学术论文 | 更新时间：2024-06-24

- 特征分离和非阴影信息引导的阴影去除网络
- Feature separation and non-shadow information-guided shadow removal network
- 通信学报 2024年45卷第5期页码：178-190
- 作者机构：
  
  1.重庆邮电大学软件工程学院，重庆 400065
  2.重庆邮电大学计算机科学与技术学院，重庆 400065
- 作者简介：
  
  [ "黄颖（1978- ），男，湖南岳阳人，博士，重庆邮电大学副教授，主要研究方向为图像处理、图像融合、图像质量评估、计算成像、智能信息处理与模式识别等。" ]
  [ "房少杰（1999- ），男，四川德阳人，重庆邮电大学硕士生，主要研究方向为图像处理、深度学习和阴影去除等。" ]
  [ "程彬（1999- ），男，江西上饶人，重庆邮电大学硕士生，主要研究方向为图像增强、深度学习等。" ]
  姜茂（1999- ），女，重庆人，重庆邮电大学硕士生，主要研究方向为图像处理、深度学习和图像去噪等。
  钱鹰（1968- ），男，上海人，博士，重庆邮电大学教授、硕士生导师，主要研究方向为数字图像处理、机器视觉与图像认知、医学成像和信息处理等。
- 基金信息：
  
  国家自然科学基金重点项目(62331008)
- DOI：10.11959/j.issn.1000-436x.2024099
  中图分类号： TP391
- 收稿日期：2024-01-03，
  
  修回日期：2024-05-06，
  
  纸质出版日期：2024-05-30
- 稿件说明：
移动端阅览
黄颖,房少杰,程彬等.特征分离和非阴影信息引导的阴影去除网络[J].通信学报,2024,45(05):178-190.

HUANG Ying,FANG Shaojie,CHENG Bin,et al.Feature separation and non-shadow information-guided shadow removal network[J].Journal on Communications,2024,45(05):178-190.
黄颖,房少杰,程彬等.特征分离和非阴影信息引导的阴影去除网络[J].通信学报,2024,45(05):178-190. DOI： 10.11959/j.issn.1000-436x.2024099.

HUANG Ying,FANG Shaojie,CHENG Bin,et al.Feature separation and non-shadow information-guided shadow removal network[J].Journal on Communications,2024,45(05):178-190. DOI： 10.11959/j.issn.1000-436x.2024099.

摘要

为了解决现有阴影去除方法中存在的性能瓶颈以及去除结果产生的色差问题，构建了一个特征分离和非阴影信息引导的阴影去除网络（FSNIG-ShadowNet）。在分离和重建阶段，利用自重建监督将阴影图像分离成直射光和环境光两部分，对光照类型和反射率进行特征解耦分离，设计解码器对分离的特征进行重新耦合以获得无阴影图像。在细化阶段，该网络关注阴影和非阴影的邻接区域，设计局部区域自适应归一化模块将局部非阴影区域颜色先验传递至阴影区域以减少两区域间的色差。实验结果表明，所提FSNIG-ShadowNet与其他优秀的方法相比取得了较有竞争力的结果。

Abstract

To tackle the performance bottlenecks and color deviation issues stemming from current shadow removal methods

a feature separation and non-shadow information guided shadow removal network (FSNIG-ShadowNet) was constructed. In the separation and reconstruction stage

the shadow image was separated into direct light and ambient light using self-reconstruction supervision

with decoupling of lighting types and reflectance. Subsequently

a decoder was employed to re-couple the separated features to yield shadow-free images. In the refinement stage

the network focused on the adjacent regions of shadow and non-shadow

incorporating a local region adaptive normalization module to transfer the color priors of local non-shadow region to shadow regions for mitigating color deviation between the two regions. Experimental results demonstrate that the proposed FSNIG-ShadowNet achieves competitive results compared to other state-of-the-art methods.

关键词

Keywords

references

LALONDE J F , EFROS A A , NARASIMHAN S G . Estimating the natural illumination conditions from a single outdoor image [J ] . International Journal of Computer Vision , 2012 , 98 ( 2 ): 123 - 145 .

WEHRWEIN S , BALA K , SNAVELY N . Shadow detection and sun direction in photo collections [C ] // Proceedings of the 2015 International Conference on 3D Vision . Piscataway : IEEE Press , 2015 : 460 - 468 .

OKABE T , SATO I , SATO Y . Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions [C ] // Proceedings of the 2009 IEEE 12th International Conference on Computer Vision . Piscataway : IEEE Press , 2009 : 1693 - 1700 .

LI Z Q , SNAVELY N . Learning intrinsic image decomposition from watching the world [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 9039 - 9048 .

LONG C J , HUA G . Correlational Gaussian processes for cross-domain visual recognition [C ] // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2017 : 4932 - 4940 .

MIKIC I , COSMAN P C , KOGUT G T , et al . Moving shadow and object detection in traffic scenes [C ] // Proceedings of the 15th International Conference on Pattern Recognition . Piscataway : IEEE Press , 2000 : 321 - 324 .

ZHANG Q Z , HU S T , SUN J C , et al . On adversarial robustness of trajectory prediction for autonomous vehicles [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 15138 - 15147 .

FINLAYSON G D , HORDLEY S D , LU C , et al . On the removal of shadows from images [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2006 , 28 ( 1 ): 59 - 68 .

LIU F , GLEICHER M . Texture-consistent shadow removal [C ] // European Conference on Computer Vision . Berlin : Springer , 2008 : 437 - 450 .

GONG H , COSKER D P . Interactive shadow removal and ground truth for variable scene categories [C ] // Proceedings of the British Machine Vision Conference (BMVC) . Durham : British Machine Vision Association , 2014 : 1 - 11 .

刘艳丽 , 石俊 , 张严辞 . 一种单幅室外图像的阴影去除算法 [J ] . 软件学报 , 2012 , 23 ( 2 ): 168 - 175 .

LIU Y L , SHI J , ZHANG Y C . Shadow removal based on single outdoor image [J ] . Journal of Software , 2012 , 23 ( 2 ): 168 - 175 .

ARBEL E , HEL-OR H . Shadow removal using intensity surfaces and texture anchor points [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 6 ): 1202 - 1216 .

WU T P , TANG C K . A Bayesian approach for shadow extraction from a single image [C ] // Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2005 : 480 - 487 .

GUO R Q , DAI Q Y , HOIEM D . Single-image shadow detection and removal using paired regions [C ] // Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2011 : 2033 - 2040 .

QU L Q , TIAN J D , HE S F , et al . DeshadowNet: a multi-context embedding deep network for shadow removal [C ] // Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2017 : 2308 - 2316 .

WANG J F , LI X , YANG J . Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 1788 - 1797 .

HU X W , FU C W , ZHU L , et al . Direction-aware spatial context features for shadow detection and removal [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 11 ): 2795 - 2808 .

CHEN Z P , LONG C J , ZHANG L , et al . CANet: a context-aware network for shadow removal [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 4723 - 4732 .

WAN J , YIN H , WU Z Y , et al . Style-guided shadow removal [C ] // European Conference on Computer Vision . Berlin : Springer , 2022 : 361 - 378 .

ZHU Y R , HUANG J , FU X Y , et al . Bijective mapping network for shadow removal [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2022 : 5617 - 5626 .

GUO L Q , WANG C , YANG W H , et al . ShadowDiffusion: when degradation prior meets diffusion model for shadow removal [C ] // Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2023 : 14049 - 14058 .

HU X W , JIANG Y T , FU C W , et al . Mask-ShadowGAN: learning to remove shadows from unpaired data [C ] // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2019 : 2472 - 2481 .

JIN Y Y , SHARMA A , TAN R T . DC-ShadowNet: single-image hard and soft shadow removal using unsupervised domain-classifier guided network [C ] // Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2021 : 5007 - 5016 .

LIU Z H , YIN H , WU X Y , et al . From shadow generation to shadow removal [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 4925 - 4934 .

LE H , SAMARAS D . From shadow segmentation to shadow removal [C ] // European Conference on Computer Vision . Berlin : Springer , 2020 : 264 - 281 .

LE H , SAMARAS D . Shadow removal via shadow image decomposition [C ] // Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2019 : 8577 - 8586 .

SHOR Y , LISCHINSKI D . The shadow meets the mask: pyramid-based shadow removal [J ] . Computer Graphics Forum , 2008 , 27 ( 2 ): 577 - 586 .

YU Q H , ZHENG N S , HUANG J , et al . CNSNet: a cleanness-navigated-shadow network for shadow removal [C ] // European Conference on Computer Vision . Berlin : Springer , 2023 : 221 - 238 .

吴文 , 万毅 . 基于低尺度细节恢复的单幅图像阴影去除方法 [J ] . 电子学报 , 2020 , 48 ( 7 ): 1293 - 1302 .

WU W , WAN Y . Single image shadow removal using low-scale detail recovering [J ] . Acta Electronica Sinica , 2020 , 48 ( 7 ): 1293 - 1302 .

LIU Z H , YIN H , MI Y , et al . Shadow removal by a lightness-guided network with training on unpaired data [J ] . IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society , 2021 , 30 : 1853 - 1865 .

王潇 , 尧思远 , 代朋纹 , 等 . 基于阴影和周边区域色差的光照条件和光谱反射率计算方法 [J ] . 电子学报 , 2023 , 51 ( 8 ): 2098 - 2109 .

WANG X , YAO S Y , DAI P W , et al . Estimating illumination condition and spectral reflectance based on the color variation between shadowed region and its surrounding regions [J ] . Acta Electronica Sinica , 2023 , 51 ( 8 ): 2098 - 2109 .

IOFFE S , SZEGEDY C . Batch normalization: accelerating deep network training by reducing internal covariate shift [C ] // Proceedings of the 32nd International Conference on International Conference on Machine Learning . New York : ACM Press , 2015 : 448 - 456 .

HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization [C ] // Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2017 : 1510 - 1519 .

XU J , SUN X , ZHANG Z , et al . Understanding and improving layer normalization [C ] // Proceedings of the Annual Conference on Neural Information Processing Systems . Massachusetts : MIT Press , 2019 : 4383 - 4393 .

YU T , GUO Z Y , JIN X , et al . Region normalization for image inpainting [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 12733 - 12740 .

LING J , XUE H , SONG L , et al . Region-aware adaptive instance normalization for image harmonization [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 9357 - 9366 .

ZHANG L , LONG C J , ZHANG X L , et al . RIS-GAN: explore residual and illumination with generative adversarial networks for shadow removal [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 12829 - 12836 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

VICENTE T F Y , HOU L , YU C P , et al . Large-scale training of shadow detectors with noisily-annotated shadow examples [C ] // European Conference on Computer Vision . Berlin : Springer , 2016 : 816 - 832 .

ZHU J J , SAMUEL K G G , MASOOD S Z , et al . Learning to recognize shadows in monochromatic natural images [C ] // Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2010 : 223 - 230 .

CUN X D , PUN C M , SHI C . Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 10680 - 10687 .

FU L , ZHOU C Q , GUO Q , et al . Auto-exposure fusion for single-image shadow removal [C ] // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2021 : 10566 - 10575 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据