摘 要

在雨天特别是在暴雨天，雨纹、雨滴会附着在户外镜头表面，导致采集到的图像出现模糊不清的现象，严重影响其应用性能。图像去雨任务的目的是去除带雨图像中的雨痕并为其他视觉任务提供更好的图像数据，减少低质量图像对模型预测的干扰。从传统的图像去雨到现在的深度学习图像去雨，去雨算法的研究取得了很大的进步，但目前的图像去雨算法仍然存在一些问题，例如去雨后会导致背景信息损失、模型去雨速度慢、去雨后图像亮度低、暗处细节信息模糊等问题。针对上述问题，本文进行了深入的研究分析，具体工作如下：

(1)针对现有去雨算法去雨后背景细节信息丢失的问题，本文设计了一种多级残差Transformer去雨网络。该算法通过将通道和空间混合注意力与密集残差网络相结合，合并扩张卷积，可加强不同层不同尺度雨痕特征信息的提取，提取更准确的雨痕特征。其中，Transformer结构可计算雨纹全局特征的关联性，有效强化上下文雨痕细节特征的提取。实验结果表明，该算法在去除雨痕的同时，最大程度的保留了背景细节信息的完整，该算法与其他去雨算法相比，在对所有数据集的去雨评价指标进行平均后，其PSNR和SSIM分别提升了1.43dB、0.047，达到了更好的去雨效果。

(2)针对去雨算法去雨速度慢，时效性差的问题，本文设计了一种基于Transformer的多分支递归单幅图像去雨网络。该算法通过Swin Transformer中的偏移窗口机制，降低计算量，设计的多尺度偏移窗口可提取不同层的雨纹特征，通过递归调用该模块可使得自注意力机制之间权重共享，该模型能有效的去除雨痕，缩短运行时间。实验结果表明，与对比算法相比，本算法在单幅图像平均处理时间上缩短0.19秒。

(3)针对去雨后图像亮度低、暗处细节信息模糊的问题，本文设计了一种基于图像子空间的低照度图像增强算法，通过提出新的损失函数控制图层分解模块中的光照强度，有效提升暗处边界光照强度，同时在反射率恢复模块中提出的多尺度光照注意机制模块，可收集更多退化信息，增强暗处细节信息的恢复。实验结果表明，本章增强算法可有效提高去雨后图像的亮度，恢复暗处细节，联合去雨增强算法与对比算法相比，其PSNR和SSIM分别提升了1.08dB、0.043，本章算法可进一步提高去雨后图像的质量，提升人眼视觉观感。

                                       ABSTRACT

Rain streaks and raindrops will adhere to the surface of the outdoor lens in rainy days, especially in heavy rainy days, which causes blur in the collected images and seriously affects its application performance. The task of the image deraining is to remove rain streaks in rainy images and provide better image data for other vision tasks, reducing the interference of low-quality images on model predictions.From the traditional image deraining to the current deep learning image deraining, the research of deraining algorithm has made great progress, however, the current image deraining algorithm still exists some problems, such as the loss of background information after deraining, the slow speed of the pattern , and the low image brightness after deraining and the blurred information in the dark details. Aiming at the above problems, this paper conducts in-depth research and analysis, and the specific work is as follows:

(1)Aiming at the problem of the loss of background information after deraining of the existing deraining algorithm, Multi-stage Residual Transformer Deraining Network (MSRTNet) is desingned in this paper.The algorithm can strengthen the extraction of rain mark feature information at different layers and scales and extract more accurate rain mark features by combining channel and space mixed attention with dense residual network and merging expanded convolution. Among this, the Transformer structure can calculate the correlation of the global features of the rain streaks, effectively strengthening the extraction of the detailed features of the contextual rain marks.As the experimental results show, the algorithm presreves the integrity of the background details to the greatest extent while removing the rain marks. Compared with other deraining algorithms,the algorithm in this chapter increases its PSNR and SSIM by 1.43dB and 0.047 respectively after averaging the deraining evaluation indexes of all data sets, which achieves a better deraining effect.

(2)Aiming at the problems of slow deraining speed and poor timeliness of the deraining algorithms, a multi-branch recursive single image deraining network based on Transformer is designed. By using the offset window mechanism in the Swin Transformer to reduce the amount of calculation, The algorithm can extract the rain streaks features of different layers with the designed multi-scale offset window, and shares the weights between the self-attention mechanisms through recursive call to the module. Hence the model can remove the rain streaks effectively and reduce the  time of program operation. Experimental results show that compared with the comparison algorithm, the average processing time of a single image is shortened by 0.19 seconds.

(3)Aiming at the problem of the low image brightness after deraining and the blurred information in the dark details, a low illuminance image enhancement algorithm based on image subspace is designed. By proposing a new loss function to control the light intensity in the layer decomposition module, the light intensity of the dark boundary is effectively improved. At the same time, the multi-scale light attention mechanism module proposed in the reflectivity restoration module can collect more degradation information and enhance the restoration of the detailed information in the dark area. The experimental results show that the enhancement algorithm in this chapter can effectively improve the image brightness after deraining and restore the detailed information. Besides, compared with the deraining enhancement algorithm and the comparison algorithm, the PSNR and SSIM are increased by 1.08dB and 0.043respectively. Therefore, the algorithm in this chapter can further improve the image quality after deraining and the visual perception.

参考文献

[1] Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2016, 28: 1-10.

[2] 王春波, 张卫东, 张文渊, 等.复杂交通环境中车辆的视觉检测[J]. 上海交通大学学报, 2000, 34(12): 1680-1682.

[3] 陈一民, 陆蓉蓉, 邹一波, 等. Branch-Activated Multi-Domain Convolutional Neural Network for Visual Tracking[J]. Journal of Shanghai Jiaotong University Science, 2018, 23(03): 360-367.

[4] 陈舒曼, 陈玮, 尹钟.单幅图像去雨算法研究现状及展望[J]. 计算机应用研究, 2022, 39(1): 9-17.

[5] Hong W, Yi W, Ming L, Qian Z, De M. Survey on rain removal from videos or a single image[J]. Science China (Information Sciences), 2022, 65(01): 72-94.

[6] Kang L W, Lin C W, Fu Y H. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition[J]. IEEE Transactions on Image Processing, 2011, 21(4): 1742-1755.

[7] Yu L, Yong X, Hui J. Removing Rain from a Single Image via Discriminative Sparse Coding[C]//International Conference on Computer Vision. Santiago: IEEE, 2015: 3397-3405.

[8] Li Y, Tan R T, Guo X, et al. Rain Streak Removal Using Layer Priors[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2736-2744.

[9] Yeh C H, Lin C Y, Muchtar K, et al. Rain streak removal based on non-negative matrix factorization[J]. Multimedia Tools and Applications, 2018, 77(15): 20001-20020.

[10] Yang W, Tan R T, Feng J, et al. Deep Joint Rain Detection and Removal from a Single Image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1357-1366.

[11] Fu X, Huang J, Zeng D, et al. Removing Rain from Single Images via a Deep Detail Network[C]//Conference on Computer Vision & Pattern Recognition. Honolulu: IEEE, 2017:1715-1723.

[12] Fu X, Huang J, Ding X, et al. Clearing the Skies: A Deep Network Architecture for Single-Image [J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956.

[13] Zhang H, Patel V M. Density-aware single image de-raining using a multi-stream dense network[J]. Proceedings of the IEEE conference on computer vision and pattern recognition,

2018: 695-704.

[14] Li X, Wu J, Lin Z, et al. Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 262-277.

[15] Yasarla R, Patel V M. Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining[J]. Conference on Computer Vision and Pattern Recognition, 2019: 8397-8406.

[16] Wei Y, Zhang Z, Zhang H, et al. A coarse-to-fine multi-stream hybrid deraining network for single image deraining[C]//2019 IEEE international conference on data mining. Beijing: IEEE, 2019: 628-637.

[17] Ren D, Zuo W, Hu Q, et al. Progressive image deraining networks: A better and simpler baseline[J]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 3937-3946.

[18] Wei W, Meng D, Zhao Q, et al. Semi-supervised Transfer Learning for Image Rain Removal[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3872-3881.

[19] Jiang K, Wang Z, Yi P, et al. Multi-scale progressive fusion network for single image deraining[J]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020: 8343-8352.

[20] Zamir S W, Arora A, Khan S, et al. Multi-stage progressive image restoration[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Nashville: IEEE, 2021: 14816-14826.

[21] Chen L, Lu X, Zhang J, et al. HINet: Half Instance Normalization Network for Image Restoration[J]. IEEE/CVF conference on computer vision and pattern recognition, 2021: 182-192.

[22] Zhang H, Xie Q, Lu B, et al. Dual attention residual group networks for single image deraining[J]. Digital Signal Processing, 2021, 116(2):103106.

[23] Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers[C]// European conference on computer vision. Cham: Springer, 2020: 213-229.

[24] Xu Y, Jin T, Xu Y, et al. Transformer Image Recognition System Based on Deep Learning[C]//International Conference on Systems and Informatics (ICSAI). Shanghai: IEEE, 2019:595-599.

[25] Cao S, Liu L, Zhao L, et al. Deep Feature Interactive Aggregation Network for Single Image Deraining[J]. IEEE Access, 2022, 10: 3872-3879.

[26] Lu L, Zhou Y, Panetta K, et al. Comparative study of histogram equalization algorithms for image enhancement[J]. Proceedings of SPIE-The International Society for Optical Engineering, 2010, 7708.

[27] Wu Y, Shi J. Image Enhancement in Non-Subsampled Contourlet Transform Domain Based on Multi-Scale Retinex[J]. Acta Optica Sinica, 2015, 35(3):79-88.

[28] Shen L, Yue Z, Feng F, et al. MSR-net:Low-light Image Enhancement Using Deep Convolutional Network[J]. 2017. arXiv.1711.02488.

[29] Li C, Guo J, Porikli F, et al. LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement[J]. Pattern recognition letters, 2018, 104(MAR.1):15-22.

[30] Jiang L, Jing Y, Hu S, et al. Deep Refinement Network for Natural Low-Light Image Enhancement in Symmetric Pathways[J]. Symmetry, 2018, 10(10).

[31] Liu Mi, Chen J, Han X. Research on Retinex Algorithm Combining with Attention Mechanism for Image Enhancement[J]. Electronics,2022,11(22).

[32] Du S, Zhao M, Liu Y, et al. Low-light image enhancement and denoising via dual-constrained Retinex model[J]. Applied Mathematical Modelling, 2023, 116:1-15.

[33] Yu Y, Qin C. An End-to-End Underwater-Image-Enhancement Framework Based on Fractional Integral Retinex and Unsupervised Autoencoder[J]. Fractal and Fractional, 2023, 7(1).

[34] Guo X, Yu L, Ling H. LIME: Low-light Image Enhancement via Illumination Map Estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2) :982-993.

[35] Ying Z, Ge L, Ren Y, et al. A New Low-Light Image Enhancement Algorithm Using Camera Response Model[C]//2017 IEEE International Conference on Computer Vision Workshops. Venice: IEEE, 2017: 3015-3022.

[36] Wei C, Wang W, Yang W, et al. Deep Retinex Decomposition for Low Light Enhancement-[J]. 2018. arXiv.1808.04560.

[37] Lv F, Li Y, Lu F. Attention guided low-light image enhancement with a large scale low-light simulation dataset[J]. International Journal of Computer Vision, 2021, 129(7): 2175-2193.

[38] Jiang Y, Gong X, Liu D, et al. Enlightengan: Deep light enhancement without paired supervision[J]. IEEE transactions on image processing, 2021, 30: 2340-2349.

[39] Luo Y, Xu Y, and Ji H. Removing Rain from a Single Image via Discriminative Sparse Coding[C]//International Conference on Computer. Santiago: IEEE, 2015: 3397-3405.

[40] Liu J, Yang W, Yang S, et al. deep joint recurrent rain removal and reconstruction in videos[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City: IEEE, 2018: 3233-3242.

[41] Hu X, Fu C W, Zhu L, et al. Depth-attentional features for single-image rain removal[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 8014-8023.

[42] 易振通,吴瑰,官端正,等.轻量化卷积神经网络的研究综述[J].工业控制计算机2022, 35(10): 109-114.

[43] Niu Z, Zhong G, Yu H. A review on the attention mechanism of deep learning [J]. Neurocomputing, 2021, 452(10): 48-62.

[44] 王红,史金钏,张志伟.基于注意力机制的LSTM的语义关系抽取[J].计算机应用研究, 2018, 35(05): 1417-1420.

[45] 唐海桃,薛嘉宾,韩纪庆.一种多尺度前向注意力模型的语音识别方法[J].电子学报, 2020, 48(07): 1255-1260.

[46] Qiang H, Li C, Pan L, Shi Z, Yan L. A Pixel–Channel Hybrid Attention Model for Image Processing[J]. Tsinghua Science and Technology, 2022, 27(05): 804-816.

[47] Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[J]. Advances in neural information processing systems, 2017, 30.

[48] Lin Z, Feng M, Santos C N D, et al. A Structured Self-attentive Sentence Embedding[J]. 2017. arXiv.1703.03130.

[49] Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment

[J]. Electronicsletters, 2008, 44(13): 800-801.

[50] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, et al. Image Quality Assessment: From Error Visibility to Structural Similarity[J]. IEEE Trans Image Process, 2004, 13(4): 600-612.

[51] Mittal A, Soundararajan R, Bovik A C. Making a completely blind image quality analyzer [J]. IEEE Signal Processing Letters, 2013, 20(3): 209-212.

[52] Woo S, Park J, Lee J Y, et al. CBAM: Convolutional Block Attention Module[J]. Proceedings of the European Conference on Computer Vision, 2018: 3-19.

[53] Huang G, Liu Z, Maaten L V D, et al. Densely Connected Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

[54] Yu F, Kolten V, Funkhouser T. Dilated residual networks[C]//Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 636-644.

[55] Wang W, Xie E, Li X et al. PVTV2:Improved baselines with Pyramid Vision Transformer[J].

Compitational Visual Media, 2022:1-10.

[56] Lai W S, Huang J B, Ahuja N, et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution[J]. IEEE Computer Society, 2017: 5835-5843.

[57] Wang G, Sun C, Sowmya A. Attentive Feature Refinement Network for Single Rainy Image Restoration[J]. IEEE Transactions on Image Processing, 2021, 30:3734-3747.

[58] Li S, Araujo I B, Ren W, et al. Single Image Deraining: A Comprehensive Benchmark Analysis[J]. Conference on Computer Vision and Pattern Recognition, 2020: 3833-3842.

[59] Wang T, Yang X, Xu K, et al. Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset[J]. Conference on Computer Vision and Pattern Recognition, 2019: 12262-12271.

[60] Liu Z, Lin Y, Cao Y, et al. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows[J]. International Conference on Computer Vision, 2021: 9992-10002.