图像在人类信息交流日渐频繁的今天扮演着高效信息传递者的角色，而高质量图像包含了更多更丰富的信息，能够满足人们对于更高效信息交流的需求。但是带来了硬件设备老旧引起的难以获取、传输和应用高质量图像等问题，因此需要一种通过算法得到高分辨率图像的技术。超分辨率重建技术无需更换高质量成像设备，仅使用深度学习模型通过低分辨图像计算得到分辨率更高、细节更丰富、轮廓更清晰的高质量图像，既满足了人类对高质量图片的需求，也不会额外增加经济成本，因此得到了学者和业界的广泛关注。因此，本文将研究改进的重点安排在基于深度学习的图像超分辨率重建算法。

本文围绕使用深度学习的神经网络在超分辨重建问题进行研究，针对传统网络模型无法充分利用深层特征和浅层特征，不能在训练中有效保存长期序列中的信息，影响重建图片质量等问题。改进了一种基于循环嵌入沙漏网络的超分辨率重建架构，其将门限循环单元嵌入到沙漏网络中进行特征提取，利用沙漏网络可以高效的融合深层特征与浅层特征和门限循环单元可以保存长期序列中信息的特点，自主迭代保存高、低分辨率图像之间存在的对重建高分辨率图像有很大增益的非线性映射关系，保持学习中图像特征信息的完整性。实验结果表明，该算法重建的图像能够有效地去除伪影，提升主观视觉上的感受，与其他方法相比峰值信噪比值平均提高了0.20，结构相似性值提高了0.023。

针对网络模型单一、重建图像细节纹理模糊等问题，本文改进了一种基于多通道合并卷积的超分辨率重建神经网络架构，通过建立不同的子网解决网络模型单一的问题，其中包括将深层特征与浅层特征融合的沙漏子网，保存网络长期相关性的门限循环子网，和具有不同卷积核的多尺度卷积子网。将原始图像扩充通道后分别输入到这三个子网络中，通过合并子网的特征通道扩充模型，通过跳跃连接收集底层像素特征。在模型最后中进行亚像素卷积，生成高分辨率图像。通过实验结证明，本章方法优于传统方法，与第三章方法相比峰值信噪比值平均提高了0.13，结构相似性值提高了0.012。

Images play the role of an efficient information transmitter in today's increasingly frequent human information exchanges, and high-quality images contain more and richer information, which can meet people's needs for more efficient information exchange. However, it has brought about problems such as difficulty in obtaining, transmitting and applying high-quality images caused by the old hardware equipment. Therefore, a technology for obtaining high-resolution images through algorithms is needed. Super-resolution reconstruction technology does not need to replace high-quality imaging equipment, and only uses deep learning models to obtain high-quality images with higher resolution, richer details, and clearer outlines through low-resolution image calculations, which not only meets human needs for high-quality images , It will not increase the economic cost, so it has received widespread attention from scholars and the industry. Therefore, this article will focus on the research and improvement of the image super-resolution reconstruction algorithm based on deep learning.

This article focuses on the use of deep learning neural networks in the super-resolution reconstruction problem. Aiming at the problems that traditional network models cannot make full use of deep and shallow features, they cannot effectively save information in long-term sequences during training, and affect the quality of reconstructed pictures. Improved a super-resolution reconstruction architecture based on the cyclic embedded hourglass network, which embeds the threshold cycle unit into the hourglass network for feature extraction. The hourglass network can efficiently integrate deep features with shallow features and the threshold cycle unit can be stored for a long time. The characteristics of the information in the sequence, independently iteratively save the non-linear mapping relationship between the high-resolution and low-resolution images that has a great gain in the reconstruction of the high-resolution image, and maintain the integrity of the image feature information in the learning. Experimental results show that the image reconstructed by this algorithm can effectively remove artifacts and improve subjective visual perception. Compared with other methods, the peak signal-to-noise ratio has increased by 0.20 on average, and the structural similarity value has increased by 0.023.

Aiming at the problems of single network model and blurred texture of reconstructed image, this paper improves a super-resolution reconstruction neural network architecture based on multi-channel merged convolution, and solves the single problem of network model by establishing different subnets, including deep layer An hourglass subnet with features fused with shallow features, a threshold loop subnet that preserves the long-term relevance of the network, and a multi-scale convolution subnet with different convolution kernels. The original image expansion channels are input into these three sub-networks, the model is expanded by merging the feature channels of the sub-networks, and the bottom-layer pixel features are collected through jump connections. At the end of the model, sub-pixel convolution is performed to generate a high-resolution image. The experimental results prove that the method in this chapter is better than the traditional method. Compared with the method in Chapter 3, the peak signal-to-noise ratio value is increased by 0.13 on average, and the structural similarity value is increased by 0.012.

[1] Zhou Wang, A.C. Bovik, et al. Image quality assessment: from error visibility to structural similarity [J]. IEEE Transactions on Image Processing, 2004: 600-612.

[2] 章毓晋. 图象工程: 图像处理[M]. 北京: 清华大学出版社, 2012.

[3] Lin Zhang, Lei Zhang, Xuanqin Mou, et al. FSIM: A feature similarity index for image quality assessment [J]. IEEE Transactions on Image Processing, 2011: 2378-2386.

[4] 苏衡, 周杰, 张志浩.超分辨率图像重建方法综述[J]. 自动化学报2013,39(8):1202-1213.

[5] Harris J L. Diffraction and Resolving Power [J]. Journal of the Optical Society of America (1917-1983), 1964, 54:931-933.

[6] Stark H, Oskoui P, et al. High-resolution image recovery from image-plane arrays, using convex projections [J]. Journal of the Optical Society of America Optics & Image Science, 1989, 6(11): 1715.

[7] Irani M, Peleg S. Improving resolution by image registration [J]. Cvgip Graphical Models & Image Processing, 1991, 53(3):231-239.

[8] Schultz R R, Stevenson R L. Extraction of high-resolution frames from video sequences[J]. IEEE Transactions on Image Processing a Publication of the IEEE Signal Processing Society, 1996, 5(6):996-1011.

[9] Freeman W T, Pasztor E C. Learning Low-Level Vision [J]. International Journal of Computer Vision, 2000, 40(l):25-47.

[10] Chang H, Yang D Y, Xiong Y. Super-resolution through neighbor embedding[C]. Computer Vision and Pattern Recognition, 2004: 1-1.

[11] C Dong, C C Loy, K He, X Tang, Image super-resolution using deep convolutional networks [J], IEEE Trans Pattern Anal Mach Intel, 2016, 38(2):295–307.

[12] C Dong, C C Loy, K He, X Tang, Learning a deep convolutional network for image super-resolution [J], in Proceedings of European Conference on Computer Vision, ECCV September 6–12, 2014: 184–199.

[13] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks [C]. International Conference on Neural Information Processing Systems. Curran Associates Inc. 2012: 1097-1105.

[14] Szegedy C, Wei L, Jia Y, et al. Going Deeper with Convolutions [J]. IEEE Computer Society, 2014: 10-13.

[15] Simonyan K, Zisserman. A Very Deep Convolutional Networks for Large-Scale Image Recognition [J]. Computer Science, 2014: 37-41

[16] He K, Zhang X, Ren S. et al. Deep residual learning for image recognition [J]. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016: 770-778.

[17] J Kim, J K Lee, K M Lee. Accurate image super-resolution using very deep convolutional networks [J], in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Las Vegas, NV, USA, 2016: 1646–1654.

[18] B. Lim, S. Son, H. Kim, S. Nah, K.M. Lee. Enhanced deep residual networks for single image super-resolution [J], in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2017: 1132–1140.

[19] D P Kingma, J Ba, Adam: a method for stochastic optimization [J], in Proceedings of International Conference on Learning Representations, 2015: 7–9.

[20] Huang G, Liu Z, Laurens V, et al. Densely Connected Convolutional Networks [J]. IEEE Computer Society, 2016.

[21] Goodfellow J, Pougetabadie J, Mirza M, et al. Generative Adversarial Networks [J]. Advaaces in Neural Information Processing Systems, 2014, 3:2672-2680.

[22] 张帅勇, 刘美琴, 姚超, 林春雨, 赵耀. 分级特征反馈融合的深度图像超分辨率重建[J]. 自动化学报, 2020, 46: 1−12

[23] 何蕾, 程佳豪, 占志钰, 杨雯博, 刘沛然. 多层次感知残差卷积网络的单幅图像超分重建[J]. 中国图像图像学报 2021, 04: 776-786

[24] 王海勇, 张开心, 管维正. 基于密集Inception的单图像超分辨率重建[J], 计算机应用. http://www.joca.cn/CN/abstract/article_24644.shtml

[25] 彭晏飞, 高艺, 杜婷婷, 等. 生成对抗网络的单图像超分辨率重建方法[J]. 激光与光电子学进展2021,3(18)

[26] 杨娟, 李文静, 汪荣贵, 等. 融合感知损失的生成式对抗超分辨率算法[J]. 中国图象图形学报, 2019, 8:1270-1282.

[27] 李昂, 宋晓莹. 基于生成对抗网络的遥感图像超分辨率重建[J]. 光学与光电技术, 2019, 17(6).

[28] 周登文, 赵丽娟, 段然, 等. 基于递归残差网络的图像超分辨率重建[J]. 自动化学报, 2019, 45(006):1157-1165.

[29] 雷鹏程, 刘丛, 唐坚刚, 彭敦陆. 分层特征融合注意力网络图像超分辨率重建[J]. 中国图象图形学报, 2020, 293(09):59-72.

[30] 李颖华, 赵春娜, 蒋慕蓉. 一种结合自注意力和门控机制的图像超分辨率重建算法[J]. 计算机科学与应用, 2020, 10(12):8.

[31] Zaremba W, Sutskever I, Vinyals O. Recurrent Neural Network Regularization [J]. Eprint Arxiv, 2014.

[32] Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model [C]. Conference of the International Speech Communication Association, Makuhari, Chiba. DBLP, 2010: 1045-1048.

[33] Graves A, Jaitly N. Towards end-to-end speech recognition with recurrent neural networks[C]. International Conference on Machine Learning. 2014: 1764-1772.

[34] Hochreiter S, Schmidhuber J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.

[35] Shi Y, Lee S G. Learning context using segment-level lstm for neural sequence labeling [J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019. 99: 1-1.

[36] Ballas N, Yao L, Pal C, Courville. A Delving Deeper into Convolutional Networks for Learning Video Representations [J], In Conf Learn Represent, 2016, abs:1511.06432.

[37] Chung J, Gulcehre C, Cho K H, et al. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling [J]. Eprint Arxiv, 2014: 46-53.

[38] Yue L, Shen H, Li J, et al. Image super-resolution: The techniques, applications, and future [J]. Signal Processing, 2016, 128: 389-408

[39] Nasrollahi K, Moeslund T B. Super-resolution: a comprehensive survey [J]. Machine vision and applications, 2014, 25(6): 1423-1468.

[40] Tian J, Ma K K. A survey on super-resolution imaging [J]. Signal Image and Video Processing, 2011, 5(3): 329-342.

[41] 王春霞, 苏红旗, 范郭亮. 图像超分辨率重建技术综述[J]. 计算机技术与发展, 2011, 21(5):124-127.

[42] Ha V K, Ren J C, Xu X Y, et al. Deep Learning Based Single Image Super-resolution: A Survey [J]. International Journal of Automation and Computing, 2019, 16(4):413-426.

[43] Shah A J, Gupta S B. Image super resolution-A survey[C]. Emerging Technology Trends in Electronics, Communication and Networking, 2012.17-20.

[44] Y Zhang, K Li, K Li, L Wang, B Zhong, et al. Image super-resolution using very deep residual channel attention networks [J], 2018, ArXiv preprint: 1807.02758.

[45] Jiang K, Wang Z, Yi P, et al. Hierarchical Dense Recursive Network for Image Super-Resolution [J], Pattern Recognition, 2020, 107:107475

[46] Tong T, Li G, Liu X, et al. Image Super-Resolution Using Dense Skip Connections[C]. IEEE International Conference on Computer Vision. IEEE Computer Society, 2017.

[47] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C], In Medical Image Computing and Computer Assisted Intervention, Munich, 2015:234-241.

[48] P Pinheiro, R Collobert. Recurrent convolutional neural networks for scene labeling [J], In Journal of Machine Learning Research, 2014 1(32):82–90.

[49] Nanyan Liu, Guangpu Gao, Xinyu Xu. Recurrent Embedded Hourglass Network for Single Image Super-Resolution [J], in IEEE Access, 2020 8: 166176-166183.