医学图像是临床诊断过程中的重要参考信息，高分辨率的医学图像能够提供更多的细节信息，帮助医生做出更好的治疗决策。因此，利用超分辨率重建技术生成高分辨率的医学图像对临床诊断有重要的应用价值。目前常用的方法包括传统方法和基于深度学习的方法。传统方法存在信息容易丢失、重建图像效果模糊等问题，难以满足临床诊断的实际需求。而利用深度学习技术重建高分辨率医学图像能够更好地恢复医学图像的细节纹理。本文基于生成对抗网络和扩散模型，研究医学图像超分辨率重建问题。主要的研究工作有：

（1）针对大尺度图像超分辨率重建时出现的过度平滑导致图像出现一些非真实感的问题，提出了一种基于生成对抗网络的医学图像超分辨率重建算法: Trans-GAN。将Transformer模型作为生成对抗模型生成器的基本模块，为了增大模型的感受野，提取更多的图像全局特征，将Transformer模型置于基础卷积操作之后，重新构建生成对抗网络的生成器架构。设计了新的残差单元，增加了新的卷积核单元，在不增加模型计算参数的条件下，提高残差计算的效果。基于改进后的残差单元，设计了新的并行多分支残差网络。同时引入了感知损失函数和总变差损失函数，避免了重建图像的过度平滑。实验结果表明，所提的Trans-GAN算法与SRGAN算法相比，在4倍放大因子下的PSNR和SSIM平均提升了11.0%和3.5%。

（2）针对Trans-GAN算法超分辨率重建的图像出现伪影的问题，对基于扩散模型的图像超分辨率算法进行了改进，提出了SRDiff++模型。该模型是一种扩散模型，训练稳定且通过迭代细化逐步恢复图像，使生成的图像质量较高。该模型对SRDiff进行了2点改进：1)使用RAI模块替换单一卷积层，将不同大小的卷积层通过并联的方式组合在一起，通过组合卷积捕获的图像特征信息在深度这个维度进行拼接，形成一个更深的矩阵，对网络的深度和宽度进行有效的扩充，从不同尺度聚合图像信息，提取图像特征。2)改进下采样过程：在下采样步骤中，依次增加通道注意力，空间注意力，捕获重要通道和重要位置，并增加CA注意力，进一步定位图像关键区域，捕获关键信息，提高模型表达能力。实验结果表明，改进后的算法在4倍放大因子下的PSNR和SSIM平均提升了1.35%和0.9%，LPIPS指标降低了16.67%。

（3）设计并实现了医学图像超分辨率重建系统。该系统集成了本文提出的Trans-GAN和SRDiff++两种算法，可对医学图像进行实时超分辨率重建，且基于该系统建立了医学图像数据集收集平台，一定程度上解决了医学图像数据集不足的问题。

本文提出了两种基于深度学习的医学图像超分辨率重建算法，分别是Trans-GAN和SRDiff++。同时设计并实现了一个集成以上两种算法的在线医学图像超分辨率重建平台，解决了医学图像重建过程中可能出现的过度平滑和伪影问题。

Medical images provide essential reference information in the clinical diagnosis process, and high-resolution medical images can provide more detailed information, helping doctors make better treatment decisions. Therefore, using super-resolution reconstruction technology to generate high-resolution medical images has important application value for clinical diagnosis. Currently, conventional methods and deep learning-based methods are commonly used. Conventional methods are prone to problems such as information loss and blurry outcomes, which cannot meet the practical requirements of clinical diagnosis. In contrast, using deep learning technology to reconstruct high-resolution medical images can better restore the texture details of medical images. This dissertation studies the problem of medical image super-resolution reconstruction based on generative adversarial networks and diffusion models. The main research work includes:

(1) A generative adversarial network-based medical image super-resolution reconstruction algorithm called Trans-GAN is proposed to address the issue of excessive smoothing in large-scale image super-resolution reconstruction that leads to unrealistic imaging. The Transformer model is utilized as the fundamental module of the generative adversarial model's generator architecture to increase the model's receptive field and extract additional global image features. To achieve this, the Transformer model is placed after the underlying convolution operation, therefore reconstructing the generator architecture of the generative adversarial network. A new residual unit design that improves residual calculation without increasing the model's computational parameters is proposed by adding new convolutional kernel units. A new parallel multi-branch residual network is designed based on the improved residual unit. The perceptual loss function and total variation loss function are used simultaneously to avoid excessive smoothing in reconstructed images. The experimental results show that the proposed Trans-GAN algorithm achieved an average improvement of 11.0% in PSNR and 3.5% in SSIM compared to the SRGAN algorithm under 4x magnification.

(2) To address the problem of image artifacts in Trans-GAN algorithm-based super-resolution image reconstruction, an improved diffusion model-based image super-resolution algorithm named SRDiff++ is proposed. The SRDiff++ model is a diffusion model that trains stably and gradually recovers images through iterative refinement to achieve high-quality image generation. Two improvements made to the SRDiff model include: 1) The RAI module is used to replace the single convolutional layer by combining various sizes of convolutional layers in parallel, capturing image feature information obtained from combined convolutions and concatenating them on the depth dimension to form a more profound matrix, effectively expanding the network's depth and width, aggregating image information from different scales, and extracting image features. 2) Improvements to the downsampling process: adding channel attention, spatial attention, and channel-spatial attention to sequentially highlight essential channel and position, precisely locate image critical regions, capture crucial information and enhance the model's expressive ability. The experimental results showed that the improved algorithm increased the average PSNR and SSIM by 1.35% and 0.9%, respectively, under four times magnification. Additionally, the LPIPS indicator decreased by 16.67%.

(3) A medical image super-resolution reconstruction system was designed and implemented, which integrates two algorithms proposed in this dissertation, Trans-GAN and SRDiff++. This system can perform real-time super-resolution reconstruction on medical images, and a medical image dataset collection platform has been developed based on this system, solving to some extent the shortage problem of medical image dataset.

This dissertation introduces two deep learning-based algorithms for medical image super-resolution reconstruction, namely Trans-GAN and SRDiff++. Additionally, an online medical image super-resolution reconstruction platform is designed and implemented by integrating the two aforementioned algorithms to mitigate potential issues such as oversmoothing and artifacts during the reconstruction process.

[1] Hounsfield G N. Computed medical imaging[J]. Science, 1980, 210(4465): 22–28.

[2] Picano E. Sustainability of medical imaging[J]. Bmj, 2004, 328(7439): 578–580.

[3] 袁灵, 成思航, 苏童, 等. 医学影像学的研究进展综述[J]. 中国科学:生命科学, 2021, 51(08): 1130–1139.

[4] Wolbarst A B, Hendee W R. Evolving and experimental technologies in medical imaging[J]. Radiology, 2006, 238(1): 16–39.

[5] Kasban H, El-Bendary M, Salama D. A comparative study of medical imaging techniques[J]. International journal of information science and intelligent system, 2015, 4(2): 37–58.

[6] 穆根, 张振辉, 石玉娇. 生物医学影像中的光声成像技术[J]. 中国激光, 2022, 49(20): 133–151.

[7] Hathcock J T, Stickle R L. Principles and concepts of computed tomography[J]. Veterinary clinics of north america: small animal practice, 1993, 23(2): 399–415.

[8] Goldman L W. Principles of CT and CT technology[J]. Journal of nuclear medicine technology, 2007, 35(3): 115–128.

[9] Rajendran K, Petersilka M, Henning A, et al. First clinical photon-counting detector CT system: technical evaluation[J]. Radiology, 2022, 303(1): 130–138.

[10] Harris J L. Diffraction and resolving power[J]. JOSA, 1964, 54(7): 931–936.

[11] Tsai R Y, Huang T S. Multiframe image restoration and registration[J]. Multiframe image restoration and registration, 1984, 1: 317–339.

[12] Prashanth H, Shashidhara H, Murthy K B. Image scaling comparison using universal image quality index[C]//2009 International conference on advances in computing, control, and telecommunication technologies. IEEE, 2009: 859–863.

[13] Gribbon K T, Bailey D G. A novel approach to real-time bilinear interpolation[C]//Proceedings. DELTA 2004. Second IEEE international workshop on electronic design, test and applications. IEEE, 2004: 126–131.

[14] Keys R. Cubic convolution interpolation for digital image processing[J]. IEEE transactions on acoustics, speech, and signal processing, 1981, 29(6): 1153–1160.

[15] 谢海平, 谢凯利, 杨海涛. 图像超分辨率方法研究进展[J]. 计算机工程与应用, 2020, 56(19): 34–41.

[16] Irani M, Peleg S. Improving resolution by image registration[J]. CVGIP: Graphical models and image processing, 1991, 53(3): 231–239.

[17] Tom B C, Katsaggelos A K. Iterative algorithm for improving the resolution of video sequences[C]//Visual communications and image processing'96. SPIE, 1996: 1430–1438.

[18] Liu C, Sun D. On Bayesian adaptive video super resolution[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 36(2): 346–360.

[19] Rahiman V A, George S N. Single image super resolution using neighbor embedding and statistical prediction model[J]. Computers & Electrical engineering, 2017, 62: 281–292.

[20] 练秋生, 石保顺, 陈书贞. 字典学习模型, 算法及其应用研究进展[J]. 自动化学报, 2015, 41(2): 240–260.

[21] 唐艳秋, 潘泓, 朱亚平, 等. 图像超分辨率重建研究综述[J]. 电子学报, 2020, 48(7): 1407–1420.

[22] 黄健, 赵元元, 郭苹, 等. 深度学习的单幅图像超分辨率重建方法综述[J]. 计算机工程与应用, 2021, 57(18): 13–23.

[23] Lepcha D C, Goyal B, Dogra A, et al. Image super-resolution: a comprehensive review, recent trends, challenges and applications[J]. Information fusion, 2022: 1–31.

[24] 李洪安, 郑峭雪, 陶若霖, 等. 基于深度学习的图像超分辨率研究综述[J]. 图学学报, 2023, 44(01): 1–15.

[25] Cao S, Wu C-Y, Krähenbühl P. Lossless image compression through super-resolution[J]. arXiv preprint arXiv:2004.02872, 2020.

[26] Guo K, Guo H, Ren S, et al. Towards efficient motion-blurred public security video super-resolution based on back-projection networks[J]. Journal of network and computer applications, 2020, 166: 1–34.

[27] Li Y, Sixou B, Peyrin F. A review of the deep learning methods for medical images super resolution problems[J]. Irbm, 2021, 42(2): 120–133.

[28] 左艳, 黄钢, 聂生东. 深度学习在医学影像智能处理中的应用与挑战[J]. 中国图象图形学报, 2021, 26(02): 305–315.

[29] 柯舒婷, 陈明惠, 郑泽希, 等. 生成对抗网络对OCT视网膜图像的超分辨率重建[J]. 中国激光, 2022, 49(15): 90–98.

[30] Wang Z, Jiang K, Yi P, et al. Ultra-dense GAN for satellite imagery super-resolution[J]. Neurocomputing, 2020, 398: 328–337.

[31] 何江, 袁强强, 李杰. 面向多光谱卫星成像的广义光谱超分辨率[J]. 光子学报, 2023, 52(02): 159–166.

[32] 祝轩, 柳欣, 兰志翔, 等. 光流估计补偿结合生成对抗网络提高视频超分辨率感知质量[J]. 西北大学学报(自然科学版), 2021, 51(02): 173–182.

[33] Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 38(2): 295–307.

[34] Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network[C]//Computer Vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer international publishing, 2016: 391–407.

[35] Shi W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 1874–1883.

[36] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 1646–1654.

[37] Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2017: 136–144.

[38] Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4681–4690.

[39] Wang X, Yu K, Wu S, et al. Esrgan: Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European conference on computer vision (ECCV) workshops. 2018: 1–16.

[40] Rakotonirina N C, Rasoanaivo A. ESRGAN+: Further improving enhanced super-resolution generative adversarial network[C]//ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2020: 3637–3641.

[41] 丁玲, 丁世飞, 张健, 等. 使用VGG能量损失的单图像超分辨率重建[J]. 软件学报, 2021, 32(11): 3659–3668.

[42] 孙超文, 陈晓. 基于多尺度特征融合反投影网络的图像超分辨率重建[J]. 自动化学报, 2021, 47(07): 1689–1700.

[43] Yang F, Yang H, Fu J, et al. Learning texture transformer network for image super-resolution[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 5791–5800.

[44] Chen H, Wang Y, Guo T, et al. Pre-trained image processing transformer[C]//Proceedings of the IEEE/CVF conference on computer cision and pattern recognition. 2021: 12299–12310.

[45] Liang J, Cao J, Sun G, et al. Swinir: Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 1833–1844.

[46] Li H, Yang Y, Chang M, et al. Srdiff: Single image super-resolution with diffusion probabilistic models[J]. Neurocomputing, 2022, 479: 47–59.

[47] Saharia C, Ho J, Chan W, et al. Image super-resolution via iterative refinement[J]. IEEE transactions on pattern analysis and machine intelligence, 2022: 1–14.

[48] Mahapatra D, Bozorgtabar B, Garnavi R. Image super-resolution using progressive generative adversarial networks for medical image analysis[J]. Computerized medical imaging and graphics, 2019, 71: 30–39.

[49] Chen L, Yang X, Jeon G, et al. A trusted medical image super-resolution method based on feedback adaptive weighted dense network[J]. Artificial intelligence in medicine, 2020, 106: 1–14.

[50] Deeba F, Kun S, Dharejo F A, et al. Wavelet-based enhanced medical image super resolution[J]. IEEE Access, 2020, 8: 37035–37044.

[51] Qiu D, Zheng L, Zhu J, et al. Multiple improved residual networks for medical image super-resolution[J]. Future generation computer systems, 2021, 116: 200–208.

[52] Ahmad W, Ali H, Shah Z, et al. A new generative adversarial network for medical images super resolution[J]. Scientific reports, 2022, 12(1): 9533.

[53] Gao Y, Li H, Dong J, et al. A deep convolutional network for medical image super-resolution[C]//2017 Chinese automation congress (CAC). IEEE, 2017: 5310–5315.

[54] Umehara K, Ota J, Ishida T. Application of super-resolution convolutional neural network for enhancing image resolution in chest CT[J]. Journal of digital imaging, 2018, 31: 441–450.

[55] Park J, Hwang D, Kim K Y, et al. Computed tomography super-resolution using deep convolutional neural network[J]. Physics in medicine & biology, 2018, 63(14): 1–26.

[56] Zhang S, Liang G, Pan S, et al. A fast medical image super resolution method based on deep learning network[J]. IEEE Access, 2018, 7: 12319–12327.

[57] Ren S, Jain D K, Guo K, et al. Towards efficient medical lesion image super-resolution based on deep residual networks[J]. Signal processing: Image communication, 2019, 75: 1–10.

[58] You C, Li G, Zhang Y, et al. CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-CIRCLE)[J]. IEEE transactions on medical imaging, 2019, 39(1): 188–203.

[59] Ma Y, Liu K, Xiong H, et al. Medical image super-resolution using a relativistic average generative adversarial network[J]. Nuclear instruments and methods in physics research section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 2021, 992: 1–6.

[60] 王一宁, 赵青杉, 秦品乐, 等. 基于轻量密集神经网络的医学图像超分辨率重建算法[J]. 计算机应用, 2022, 42(08): 2586–2592.

[61] Chi J, Sun Z, Wang H, et al. CT image super-resolution reconstruction based on global hybrid attention[J]. Computers in biology and medicine, 2022, 150: 1–18.

[62] 章伟帆, 曾庆鹏. 多重放大的医学图像超分辨率重建[J]. 计算机工程与应用, 2022, 58(23): 230–237.

[63] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139–144.

[64] Pan Z, Yu W, Yi X, et al. Recent progress on generative adversarial networks (GANs): A survey[J]. IEEE Access, 2019, 7: 36322–36333.

[65] Wiatrak M, Albrecht S. Stabilizing generative adversarial network training: a survey. arXiv preprint arXiv: 191000927[J]. 2019.

[66] Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis[J]. Artificial intelligence in medicine, 2020, 109: 1–40.

[67] Wang L, Chen W, Yang W, et al. A state-of-the-art review on image synthesis with generative adversarial networks[J]. IEEE Access, 2020, 8: 63514–63537.

[68] Gui J, Sun Z, Wen Y, et al. A review on generative adversarial networks: Algorithms, theory, and applications[J]. IEEE transactions on knowledge and data engineering, 2021: 1–28.

[69] Liu M-Y, Huang X, Yu J, et al. Generative adversarial networks for image and video synthesis: Algorithms and applications[J]. Proceedings of the IEEE, 2021, 109(5): 839–862.

[70] Saxena D, Cao J. Generative adversarial networks (GANs) challenges, solutions, and future directions[J]. ACM computing surveys (CSUR), 2021, 54(3): 1–42.

[71] Wang Z, She Q, Ward T E. Generative adversarial networks in computer vision: A survey and taxonomy[J]. ACM computing surveys (CSUR), 2021, 54(2): 1–38.

[72] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30: 5998–6008.

[73] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE transactions on image processing, 2004, 13(4): 600–612.

[74] Zhang R, Isola P, Efros A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 586–595.

[75] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.

[76] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 2015: 234–241.

[77] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770–778.

[78] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1–9.

[79] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2818–2826.

[80] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132–7141.

[81] Woo S, Park J, Lee J-Y, et al. Cbam: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 3–19.

[82] Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 13713–13722.

[83] Lugmayr A, Danelljan M, Van Gool L, et al. Srflow: Learning the super-resolution space with normalizing flow[C]//Computer vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16. Springer, 2020: 715–732.