近些年来，随着通信业务量的增加，人们对通信系统的要求也越来越高，多输入多输出技术（Multiple-Input Multiple-Output，MIMO）、正交频分复用技术（Orthogonal Frequency Division Multiplexing，OFDM）和非正交多址技术（Non Othogonal Multiple Access，NOMA）作为5G的关键技术再次为人们所关注。这三种技术可以极大地提高系统的频谱效率和用户接入量，但是技术本身存在着峰均功率比（Peak to Average Power Ratio，PAPR）过高、信道估计困难和接收机复杂度过高等缺点，这些问题极大地限制了这些技术的推广。面对这些端到端系统中存在的难题，深度学习提供了一种新的解决方案。本文结合MIMO-OFDM技术、NOMA技术和深度学习算法分别针对PAPR过高问题、系统复杂度过高和多用户干扰等问题，提出了两种神经网络接收机用于解决上述问题。

（1）针对MIMO-OFDM系统中接收机的分布式优化无法确保全局最优的问题，本文提出一种神经网络接收机模型。该接收机模型利用神经网络替代传统接收机的整个信息恢复环节，通过数据驱动来代替模型驱动从而达到简化系统模型的目的。进一步，本文给出了2D-Conv-DensNet神经网络接收机的具体模型，在此基础上还设计了多个二分类器用来实现多位数据同时恢复。为了能够实现神经网络接收机对不同长度用户数据恢复的适应性，本文还在神经网络分类器前引入全局池化层。在神经网络得到足量训练后进行仿真实验，结果表明所提出的神经网络接收机在考虑的信噪比范围内的平均误码率分别是传统硬判决、最大似然接收机和现存其他神经网络接收机的8.1%、78.2%和43.7%。此外，实验还验证了所提出的神经网络接收机无论在何种天线组合、循环前缀长度和调制方式下，甚至在动态环境下的盲接收都拥有着较传统接收机更优的误码率性能。最后，本文提出的2D-Conv-DensNet神经网络接收机还可以在一定程度上补偿由PAPR过高带来的信号失真。

（2）在多用户MIMO-NOMA系统中存在严重的用户间干扰，所以本文将神经网络接收机引入到MIMO-NOMA系统中来。针对该问题，本文提出直接采用神经网络作为接收机以及联合预编码的神经网络接收机两种方案。本文所提的方案二是在方案一的基础上加入了另一个神经网络用于实现预编码，这与神经网络接收机相呼应进而实现以用户信息误差最小化的联合优化，在保证系统可靠性的同时进一步降低系统复杂度。仿真结果显示在所考虑的信噪比范围内，所提方案一和方案二在拥有最佳接收机误码率性能的78.4%和63.2%的情况下，仅仅具有最佳接收机复杂度的21.6%和5.2%。这表明所提方案拥有较好的误码率性能的同时还具备较低的复杂度。此外，实验中还发现在所提出的方案中信道状态较差的用户对于循环前缀、学习率和信噪比的变化更为敏感。

In recent years, with the increase of communication business, 5G came into being. Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiplexing (OFDM), and Non-Orthogonal Multiple Access (NOMA) are the keys of 5G. These three technologies can greatly improve the spectral efficiency of the system and the amount of user access. These techniques have disadvantages such as excessive Peak to Average Power Ratio (PAPR), difficulty in channel estimation, and excessive receiver complexity. Faced with these challenges in end-to-end systems, deep learning (DL) provides a new solution. In this paper, a neural network receiver is proposed to solve the above problems with MIMO-OFDM, NOMA and DL.

(1) Aiming at the problem of bad data recovery performance caused by PAPR in MIMO-OFDM systems, a neural network receiver model is proposed in this paper. The receiver to replace the entire information recovery link of the traditional receiver with neural network to simplify the system model. In this paper, a neural network receiver model is proposed. Further, a specific model of the 2D-Conv-DensNet neural network receiver was proposed, and on this basis, multiple binary classifiers are designed to recover multi-bit data. After sufficient training of the neural network, simulation experiments are carried out. The results show that the average bit error rate of the proposed neural network receiver in the range of the considered signal-to-noise ratio is 8.1% of the traditional hard-decision receiver, 78.2% of the maximum likelihood receiver and 43.7% of other existing neural network receivers, respectively. In addition, the experiment also verifies that the proposed neural network receiver has better bit error (BER) rate performance than the traditional receiver regardless of the antenna combination, cyclic prefix length and modulation method, and even blind reception in a dynamic environment. Finally, the 2D-Conv-DensNet neural network receiver proposed in this paper can also compensate the signal distortion caused by the high PAPR to a certain extent.

(2) There is serious inter-user interference in multi-user MIMO-NOMA system, so neural network receiver is introduced into MIMO-NOMA system. Two schemes are proposed, one of which directly uses a neural network as the receiver and the other is to add a neural network for precoding. The second scheme proposed in this paper is to add another neural network for precoding on the basis of the first scheme, which echoes the neural network receiver and realizes the joint optimization of minimizing the user information error, which makes the system less complex. The simulation results show that in the range of the considered signal-to-noise ratio, the proposed scheme 1 and scheme 2 have only 78.4% and 63.2 % of the best receiver BER performance, but only 21.6 % and 5.2% complexity of the best receiver. This shows that the proposed scheme has better BER performance and lower complexity. In addition, it is also found in experiments that users with bad channel status in the proposed scheme are more sensitive to changes in cyclic prefix, learning rate and signal-to-noise ratio.

[1] 申京. MIMO-OFDM系统中信道估计及信号检测算法的研究[D].北京：北京邮电大学,2012.

[2] 陆震. 宽带无线MIMO OFDM移动通信系统中的同步和信道估计技术研究[D].上海：上海交通大学,2008.

[3] Liu Z, Hu X, Han K, et al. Low-Complexity PAPR Reduction Method for OFDM Systems Based on Real-Valued Neural Networks[J]. IEEE Wireless Communication Letters, 2020, 9(11):1840-1844.

[4] Li H, Wei J, N Jin. Low-Complexity Tone Reservation Scheme Using Pre-Generated Peak-Canceling Signals[J]. IEEE Communications Letters, 2019,23(9): 1586-1589.

[5] W. Guo, C. Li, H. Zhao, R. Wen, and Y. Tang. Comprehensive effects of imperfect synchronization and channel estimation on known interference cancellation[J]. IEEE Transactions on Vehicular Technology, 2020, 69(1): 457-470.

[6] V. Kishore, V. S. Prasad and V. V. Mani. A Blind Timing Synchronization Algorithm for DCO-OFDM VLC Systems[J]. IEEE Photonics Technology Letters, 2020, 32(17): 1121-1124.

[7] Shao H, Yang L, Tan J, et al. Robust timing and frequency synchronization based on constant amplitude zero autocorrelation sequence for OFDM systems. IEEE, 2015, 14-17.

[8] Schmidl T M, Cox C. Robust frequency and timing synchronization for OFDM[J]. IEEE Transctions on Communication, 1997, 45(12):1613-1621.

[9] Singya P K, Gandhi A S. A data-aided timing offset estimator for OFDM synchronization. 2015 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES). IEEE, 2015, 1-5.

[10] Nasraoui L , Atallah L N , Siala M . Reduced-Complexity synchronization technique for MIMO-OFDM WLAN systems[C]// International Conference on Communications & Networking. IEEE, 2016, 1-5.

[11] Mata T, Boonsrimuang P, Boontra P. A PAPR Reduction Scheme based on Improved PTS with ABC Algorithm for OFDM Signal[C]// The 15th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTICON2018). 2018,469-472.

[12] Zhao J, Ni S, Gong Y. Peak-to-Average Power Ratio Reduction of FBMC/OQAM Signal Using a Joint Optimization Scheme[J]. IEEE Access, 2017, 5: 15810-15819.

[13] Joo H S, Kim K H, No J S, et al. New PTS Schemes for PAPR Reduction of OFDM Signals Without Side Information[J]. IEEE Transactions on Broadcasting, 2017, 63(3): 562-570.

[14] Anand K, Guan Y L, Liu X, et al. Pilot Design for BEM-Based Channel Estimation in Doubly Selective Channel[J]. IEEE Transactions on Vehicular Technology, 2020, 69(2):1679-1694.

[15] Singh P. Semi-Blind, Training and Data-Aided Channel Estimation Schemes for MIMO-FBMC-OQAM Systems[J]. IEEE Transactions on Signal Processing, 2019, 67(18): 4668-4682.

[16] He H, Jin S, Wen C K, et al. Model-Driven Deep Learning for Physical Layer Communications[J]. IEEE Wireless Communications, 2019,26(5):77-83.

[17] Tianqi, Wang, Chao-Kai, et al. Deep Learning-Based CSI Feedback Approach for Time-Varying Massive MIMO Channels[J]. Communications Letters IEEE, 2018,8(2):416-419.

[18] Li Y, Cheng X, Gui G. Co-Robust-ADMM-Net: Joint ADMM Framework and DNN for Robust Sparse Composite Regularization[J]. IEEE Access, 2018,6:47943-47952.

[19] Liang F, Shen C, Wu F. An Iterative BP-CNN Architecture for Channel Decoding[J]. IEEE Journal of Selected Topics in Signal Processing, 2018,12(1):144-159.

[20] D. Wu, M. Nekovee, and Y. Wang, Deep learning-based autoencoder for m-user wireless interference channel physical layer design[J]. IEEE Access,2020,4:174679-174691.

[21] Nachmani E, Marciano E, Lugosch L, et al. Deep Learning Methods for Improved Decoding of Linear Codes[J]. IEEE Journal of Selected Topics in Signal Processing, 2018,12(1):1-1.

[22] Cammerer S, Gruber T, Hoydis J, et al. Scaling Deep Learning-based Decoding of Polar Codes via Partitioning[C]//2017 IEEE Global Communications Conference, 2017:1-6.

[23] Fang L, Wu L. Deep learning detection method for signal demodulation in short range multipath channel[C]// IEEE International Conference on Opto-electronic Information Processing. IEEE, 2017:16-20.

[24] Ouyang X, Wu L. Faster-than-Nyquist rate communication via convolutional neural networks-based demodulators[J]. 东南大学学报(英文版), 2016.

[25] Zhang L, Yang L. Machine Learning for Joint Channel Equalization and Signal Detection[M]. John Wiley & Sons, Ltd, 2019:213-241.

[26] Yang Y, Gao F, Ma X, et al. Deep Learning-Based Channel Estimation for Doubly Selective Fading Channels[J]. IEEE Access, 2019,7: 36579-36589.

[27] X Ma, Hao Y, Ye L. Learning Assisted Estimation for Time- Varying Channels[C]// 2018 15th International Symposium on Wireless Communication Systems (ISWCS). IEEE, 2018: 1-5.

[28] Cheng X, Liu D, Yan S, et al. Channel Estimation and Equalization Based on Deep BLSTM for FBMC-OQAM Systems[C]// ICC 2019 - 2019 IEEE International Conference on Communications (ICC). IEEE, 2019: 1-6.

[29] Ye H, Li G Y, Juang B. Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems[J]. IEEE Wireless Communication Letters, 2017,7(1): 114-117.

[30] Ye H, Li G Y, Juang B, et al. Channel Agnostic End-to-End Learning based Communication Systems with Conditional GAN[C]. 2018 IEEE Globecom Workshops. IEEE, 2018:1-5.

[31] Lu H, Jiang M, Cheng J. Deep Learning Aided Robust Joint Channel Classification, Channel Estimation, and Signal Detection for Underwater Optical Communication[J]. IEEE Transactions on Communications, 2020,69(4): 2290-2303.

[32] Baek, Myung-Sun, Kwak, et al. Implementation Methodologies of Deep Learning-Based Signal Detection for Conventional MIMO Transmitters. IEEE Transactions on Broadcasting. 2019, 65(3):636-642.

[33] Huang G, Liu Z, Laurens V, et al. Densely Connected Convolutional Networks[J]. IEEE Computer Society, 2016.

[34] Simeone O. A Very Brief Introduction to Machine Learning With Applications to Communication Systems[J]. IEEE Transactions on Cognitive Communications and Networking, 2018, 4(4): 648-664.

[35] Nachmani E, Beery Y, Burshtein D. Learning to Decode Linear Codes Using Deep Learning[J]. 2016 54th Annual Allerton Conference on Communication, Control, and Computing IEEE, 2016: 341-346.

[36] Gruber T, Cammerer S, Hoydis J, et al. On Deep Learning-Based Channel Decoding[C]// IEEE. 2017: 1511-1514.

[37] Liang F, Shen C, Wu F. An Iterative BP-CNN Architecture for Channel Decoding[J]. IEEE Journal of Selected Topics in Signal Processing, 2018,12(1): 144-159.

[38] Sun B, Feng H. Efficient Compressed Sensing for Wireless Neural Recording: A Deep Learning Approach[J]. IEEE Signal Processing Letters, 2017, 24(6): 863-867.

[39] Sun B, Feng H. Efficient Compressed Sensing for Wireless Neural Recording: A Deep Learning Approach[J]. IEEE Signal Processing Letters, 2017: 863-867.

[40] [1] Sun X Y, Wang S H. Application of deep packet inspection in peer-to-peer traffic identification. IEEE, 2011: 2963-2965.

[41] Wang P, Chen X, Ye F, et al. A Survey of Techniques for Mobile Service Encrypted Traffic Classification Using Deep Learning[J]. IEEE Access, 2019, 7: 54024-54033.

[42] Aceto G, Ciuonzo D, Montieri A, et al. Mobile Encrypted Traffic Classification Using Deep Learning: Experimental Evaluation, Lessons Learned, and Challenges[J]. IEEE Transactions on Network and Service Management, 2019, 16(2): 445-458.

[43] Jeon Y S, Hong S N, Lee N. Blind detection for MIMO systems with low-resolution ADCs using supervised learning[C]. IEEE, 2017: 1-6.

[44] Wang C. Research and Application of Traffic Sign Detection and Recognition Based on Deep Learning[C]// 2018 International Conference on Robots & Intelligent System (ICRIS). IEEE, 2018: 150-152.

[45] Lu H, Jiang M, Cheng J. Deep Learning Aided Robust Joint Channel Classification, Channel Estimation, and Signal Detection for Underwater Optical Communication[J]. IEEE Transactions on Communications, 2020, 69(4): 2290-2303.

[46] O'Shea T J, Member S, IEEE, et al. Deep Learning Based MIMO Communications[J]. 2017.

[47] Guan G, Huang H, Song Y, et al. Deep Learning for an Effective Nonorthogonal Multiple Access Scheme[J]. IEEE Transactions on Vehicular Technology, 2018, 67(9): 8440-8450.

[48] Zhang N, Cheng K, Kang G. A Machine-Learning-Based Blind Detection on Interference Modulation Order in NOMA Systems[J]. Communications Letters, IEEE, 2018, 20(20): 2463-2466.

[49] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Conference on Computer Vision and Pattern Recognition. (CVPR), 2016: 770-778.

[50] C. Szegedy, L. Wei, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,V. Vanhoucke, and A. Rabinovichet, “Going deeper with convolutions,” Conference on Computer Vision and Pattern Recognition. (CVPR), Boston, USA, Jun. 2015: 1-9.