- 无标题文档
上一篇
免费浏览
查看论文信息
阅读器下载

论文中文题名：

 <font color='red'></font>基<font color='red'></font>于<font color='red'></font>强<font color='red'></font>化<font color='red'></font>学<font color='red'></font>习<font color='red'></font>的<font color='red'></font>水<font color='red'></font>声<font color='red'></font>传<font color='red'></font>感<font color='red'></font>器<font color='red'></font>网<font color='red'></font>络<font color='red'></font>路<font color='red'></font>由<font color='red'></font>协<font color='red'></font>议<font color='red'></font>研<font color='red'></font>究<font color='red'></font>    

姓名：

 冯晓美    

学号：

 <font color='red'></font>2<font color='red'></font>1<font color='red'></font>2<font color='red'></font>0<font color='red'></font>7<font color='red'></font>0<font color='red'></font>4<font color='red'></font>0<font color='red'></font>0<font color='red'></font>3<font color='red'></font>2<font color='red'></font>    

保密级别：

 <font color='red'></font>公<font color='red'></font>开<font color='red'></font>    

论文语种：

 <font color='red'></font>c<font color='red'></font>h<font color='red'></font>i<font color='red'></font>    

学科代码：

 <font color='red'></font>0<font color='red'></font>8<font color='red'></font>1<font color='red'></font>0<font color='red'></font>0<font color='red'></font>1<font color='red'></font>    

学科名称：

 <font color='red'></font>工<font color='red'></font>学<font color='red'></font> <font color='red'></font>-<font color='red'></font> <font color='red'></font>信<font color='red'></font>息<font color='red'></font>与<font color='red'></font>通<font color='red'></font>信<font color='red'></font>工<font color='red'></font>程<font color='red'></font> <font color='red'></font>-<font color='red'></font> <font color='red'></font>通<font color='red'></font>信<font color='red'></font>与<font color='red'></font>信<font color='red'></font>息<font color='red'></font>系<font color='red'></font>统<font color='red'></font>    

学生类型：

 <font color='red'></font>硕<font color='red'></font>士<font color='red'></font>    

学位级别：

 <font color='red'></font>工<font color='red'></font>学<font color='red'></font>硕<font color='red'></font>士<font color='red'></font>    

学位年度：

 <font color='red'></font>2<font color='red'></font>0<font color='red'></font>2<font color='red'></font>4<font color='red'></font>    

培养单位：

 <font color='red'></font>西<font color='red'></font>安<font color='red'></font>科<font color='red'></font>技<font color='red'></font>大<font color='red'></font>学<font color='red'></font>    

院系：

 通信与信息工程学院    

专业：

 信息与通信工程    

研究方向：

 <font color='red'></font>水<font color='red'></font>声<font color='red'></font>通<font color='red'></font>信<font color='red'></font>    

第一导师姓名：

 张育芝    

第一导师单位：

 西安科技大学    

论文提交日期：

 <font color='red'></font>2<font color='red'></font>0<font color='red'></font>2<font color='red'></font>4<font color='red'></font>-<font color='red'></font>0<font color='red'></font>6<font color='red'></font>-<font color='red'></font>1<font color='red'></font>3<font color='red'></font>    

论文答辩日期：

 <font color='red'></font>2<font color='red'></font>0<font color='red'></font>2<font color='red'></font>4<font color='red'></font>-<font color='red'></font>0<font color='red'></font>5<font color='red'></font>-<font color='red'></font>2<font color='red'></font>9<font color='red'></font>    

论文外文题名：

 <font color='red'></font>R<font color='red'></font>e<font color='red'></font>s<font color='red'></font>e<font color='red'></font>a<font color='red'></font>r<font color='red'></font>c<font color='red'></font>h<font color='red'></font> <font color='red'></font>o<font color='red'></font>n<font color='red'></font> <font color='red'></font>R<font color='red'></font>o<font color='red'></font>u<font color='red'></font>t<font color='red'></font>i<font color='red'></font>n<font color='red'></font>g<font color='red'></font> <font color='red'></font>P<font color='red'></font>r<font color='red'></font>o<font color='red'></font>t<font color='red'></font>o<font color='red'></font>c<font color='red'></font>o<font color='red'></font>l<font color='red'></font> <font color='red'></font>F<font color='red'></font>o<font color='red'></font>r<font color='red'></font> <font color='red'></font>U<font color='red'></font>n<font color='red'></font>d<font color='red'></font>e<font color='red'></font>r<font color='red'></font>w<font color='red'></font>a<font color='red'></font>t<font color='red'></font>e<font color='red'></font>r<font color='red'></font> <font color='red'></font>A<font color='red'></font>c<font color='red'></font>o<font color='red'></font>u<font color='red'></font>s<font color='red'></font>t<font color='red'></font>i<font color='red'></font>c<font color='red'></font> <font color='red'></font>S<font color='red'></font>e<font color='red'></font>n<font color='red'></font>s<font color='red'></font>o<font color='red'></font>r<font color='red'></font> <font color='red'></font>N<font color='red'></font>e<font color='red'></font>t<font color='red'></font>w<font color='red'></font>o<font color='red'></font>r<font color='red'></font>k<font color='red'></font>s<font color='red'></font> <font color='red'></font>B<font color='red'></font>a<font color='red'></font>s<font color='red'></font>e<font color='red'></font>d<font color='red'></font> <font color='red'></font>O<font color='red'></font>n<font color='red'></font> <font color='red'></font>R<font color='red'></font>e<font color='red'></font>i<font color='red'></font>n<font color='red'></font>f<font color='red'></font>o<font color='red'></font>r<font color='red'></font>c<font color='red'></font>e<font color='red'></font>m<font color='red'></font>e<font color='red'></font>n<font color='red'></font>t<font color='red'></font> <font color='red'></font>L<font color='red'></font>e<font color='red'></font>a<font color='red'></font>r<font color='red'></font>n<font color='red'></font>i<font color='red'></font>n<font color='red'></font>g<font color='red'></font>    

论文中文关键词：

 强化学习 ; 蚁群算法 ; 链路连通预测 ; 可靠性 ; 低能耗 ; 机会路由    

论文外文关键词：

 Reinforcement learning ; ant colony algorithm ; link connectivity prediction ; reliability ; low energy consumption ; opportunistic routing    

论文中文摘要：

水声路由对于实现高效的水下通信和目标定位具有重要意义。然而，设计水声路由协议面临众多问题，如高传播损耗、窄带宽、高端到端时延、高噪声、能量受限、链路不稳定等，在路由协议设计时这些问题均具有一定影响。针对这些问题，本文采用对复杂、动态、未知环境下表现良好的强化学习算法提出了两种水声路由协议。

（1）面向节点固定的水声传感器网络，提出了一种蚁群辅助强化学习的水声路由协议。协议提出了蚁群算法辅助强化学习的全局决策函数，使用奖励函数和人工蚂蚁来确定全局最优路由选择，将信息素和Q值综合作为动作策略，选择总体评价高的动作策略以确定全局最优路由策略。同时，将锦标赛选择算法与强化学习算法相结合，提出了基于锦标赛算法的两步选择 贪婪算法，对 贪婪算法中随机选择动作进行改进，改善 贪婪算法选择动作的随机性。此外，为了解决空洞问题，提出了空洞检测恢复机制，在数据包路由时，检测陷阱节点以避免空区，并在数据包落入空洞节点时，向下回传，降低Q值重新选择动作以避免空区。仿真结果表明，与其他的水声路由协议相比，本文所提协议在包交付率、能耗、网络寿命和端到端时延方面等都表现良好。

（2）面向节点位移的水声传感器网络，提出了一种基于强化学习的链路连通预测水声路由协议。为了避免因节点位移造成的路由失效和通信中断问题，协议提出了节点位置预测算法以确定节点移动后的新位置。同时，由于节点的移动特性和链路动态性，节点在移动时存在间歇性连接问题，所以本文基于洋流模型提出了链路连通时间预测算法，将预测出的节点新位置坐标用于所提出的链路连通时间预测算法，并将链路连通时间考虑进强化学习奖励函数的设计中，从而提高水声网络的通信性能和稳定性。仿真结果表明，在网络动态变化，节点发生位移的情况下，本文所提协议性能仍然良好。

综上，在节点位置固定或节点位置发生变化时，本文所提的两种水声路由协议均具有良好的性能。

﹀
论文外文摘要：

Underwater acoustic routing is of great significance for achieving efficient underwater communication and target positioning. However, designing underwater acoustic routing protocols faces many problems, such as high propagation loss, narrow bandwidth, high end-to-end delay, high noise, limited energy, link instability, etc. These problems all have a certain impact on the design of routing protocols. In response to these problems, this thesis proposes two underwater acoustic routing protocols using reinforcement learning algorithms that perform well in complex, dynamic, and unknown environments.

(1) For underwater acoustic sensor networks with fixed nodes, an underwater acoustic routing protocol based on ant colony-assisted reinforcement learning is proposed. In this thesis, the global decision-making function of ant colony algorithm assisted reinforcement learning is proposed, the reward function and artificial ants are used to determine the global optimal routing choice, the pheromone and Q value are synthesized as the action strategy, and the action strategy with high overall evaluation is selected to determine the global optimal routing strategy. At the same time, the tournament selection algorithm is combined with the reinforcement learning algorithm, and a two-step selection  greedy algorithm based on the tournament algorithm is proposed to improve the random selection of actions in the  greedy algorithm and improve the randomness of the actions selected by the  greedy algorithm. In addition, in order to solve the void problem, a void detection and recovery mechanism is proposed, which detects the trap node to avoid the void area when the packet is routed, and when the packet falls into the void node, it is sent back downward, and the Q value is lowered to reselect the action to avoid the empty area. The simulation results show that compared with other underwater acoustic routing protocols, the protocol proposed in this thesis performs well in terms of packet delivery rate, energy consumption, network life and end-to-end delay.

(2) For underwater acoustic sensor networks oriented to node displacement, a link connectivity prediction underwater acoustic routing protocol based on reinforcement learning is proposed. In order to avoid routing failures and communication interruptions caused by node displacement, the protocol proposes a node location prediction algorithm to determine the new location of the node after movement. At the same time, due to the mobility characteristics of nodes and link dynamics, nodes have intermittent connection problems when moving. Therefore, this thesis proposes a link connection time prediction algorithm based on the ocean current model, and uses the predicted new location coordinates of nodes to predict the proposed link connection time prediction algorithm, and considers the link connection time into the design of the reinforcement learning reward function, thereby improving the underwater acoustic communication performance and stability of the network. The simulation results show that the performance of the protocol proposed in this thesis is still good under network dynamic changes, and node displacements.

In summary, both underwater acoustic routing protocols proposed in this thesis have good performance when the node position is fixed or the node position changes.

﹀
参考文献：

[1]韩增林，李博，陈明宝，等.“海洋经济高质量发展”笔谈[J].中国海洋大学学报(社会科学版), 2019, (05): 13-21.

[2]马晓晓. 水下通信技术综述[J].电子世界，2020(14):104.

[3]许克平，许天增，许茹等. 基于水声的水下无线通信研究[J].厦门大学学报（自然科学版），2001(02):311-319.

[4]Johnson D J, Li J, Chen Z. Reinforcement Learning: An Introduction[J]. Neurocomputing,2000,35(1):205-206.

[5]王明华，桑恩方，乔钢. 高速水声通信中正交频分复用技术试验研究[J]. 海洋工程, 2006, (04): 100-104+115.

[6]Rice J, Creber B, Fletcher C, et al. Evolution of Seaweb underwater acoustic networking[C]// OCEANS 2000 MTS/IEEE Conference and Exhibition. 2000:2007-2017.

[7]Zhang J, Dong Y. Simulation and performance analysis of distributed attack against MAC layer of underwater acoustic network based on OPNET[C]// IEEE International Conference on Information and Automation. 2017:1072-1076.

[8]Nie W, Ling M, Zhang B, et al. A Brief Analysis of United States Unmanned Underwater Combat System[J]. Journal of Unmanned Undersea Systems, 2017, 25(4): 310-318.

[9]陈俏君，李德磊，冯建龙. 基于CiteSpace的海洋热浪研究可视化分析[J]. 海洋科学, 2023, 47(02): 109-124.

[10]闫敬，关新平，罗小元，等. 水下信息物理系统探测–通信–控制一体化:挑战与进展[J]. 控制理论与应用, 2022, 39(11): 1996-2008.

[11]BeaujeanJ P, BernaultP E. A new multichannel spatial diversity technique for long-range acoustic communications in shallow water[J]. The Journal of the Acoustical Society of America,2002,112(5):2420.

[12]D. R, J. L W F. Energy Consumption Model for a Broadband Shallow-Water Acoustic Communications Network[J]. IEEE Journal of Oceanic Engineering, 2008, 33(3):335-340.

[13]Van Walree P A, Driesenaar M L. An Acoustic Communication Network for Monitoring of the Underwater Environment[C]//EGS-AGU-EUG Joint Assembly. 2003: 11507.

[14]Acar G, Adams A E. ACMENet: an underwater acoustic sensor network protocol for real-time environmental monitoring in coastal areas[J]. IEE Proceedings-Radar, Sonar and Navigation, 2006, 153(4): 365-380.

[15]Xie P, Cui J, Lao L. VBF: Vector-based forwarding protocol for underwater sensor networks[C]//NETWORKING, 2006: 1216-1221.

[16]Nicolaou N, See A, Xie P, et al. Improving the robustness of location-based routing for underwater sensor networks[C]//Oceans 2007-Europe. 2007: 1-6.

[17]Yu H, Yao N, Liu J. An adaptive routing protocol in underwater sparse acoustic sensor networks[J]. Ad Hoc Networks,2015,34121-143.

[18]Coutinho R W L, Boukerche A, Vieira L F M, et al. GEDAR: Geographic and opportunistic routing protocol with depth adjustment for mobile underwater sensor networks[C]//2014 IEEE International Conference on communications (ICC). 2014: 251-256.

[19]Guan Q, Ji F, Liu Y, et al. Distance-vector-based opportunistic routing for underwater acoustic sensor networks[J]. IEEE Internet of Things Journal, 2019, 6(2): 3831-3839.

[20]闫磊，何天恺，王易因，等. 适用于水下物联网的混合地理路由协议设计[J]. 电子与信息学报, 2022, 44(06): 1966-1973.

[21]Yan H, Shi Z, Cui J. DBR: Depth-based routing for underwater sensor networks[C]//NETWORKING 2008 Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet: 7th International IFIP-TC6 Networking Conference Singapore, May 5-9, 2008 Proceedings 7. Springer Berlin Heidelberg, 2008: 72-86.

[22]Liu G, Li Z. Depth-based multi-hop routing protocol for underwater sensor network[C]//2010 The 2nd international conference on industrial mechatronics and automation. IEEE, 2010, 2: 268-270.

[23]Wahid A, Lee S, Jeong H J, et al. Eedbr: Energy-efficient depth-based routing protocol for underwater wireless sensor networks[C]//Advanced Computer Science and Information Technology: Third International Conference, AST 2011, Seoul, Korea, September 27-29, 2011. Proceedings. 2011: 223-234.

[24]金志刚，梁嘉伟，羊秋玲. 融合深度调整和自适应转发的水声网络机会路由[J]. 电子与信息学报, 2024, 46(01): 49-57.

[25]Su Y, Zhang L, Fu X, et al. ACAR: an ant colony algorithm‐based routing protocol for underwater acoustic sensor network[J]. IET Communications, 2020, 14(22): 3945-3954.

[26]Gola K K, Gupta B. Void Node Avoidance in Underwater Acoustic Sensor Network Using Black Widow Optimization Algorithm[J]. Adhoc and Sensor Wireless Networks, 2022, 52.

[27]B. P, S. M. Emperor Penguin Optimized Q Learning Method for Energy Efficient Opportunistic Routing in Underwater WSN[J]. Wireless Personal Communications, 2022, 128(3): 2039-2072.

[28]Wang Z, Du J, Xia Z, et al. Secure routing in underwater acoustic sensor networks based on AFSA-ACOA fusion algorithm[C]//ICC 2022-IEEE International Conference on Communications. IEEE, 2022: 1409-1414.

[29]Zhang L, Qi J, Wu H. A novel data aggregation method for underwater wireless sensor networks using ant colony optimization algorithm[J]. International Journal of Advanced Computer Science and Applications (UACSA), 2023, 14(4).

[30]Hu T, Fei Y. QELAR: A machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks[J]. IEEE transactions on mobile computing, 2010, 9(6): 796-809.

[31]Yuan Y, Song S, Xu C, et al. QPCR: A Power Controlled Routing Protocol Based On Q-learning For Energy-constrained Underwater Wireless Sensor Networks[C]//2023 IEEE/CIC International Conference on Communications in China (ICCC). IEEE, 2023: 1-6.

[32]Zhou Y, Tao C, Wei X. Anypath routing protocol design via Q-learning for underwater sensor networks[J]. IEEE Internet of Things Journal, 2020, 8(10): 8173-8190.

[33]Jin Z, Duan C, Yang Q, et al. Q-learning-Based Opportunistic Routing with an on-site architecture in UASNs[J]. Ad Hoc Networks, 2021, 119: 102553.

[34]Sukanya C N, Hee-Won K, Ho-Shin C. QTAR: A Q-learning-based topology-aware routing protocol for underwater wireless sensor networks[J]. Computer Networks, 2023,222.

[35]Lu Y, He R, Chen X, et al. Energy-efficient depth-based opportunistic routing with Q-learning for underwater wireless sensor networks[J]. Sensors, 2020, 20(4): 1025.

[36]Ali Z K, Abdul O K, Shahid A, et al. Q-learning based energy-efficient and void avoidance routing protocol for underwater acoustic sensor networks[J]. Computer Networks,2021,197.

[37]Zhang Y, Zhang Z, Chen L, et al. Reinforcement learning-based opportunistic routing protocol for underwater acoustic sensor networks[J]. IEEE Transactions on Vehicular Technology, 2021, 70(3): 2756-2770.

[38]Zhu R, Jiang Q, Huang X, et al. A reinforcement-learning-based opportunistic routing protocol for energy-efficient and Void-Avoided UASNs[J]. IEEE Sensors Journal, 2022, 22(13): 13589-13601.

[39]Wang C, Shen X, Wang H, et al. Reinforcement learning-based opportunistic routing protocol using depth information for energy-efficient underwater wireless sensor networks[J]. IEEE Sensors Journal, 2023.

[40]Fang Z, Wang J, Jiang C, et al. QLACO: Q-learning aided ant colony routing protocol for underwater acoustic sensor networks[C]//2020 IEEE wireless communications and networking conference (WCNC). IEEE, 2020: 1-6.

[41]Zhao W, Han G, Wang X, et al. A Node Location Algorithm Based on Node Movement Prediction in Underwater Acoustic Sensor Networks[J].IEEE Transactions on Vehicular Technology,2020,69(3):3166-3178.

[42]Li Y, Liu M, Zhang S, et al. Node dynamic localization and prediction algorithm for internet of underwater things[J]. IEEE Internet of Things Journal, 2021, 9(7): 5380-5390.

[43]Mamta N, Nitin G. Energy Efficient Localization Through Node Mobility and Propagation Delay Prediction in Underwater Wireless Sensor Network[J]. Wireless Personal Communications, 2021, 122(3): 2667-2685.

[44]张育芝，张效民，王安义等. 水声通信网络信道建模与仿真研究进展[J]. 科学技术与工程, 2021, 21（04）: 1249-1261.

[45]Stojanovic M. On the relationship between capacity and distance in an underwater acoustic communication channel[J]. ACM SIGMOBILE Mobile Computing and Communications Review, 2007, 11(4): 34-43.

[46]Brekhovskikh L M, Lysanov Y P. Fundamentals of ocean acoustics[M]. Springer Science and Business Media, 2003.

[47]Faheem M, Tuna G, Gungor V C. LRP: Link quality‐aware queue‐based spectral clustering routing protocol for underwater acoustic sensor networks[J]. International Journal of Communication Systems, 2017, 30(12): e3257.

[48]Han X, Yin J, Tian Y, et al. Underwater acoustic communication to an unmanned underwater vehicle with a compact vector sensor array[J]. Ocean Engineering, 2019, 184: 85-90.

[49]Dorigo M, Birattari M, Stutzle T. Ant colony optimization[J]. IEEE computational intelligence magazine, 2006, 1(4): 28-39.

[50]Liu L, Wang R, Xiao F. Topology control algorithm for underwater wireless sensor networks using GPS-free mobile sensor nodes[J]. Journal of Network and Computer Applications, 2012, 35(6): 1953-1963.

[51]Pawlowicz R, Beardsley B, Lentz S. Classical tidal harmonic analysis including error estimates in MATLAB using T_TIDE[J]. Computers and Geosciences, 2002, 28(8): 929-937.

﹀
中图分类号：

 <font color='red'></font>T<font color='red'></font>N<font color='red'></font>9<font color='red'></font>2<font color='red'></font>9<font color='red'></font>.<font color='red'></font>3<font color='red'></font>    

开放日期：

 <font color='red'></font>2<font color='red'></font>0<font color='red'></font>2<font color='red'></font>4<font color='red'></font>-<font color='red'></font>0<font color='red'></font>6<font color='red'></font>-<font color='red'></font>1<font color='red'></font>3<font color='red'></font>    

无标题文档

   建议浏览器： 谷歌 火狐 360请用极速模式，双核浏览器请用极速模式　

杭州麦达电子有限公司版权所有 Copyright 2022 All Rights Reserved