当前我国煤矿智能化建设面临“采掘失衡”的挑战，虽然掘进工作面智能化水平不断提高，但施工过程中仍高度依赖于人工对设备的控制。掘进工作环境恶劣，粉尘含量高。现有掘进装备自动控制方法及技术难以充分应对复杂多变的环境，容易导致超挖或欠挖；并且，随着装备智能化程度的提升，控制程序不经充分验证则难以保障其性能与安全性，而井下调试成本过高并伴有诸多安全风险，发展掘进装备的智能化决策控制与高效调试对于推进智慧矿山的建设具有关键作用。因此，本文以巷道断面成型截割过程中掘进装备的决策控制与虚拟调试方法为研究对象，借助数字孪生、三维重建、深度强化学习、掘进装备控制等先进技术，通过构建掘进装备智能体、虚拟巷道场景，建立虚拟调试平台并验证断面成型截割自主决策控制系统，实现掘进装备断面成型的自主决策控制与虚拟调试。 

针对现有巷道掘进装备决策能力低的问题，提出一种基于深度强化学习改进PPO算法的悬臂式掘进机决策控制方法。进行复杂工况未知环境下规定步长内的截割部控制自主决策，结合系统规划的断面成型的高精度截割路径，使用深度强化学习算法补充轨迹时间、速度信息并进行轨迹跟踪，使其能够根据复杂的井下环境与机身状态自动规划最优的截割轨迹，并实时调整掘进装备的运动状态以适应环境与掘进装备状态变化。

针对掘进装备在复杂多变的工况环境下自动成型控制调试周期长、安全风险高、难度大等问题，本文提出了一种基于数字孪生的掘进装备自动成形控制虚拟调试方法。借助RTAP MAP技术重构巷道三维模型，并构建虚拟掘进机模型，结合虚拟传感器映射方法，在Unity3D平台建立掘进工作面虚拟调试场景。建立控制流程记录与评价方法，为机器学习模型针对具体巷道的二次训练提供大量高质量反馈数据。

针对远程状态监测与人机交互不够直观的问题，结合掘进装备智能体与虚拟巷道场景构建掘进装备决策控制数字孪生体模型，动态、高精度地映射掘进巷道装备与环境交互状况，设计人机友好的异常状况人员干预功能，并集成自主决策控制算法到人机交互平台中，以虚实数据交互的方法，实现远程巷道断面成型截割控制过程中的装备自主决策、状态监测与人工干预。

最后，通过搭建系统实验平台，针对断面成型截割决策控制方法和具体工作面环境进行虚拟调试应用实验，对自主决策控制程序进行了现实迁移训练，并验证调试改进效果。实验结果表明，自主决策控制方法可确保在机身偏移工况下的断面成型效果，并且其自主决策巷道断面成型截割控制精度符合使用要求；证明虚拟调试功能模块能够虚拟调试方法能够验证并改进断面成型自动控制程序，确保掘进作业的顺利进行，实现低成本的实现煤矿掘进装备控制程序试错，从而提高掘进装备智能化水平。通过自主决策控制配合人工干预，提高掘进效率和安全性。

The current challenge faced by the intelligent construction of coal mines in China is "mining imbalance",where the level of automation in excavation work has been gradually increasing, but it still relies heavily on manual control of equipment. The excavation environment is harsh, with high levels of dust. The existing methods and technologies for automatic control of excavation equipment are difficult to cope with complex and variable environments, leading to either over-excavation or under-excavation. As the level of equipment intelligence increases, the performance and safety of unverified control programs cannot be guaranteed, and the cost of on-site commissioning is high, carrying multiple safety risks. The development of intelligent decision-making control and efficient commissioning of excavation equipment is crucial for the construction of smart coal mines.

This study focuses on the decision-making control and virtual commissioning method of the excavation equipment during the cross-section formation process. By using advanced technologies such as digital twins, 3D reconstruction, deep reinforcement learning, and excavation equipment control, a virtual commissioning platform is constructed to verify the autonomous decision-making control system for cross-section formation cutting. This approach enables the intelligent and modern excavation of coal mines.

To address the low decision-making ability of existing excavation equipment, this study proposes an improved PPO algorithm based on deep reinforcement learning for the decision-making control of a cantilever excavator in complex working conditions. The algorithm controls the excavation process and automatically plans the optimal cutting trajectory combined with the high-precision cross-section formation planning system. The deep reinforcement learning algorithm supplements the trajectory information and tracks the trajectory to adapt to complex underground environments and changes in the status of the excavator.

To solve the problem of long commissioning cycles, high safety risks, and difficulties in automatic formation control of excavation equipment in complex working conditions, this study proposes a virtual commissioning method for excavation equipment based on digital twins. By using the RTAP MAP technology to rebuild the 3D model of the tunnel and establishing a virtual excavator model, a virtual commissioning scene is built in Unity3D. The control flow recording and evaluation method provides high-quality feedback data for the machine learning model retraining specific to each tunnel.

To solve the problem of insufficient remote monitoring and human-machine interaction, a digital twin model of the excavation equipment decision-making control system is built to map the interaction status between the excavator and the environment in real time. The design includes a friendly intervention function for abnormal situations, integrating the autonomous decision-making control algorithm into the system. Through virtual data interaction, it realizes remote cross-section formation cutting control with autonomous decision-making and manual intervention.

Finally, an experimental platform was set up to conduct virtual commissioning applications experiments on specific working face environments, verifying the improvement effect of the self-decision program retraining through actual migration training. The results show that the autonomous decision-making control method can ensure the cross-section formation effect under the condition of machine offset, proving that the virtual commissioning function module can effectively verify and improve the automatic control program of cross-section formation cutting, ensuring the smooth progress of excavation operations and achieving cost-effective commissioning of coal mine excavation equipment control programs, improving the level of intelligence and efficiency of excavation operations while enhancing their safety.

[1]王国法,赵国瑞,任怀伟.智慧煤矿与智能化开采关键核心技术分析[J].煤炭学报,2019,44(01)：34-41.

[2]王虹,王步康,张小峰,等.煤矿智能快掘关键技术与工程实践[J].煤炭学报,2021,46(07):2068-2083.

[3]王国法,张建中,薛国华,等.煤矿回采工作面智能地质保障技术进展与思考[J].煤田地质与勘探,2023,51(02)：12-26.

[4]Zhou M, Yan J, Feng D. Digital twin framework and its application to power grid online analysis[J]. CSEE Journal of Power and Energy Systems, 2019, 5(3): 391-398.

[5]李刚.基于数字孪生的工业机器人三维可视化监控[J].中国高新科技, 2023(5):40-42.

[6]Lu Q C, Parliksd A K, Woodall P, et al. Developing a digital twin at building and city lev-els: case study of West Cambridge campus[J]. Journal of Management in Enginneering, 2020, 36(3): 0502004.

[7]王妙云,张旭辉,马宏伟等.远程控制综采设备碰撞检测与预警方法[J].煤炭科学技术,2021,49(09):110-116.

[8]Liu Q, Zhang H, Leng J W, et al. Digital twin-driven rapid individualized designing of au-tomated flow-shop manufacturing system[J]. International Journal of Production Research, 2019, 57(12): 3903-3919.

[9]Bao J, Guo D S, Li J, et al. The modelling and operations for the digital twin in the conte-xt of manufacturing[J]. Enterprise Information Systems, 2019, 13(4): 534-556.

[10]谢苗,李晓婧,刘治翔.基于PID的掘进机横摆速度智能控制[J].机械设计与研究,2019,35(01):125-127+132.

[11]杨春雨,张鑫.煤矿机器人环境感知与路径规划关键技术[J].煤炭学报,2022,47(07)：2844-2872.

[12]宁振波.《智能制造的本质》[J]. 自动化博览, 2021, 38(12): 7.

[13]毛清华,陈磊,闫昱州,等.煤矿悬臂式掘进机截割头位置精确控制方法[J].煤炭学报,2017,42(S2):562-567.

[14]张旭辉, 杨文娟, 薛旭升, 等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报, 2022, 47(01): 579-597.

[15]谢嘉成,郑子盈,王学文,等.基于工业元宇宙的综采工作面虚实融合运行模式初步探索[J].煤炭科学技术, 2023, 51(10):266.DOI:10.13199/j.cnki.cst.2022-1864.

[16]丁恩杰, 俞啸, 夏冰, 等. 矿山信息化发展及以数字孪生为核心的智慧矿山关键技术[J]. 煤炭学报, 2022, 47(01): 564-578.

[17]邢震.面向智能矿山的数字孪生技术研究进展[J].工矿自动化, 2024, 50(3):22-34, 41.DOI:10.13272/j.issn.1671-251x.2024010079.

[18]陶飞, 马昕, 胡天亮, 等. 数字孪生标准体系[J]. 计算机集成制造系统, 2019, 25(10): 2405-2418BAE H,KIM G,KIM J,et al.Multi-robot path planning method using reinforcement learni-ng[J]. AppliedSciences,2019,9(15)：3057

[19] Farhadi A, Lee S K H, Hinchy E P, et al. The development of a digital twin framework fo-r an industrial robotic drilling process[J]. Sensors, 2022, 22(19): 7232.

[20]杨健健,张强,吴淼,等.巷道智能化掘进的自主感知及调控技术研究进展[J].煤炭学报,2020,45(6):2045-2055.

[21]张旭辉,赵建勋,杨文娟,等.悬臂式掘进机视觉导航与定向掘进控制技术[J].煤炭学报,2021,46(07):2186-2196.

[22]张旭辉,王甜,张超,等.数字孪生驱动的悬臂式掘进机虚拟示教记忆截割方法[J].煤炭学报,2023,48(11):4247-4260.

[23]王苏彧,马登成,任泽,等.悬臂式掘进机断面成型轨迹多目标优化方法研究[J].仪器仪表学报,2021,41(08):183-192.

[24]张旭辉,吕欣媛,王甜,等.数字孪生驱动的掘进机器人决策控制系统研究[J].煤炭科学技术,2022,50(7):36-49.

[25]王国法,王虹,任怀伟,et al.智慧煤矿2025情景目标和发展路径[J].煤炭学报, 2018, 43(2):11.DOI:10.13225/j.cnki.jccs.2018.0152.

[26]陆新时,马嵩华,胡天亮.基于数字孪生的力能控制式压力机虚拟调试[J].小型微型计算机系统, 2022(043-007).

[27]樊启高,李威,王禹桥,等.一种采用灰色马尔科夫组合模型的采煤机记忆截割算法[J].中南大学学报(自然科学版),2011,42(10):3054-3058.

[28]李娟莉,沈宏达,谢嘉成,et al.基于数字孪生的综采工作面工业虚拟服务系统[J].计算机集成制造系统, 2021, 27(2):11.

[29]Huang X, Liu Q, Shi K, et al. Application and prospect of hard rock TBM for deep roadway construction in coal mines[J]. Tunnelling and Underground Space Technology, 2018, 73: 105-126.

[30]葛兆亮.基于采煤机绝对位姿的自适应控制技术研究[D].中国矿业大学,2015.

[31]王丹.纵轴式硬岩掘进机截割机构的力学性能与参数优化[D].辽宁工程技术大学,2009.

[32]HU Xingtao,ZHU Tao,SU Jimin,et al. Key technology of intelligent drivage perception in- coal mine roadway[J]. Journal of China Coal Society,2021,46(7)：2123-2135.

[33]马宏伟,张璞,毛清华,周颖.基于捷联惯导和里程计的井下机器人定位方法研究[J].工矿自动化,2019,45(04):35-42.

[34]雷孟宇, 张旭辉, 杨文娟, 等. 煤矿掘进设备视觉位姿检测与控制研究现状与趋势[J]. 煤炭学报: 1-14[2022-03-17].

[35]杜春晖. 基于多技术融合的煤矿井下采掘运输设备防碰撞系统[J]. 煤炭学报, 2020, 45(S2): 1060-1068.

[36]李小华,杨瑞芳,刘辉,等.一类机械臂系统自适应有限时间有界H∞跟踪控制[J].控制理论与应用, 2021.

[37]Fan Y, An Y, Wang W, et al. TS fuzzy adaptive control based on small gain approach for an uncertain robot manipulators[J]. International Journal of Fuzzy Systems, 2020, 22(3): 930-942.

[38]谢苗,李玉岐,侯文博,等.掘支锚联合机组双截割部截割轨迹控制方法[J].辽宁工程技术大学学报(自然科学版),2019,38(04):339-344.

[39]孙军,张鹏,沈卓群,等.LabVIEW环境下的机械臂轨迹跟踪控制算法研究[J].机械设计与制造, 2020(6):4.

[40]孙彧,曹雷,陈希亮,等.多智能体深度强化学习研究综述[J].计算机工程与应用, 2020, 56(5):12.

[41]Fu X, Peng J. Iterative learning control for UAVs formation based on point-to-point traje-ctory update tracking[J]. Mathematics and Computers in Simulation, 2023, 209: 1-15.

[42]王刚,宋英杰,唐武生,等.基于迭代学习的三自由度机械臂轨迹跟踪控制[J].吉林大学学报(信息科学版),2021,39(04):389-396.

[43]赵毓, 管公顺, 郭继峰, 等. 基于多智能体强化学习的空间机械臂轨迹规划[J]. 航空学报, 2021, 42(01): 266-276.

[44]孙世光, 兰旭光, 张翰博, 等. 基于模型的机器人强化学习研究综述[J]. 模式识别与人工智能, 2022, 35(01): 1-16.

[45]王学宁. 策略梯度增强学习的理论、算法及应用研究[D]. 长沙: 国防科学技术大学, 2006.

[46]Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXi-v: 1707.06347, 2017

[47]Heess N,Sriram S,Lemmon J,et al. Emergence of locomotion behaviours in rich environ-ments[J]. arXiv: 1707.06347, 2017.

[48]李跃, 邵振洲, 赵振东, 等. 面向轨迹规划的深度强化学习奖励函数设计[J]. 计算机工程与应用, 2020, 56(02): 226-232.

[49]Bae H, Kim G, Kim J, et al. Multi-robot path planning method using reinforcement learni-ng[J]. Applied Sciences, 2019, 9(15): 3057.

[50]Liu S, Wang L, Wang X V, et al. A framework of data-driven dynamic optimisation for smart production logistics[C]//Advances in Production Management Systems. Towards Smart and Digital Manufacturing: IFIP WG 5.7 International Conference, APMS 2020, Novi Sad, Serbia, August 30–September 3, 2020, Proceedings, Part II. Springer International Publishing, 2020: 213-221.

[51]Kher A A, Yerpude R R. Modeling accident data for decision support in underground coal mines[J]. Int. J. Eng. Res. Technol.(IJERT), 2014, 3(7): 654-657.

[52]MAzurkiewicz D. Computer-aided maintenance and reliability management systems for conveyor belts[J]. Eksploatacja i Niezawodność, 2014, 16(3).

[53]方新秋,冯豪天,梁敏富,等.煤矿开采光纤光栅智能感知与安全决策关键技术研究[J].中国煤炭,2022,48(11):46-56.

[54]Perrusquía A, Yu W, Li X. Multi-agent reinforcement learning for redundant robot contro-l in task-space[J]. International Journal of Machine Learning and Cybernetics, 2021, 12: 231-241.

[55]Meyn S. Control systems and reinforcement learning[M]. Cambridge University Press, 2022.

[56]Song Y, Romero A, Müller M, et al. Reaching the limit in autonomous racing: Optimal co-ntrol versus reinforcement learning[J]. Science Robotics, 2023, 8(82): eadg1462.

[57]Chai R, Niu H, Carrasco J, et al. Design and experimental validation of deep reinforcement learning-based fast trajectory planning and control for mobile robot in unknown environment[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 35(4): 5778-5792.

[58]Yang C, Peng G, Li Y, et al. Neural networks enhanced adaptive admittance control of optimized robot–environment interaction[J]. IEEE transactions on cybernetics, 2018, 49(7): 2568-2579.

[59]Holubek R, Delgado Sobrino D R, Košťál P, et al. Offline programming of an ABB robot using imported CAD models in the RobotStudio software environment[J]. Applied Mechanics and Materials, 2014, 693: 62-67.

[60]Sebastian Meyer, Renner P, Lier F, et al. Improving human-robot handover research by mixed reality techniques[J]. VAM-HRI, 2018: 2018-03.

[61]Diego V, Peter W. SYSTEM FOR VIRTUAL COMMISSIONING:, EP3224681A1[P]. 2017.

[62]Jain A, Vera D A, Harrison R. Virtual commissioning of modular automation systems[J]. IFAC Proceedings Volumes, 2010, 43(4): 72-77.

[63]高赟,成哲.虚拟调试技术在某车间输送系统的应用[J].工业控制计算机, 2023, 36(6):28-29.

[64]Hoppenstedt B, Witte T, Ruof J, et al. Debugging quadrocopter trajectories in mixed reality[C]//Augmented Reality, Virtual Reality, and Computer Graphics: 6th International Conference, AVR 2019, Santa Maria al Bagno, Italy, June 24–27, 2019, Proceedings, Part II 6. Springer International Publishing, 2019: 43-50.

[65]马飞,代锟,孙巍伟.基于数字孪生的物流拣选虚拟调试系统设计[J].机床与液压, 2023, 51(16):95-100. James T.Klosowski,Martin Held,Joseph S.B.Mitchell.ed al.Efficient Collision Detection Using Bounding Volume Hierarchies of k-DOPs[J].IEEE Trans.Vis.Comput.Graph.1998,4(1).

[66]Echevarria P, Aldekoa E, Jugo J, et al. Superconducting radio-frequency virtual cavity for control algorithms debugging[J]. Review of Scientific Instruments, 2018, 89(8).

[67]Liu J, Zhang K. Design and Simulation Debugging of Automobile Connecting Rod Production Line Based on the Digital Twin[J]. Applied Sciences, 2023, 13(8): 4919.

[68]田立勇,戴渤鸿,王启铭.基于采煤机摇臂销轴多应变数据融合的煤岩识别方法[J].煤炭学报, 2020, 45(3):8.

[69]施俊屹、查富生、孙立宁、郭伟、王鹏飞、李满天.移动机器人视觉惯性SLAM研究进展[J].机器人, 2020, 42(6):15.

[70]邓仕超,陈艺海,黄扬,等.基于结构光三维重建系统的改进相位研究[J].组合机床与自动化加工技术, 2021(11):5.