目前掘进工作面中悬臂式掘进机断面截割配合人工钻锚作业方式仍然占一定的比例，悬臂式掘进机适应性高，适合煤岩硬度高的巷道施工，但掘进效率低且危险系数高，存在很大的安全隐患，因此提升其机身定位、导向导航和巷道成形截割的智能化水平迫在眉睫。课题组前期研发了基于激光合作标靶的悬臂式掘进机视觉定位方法，现场应用中效果突出，但是该方法中的合作标靶要随着推进距离不断前向移动，且每次需要通过标定来获得标靶在巷道坐标系下的位姿数据，增大了现场使用的难度。因此，本文提出一种视觉里程计与惯导融合的掘进机定位方法，通过无合作标靶的视觉里程计测量掘进机位姿，同时借助巷道空间结构对掘进机的空间数据进行约束，实现掘进机机身定位数据优化，并结合联邦滤波器方法实现掘进机位姿的稳定测量，有效提升巷道智能掘进的智能化水平。

目前掘进机视觉定位方法采用特制标靶作为特征，存在频繁移动标靶和视觉标定方面的难题，论文提出基于视觉里程计的掘进机机身无标靶位姿测量方法。通过建立相机投影和畸变模型求解相机内参，借助改进的ORB特征检测方法提取环境特征并进行特征点的描述，采用快速最近邻方法完成特征匹配并利用随机采样一致性算法进行误匹配点对的剔除。建立图优化模型，利用特征点对相机位姿进行非线性优化，达到位姿局部数据优化的目的，从而实现对掘进装备位姿的求解。

针对视觉里程计长时间运行存在的累计误差问题，提出了基于巷道空间结构约束的掘进机位姿测量优化方法。首先分析掘进巷道的空间环境并总结其环境特征，结合井巷工程布置和工作流程，利用煤矿工作面巷道空间结构和现场布置的反光管建立基于标签的词袋模型，然后设置约束关键帧阈值降低误匹配率，采用基于汉明距离的K均值层次聚类算法加快匹配速度，最终实现对定位数据的矫正。

针对煤矿视觉定位时存在视觉特征丢失、相机遮挡等情形，且定位数据稳定性受影响的问题，提出了一种基于视觉里程计和惯导融合的掘进机定位方法。首先建立视觉惯导联邦滤波器结构，利用子滤波器的滤波值和估计误差状态矢量进行逐层融合；然后建立惯导坐标系，基于零速修正方法纠正惯导数据；最后设计视觉惯导模式切换过程，提高了算法框架的扩展能力与容错性能。

最后，搭建实验平台验证上述理论方法，并对系统测量性能进行评价。通过对视觉里程计与惯导融合的定位方法的功能和定位性能进行实验验证，结果表明，基于视觉里程计与惯导融合的掘进机定位方法，在保证无合作标靶视觉定位的同时可实现较好的定位精度。

Currently, the cantilevered shearer combined with manual drilling and anchoring operations still accounts for a certain proportion in the excavation of working faces. Cantilevered shearer has high adaptability and is suitable for tunnel construction in high coal and rock hardness conditions. However, it has low excavation efficiency and high risk factors, posing significant safety hazards. Therefore, it is imperative to enhance the intelligence level of its body positioning, guidance and navigation, as well as tunnel shaping and cutting. The research team has developed a visual positioning method for cantilevered shearer based on cooperative laser target. This method has shown outstanding results in field applications. However, the cooperative target used in this method needs to be continuously moved forward with the advancing distance, and each time the pose data of the target in the tunnel coordinate system needs to be obtained through calibration, which increases the difficulty of on-site use. Therefore, this thesis proposes a positioning method for the shearer based on the fusion of visual odometry and inertial navigation. It utilizes visual odometry without cooperative targets to measure the pose of the shearer, while leveraging the spatial structure of the tunnel to constrain the spatial data of the shearer. This optimization of the shearer's body positioning data is achieved. Furthermore, the proposed method combines the federated filter approach to achieve stable measurement of the shearer's pose, effectively improving the level of intelligent tunnel excavation.

At present, the visual positioning method of roadheader is characterized by special target, which has the problems of frequent moving target and visual calibration. This thesis proposes a targetless positioning method of roadheader based on visual odometer. The camera internal parameters are solved by establishing the camera projection and distortion model. The improved ORB feature detection method is used to extract environmental features and describe feature points. The fast nearest neighbor method is used to complete feature matching and the random sampling consistency algorithm is used to eliminate mismatched point pairs. A graph optimization model is established, and the camera pose is nonlinearly optimized by using feature points to achieve the purpose of local data optimization of pose, so as to solve the pose of tunneling equipment.

Aiming at the cumulative error problem of long-term operation of visual odometer, a positioning optimization method of roadheader based on spatial structure constraint of roadway is proposed. Firstly, the spatial environment of the excavation roadway is analyzed and its environmental characteristics are summarized. Combined with the engineering layout and workflow of the roadway, the label-based bag-of-words model is established by using the spatial structure of the roadway in the coal mine working face and the reflective tube arranged on site. Then, the threshold of the constraint key frame is set to reduce the mismatch rate. The K-means hierarchical clustering algorithm based on Hamming distance is used to speed up the matching speed, and finally the correction of the positioning data is realized.

Aiming at the problems of visual feature loss, camera occlusion and other situations in coal mine visual positioning, and the stability of positioning data is affected, a positioning method of roadheader based on visual odometer and inertial navigation fusion is proposed. Firstly, the visual inertial navigation federated filter structure is established, and the filter value of the sub-filter and the estimated error state vector are used for layer-by-layer fusion. Then the inertial navigation coordinate system is established, and the inertial navigation data is corrected based on the zero velocity correction method. Finally, the switching process of visual inertial navigation mode is designed to improve the expansion ability and fault tolerance of the algorithm framework.

Finally, an experimental platform is built to verify the above theoretical method and evaluate the measurement performance of the system. Through the experimental verification of the function and positioning performance of the positioning method based on the fusion of visual odometer and inertial navigation, the results show that the positioning method of roadheader based on the fusion of visual odometer and inertial navigation can achieve better positioning accuracy while ensuring the visual positioning of non-cooperative target.

[1] 国家统计局. 中国能源统计年鉴2020[M]. 北京: 中国统计出版社, 2021.

[2] 朱妍. 中国工程院院士康红普: “十四五”煤炭消费需多少产多少[N]. 中国能源报.

[3] 张博, 孙旭东, 刘颖, 等. 能源新技术新兴产业发展动态与2035战略对策[J]. 中国工程科学, 2020,22(2):38-46.

[4] 康红普, 王国法, 王双明, 等. 煤炭行业高质量发展研究[J]. 中国工程科学, 2021,23(5):130-138.

[5] 王国法, 刘峰, 庞义辉, 等. 煤矿智能化——煤炭工业高质量发展的核心技术支撑[J]. 煤炭学报, 2019,44(02):349-357.

[6] 王国法, 任怀伟, 赵国瑞, 等. 煤矿智能化十大“痛点”解析及对策[J]. 工矿自动化, 2021,47(06):1-11.

[7] 杨文娟, 张旭辉, 张超, 等. 基于三激光束标靶的煤矿井下长距离视觉定位方法[J]. 煤炭学报, 2022,47(02):986-1001.

[8] 沈阳, 王鹏江, 吉晓冬, 等. 二维里程辅助的掘进机自主导航方法研究[J]. 仪器仪表学报, 2021,42(11):96-105.

[9] 薛光辉, 李圆, 张云飞. 基于激光靶向跟踪的悬臂式掘进机位姿测量系统研究[J]. 工矿自动化, 2022,48(07):13-21.

[10] 薛光辉, 张云飞, 候称心, 等. 基于激光靶向扫描的掘进机位姿测量方法[J]. 煤炭科学技术, 2020,48(11):19-25.

[11] 陈慎金, 成龙, 王鹏江, 等. 基于激光测量技术的掘进机航向角精度研究[J]. 煤炭工程, 2018,50(7):107-110.

[12] 贾文浩, 陶云飞, 张敏骏, 等. 基于iGPS的煤巷狭长空间中掘进机绝对定位精度研究[J]. 仪器仪表学报, 2016,37(08):1920-1926.

[13] 吴淼, 贾文浩, 华伟, 等. 基于空间交汇测量技术的悬臂式掘进机位姿自主测量方法[J]. 煤炭学报, 2015,40(11):2596-2602.

[14] 刘一鸣, 刘万里, 张博渊, 等. 基于UWB的采煤机定位精度提升算法研究[J]. 工矿自动化, 2016,42(12):25-30.

[15] 刘超, 符世琛, 成龙, 等. 基于TSOA定位原理混合算法的掘进机位姿检测方法[J]. 煤炭学报, 2019,44(4):1255-1264.

[16] 符世琛, 成龙, 陈慎金, 等. 面向掘进机的超宽带位姿协同检测方法[J]. 煤炭学报, 2018,43(10):2918-2925.

[17] 齐保卫, 孟帅尧, 李东利, 等. 面向竖井掘进机的超宽带分层平面定位系统设计与应用[J]. 隧道建设（中英文）, 2022,42(6):1077-1083.

[18] 田伟琴, 田原, 贾曲, 等. 悬臂式掘进机导航技术研究现状及发展趋势[J]. 煤炭科学技术, 2022,50(3):267-274.

[19] Einicke G, Ralston J, Hargrave C, et al. Longwall mining automation an application of minimum-variance smoothing [Applications of Control][Z]. New York: IEEE, 2008: 28, 28-37.

[20] Shen Y, Wang P, Zheng W, et al. Error Compensation of Strapdown Inertial Navigation System for the Boom-Type Roadheader under Complex Vibration[J]. Axioms, 2021,10(3):224.

[21] 毛清华, 安炎基, 马宏伟, 等. 煤矿掘进装备光纤惯导多源异构噪声降噪方法研究[J]. 煤炭科学技术, 2023,51(02):390-399.

[22] 杨秀宇. 基于光电传感器的掘进机位姿测量方法[J]. 机械设计, 2022,39(07):117-121.

[23] 王学, 周红旭, 张雷, 等. 基于近红外双目立体视觉的悬臂式掘进机定位研究[J]. 工矿自动化, 2022,48(07):43-51.

[24] 张旭辉, 沈奇峰, 杨文娟, 等. 基于三激光点标靶的掘进机机身视觉定位技术研究[J]. 电子测量与仪器学报, 2022,36(06):178-186.

[25] 曾庆化, 罗怡雪, 孙克诚, 等. 视觉及其融合惯性的SLAM技术发展综述[J]. 南京航空航天大学学报, 2022,6(54):1007-1020.

[26] J. C, A. J D, J. M M M. Inverse Depth Parametrization for Monocular SLAM[J]. IEEE Transactions on Robotics, 2008,24(5):932-945.

[27] 朱晨, 王宇杰, 郭杭. 基于双目相机的SLAM回环检测改进算法研究[J]. 测控技术, 2023,42(01):113-118.

[28] Khoshelham K, Elberink S O. Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications[J]. Sensors (Basel, Switzerland), 2012,12:1437-1454.

[29] Forster C, Pizzoli M, Scaramuzza D. SVO: Fast semi-direct monocular visual odometry[J]. 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014:15-22.

[30] 田立勇, 马春莹. 基于LINS/激光传感器的掘进机姿态识别系统[J]. 传感器与微系统, 2022,41(05):91-94.

[31] 马宏伟, 张璞, 毛清华, 等. 基于捷联惯导和里程计的井下机器人定位方法研究[J]. 工矿自动化, 2019,45(04):35-42.

[32] 刘超, 符世琛, 成龙, 等. 基于TSOA定位原理混合算法的掘进机位姿检测方法[J]. 煤炭学报, 2019,44(04):1255-1264.

[33] Wan J, Zhang X, Zhang C, et al. Vision and Inertial Navigation Combined-Based Pose Measurement Method of Cantilever Roadheader[J]. Sustainability (Basel, Switzerland), 2023,15(5):4018.

[34] 刘豪. 捷联惯导与里程计组合的矿用掘进机自主导航定位系统[D]. 重庆大学, 2020.

[35] 马宏伟, 毛金根, 毛清华, 等. 基于惯导/全站仪组合的掘进机自主定位定向方法[J]. 煤炭科学技术, 2022,50(08):189-195.

[36] 卢新明, 闫长青, 袁照平. 掘进机精准定位方法与掘进机器人系统[J]. 通信学报, 2020,41(02):58-65.

[37] Yang W, Zhang X, Ma H, et al. Laser Beams-Based Localization Methods for Boom-Type Roadheader Using Underground Camera Non-Uniform Blur Model[J]. IEEE access, 2020,8:190327-190341.

[38] 杨文娟. 煤矿智能掘进机位姿视觉测量方法及系统研究[D]. 西安科技大学, 2021.

[39] 毛清华, 张旭辉, 马宏伟, 等. 多传感器信息的悬臂式掘进机空间位姿监测系统研究[J]. 煤炭科学技术, 2018,46(12):41-47.

[40] Yang W, Zhang X, Ma H, et al. Non-central refractive camera calibration using co-planarity constraints for a photogrammetric system with an optical sphere cover[J]. Optics and Lasers in Engineering, 2021,139:106487.

[41] Gao X, Tao Z. Unsupervised learning to detect loops using deep neural networks for visual SLAM system[J]. Autonomous Robots, 2017,41:1-18.

[42] Zhang G, Zhang T, Zhang C. Accurate real-time SLAM based on two-step registration and multimodal loop detection[J]. Measurement science & technology, 2023,34(2):25201.

[43] Yang L, Chao P C, Nizamuddin M, et al. Deep Neural Network–Based Loop Detection for Visual Simultaneous Localization and Mapping Featuring Both Points and Lines[J]. Advanced Intelligent Systems, 2020,2(1).

[44] Magnusson M, Andreasson H, Nüchter A, et al. Appearance-based loop detection from 3D laser data using the normal distributions transform[J]. 2009 IEEE International Conference on Robotics and Automation, 2009:23-28.

[45] He L, Wang X, Zhang H. M2DP: A novel 3D point cloud descriptor and its application in loop closure detection[J]. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016:231-237.

[46] Muhammad N, Lacroix S. Loop closure detection using small-sized signatures from 3D LIDAR data[J]. 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics, 2011:333-338.

[47] 王朋, 郝伟龙, 倪翠, 等. 视觉SLAM方法综述[J]. 北京航空航天大学学报, 2022:1-9.

[48] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004,60:91-110.

[49] Bay H, Ess A, Tuytelaars T, et al. Speeded-Up Robust Features (SURF)[J]. Computer Vision and Image Understanding, 2007,110(3).

[50] Rublee E, Rabaud V, Konolige K, et al. ORB: An efficient alternative to SIFT or SURF[J]. 2011 International Conference on Computer Vision, 2011:2564-2571.

[51] 李同, 张奇志. 基于ORB词袋模型的SLAM回环检测研究[J]. 信息通信, 2017(10):20-25.

[52] 林俊钦, 韩宝玲, 罗庆生, 等. 基于NDT匹配和改进回环检测的SLAM研究[J]. 光学技术, 2018,44(02):152-157.

[53] Duan R, Feng Y, Wen C. Deep Pose Graph-Matching-Based Loop Closure Detection for Semantic Visual SLAM[J]. Sustainability, 2022.

[54] 贾松敏, 李柏杨, 张国梁. 采用混合回环检测与闭环优化的TSDF地图创建[J]. 光学精密工程, 2018,26(06):1497-1506.

[55] 董蕊芳, 柳长安, 杨国田. 一种基于改进TF-IDF的SLAM回环检测算法[J]. 东南大学学报(自然科学版), 2019,49(02):251-258.

[56] 柴梦娜, 刘元盛, 任丽军. 基于激光点云NDT特征的两步回环检测[J]. 激光与红外, 2020,50(01):17-24.

[57] 周风余, 庄文密, 万方, 等. 基于路标与云架构的多机器人建图及融合方法[J]. 华中科技大学学报(自然科学版), 2020,48(11):30-36.

[58] 张康皓, 董希旺, 于江龙, 等. 多传感器融合状态估计方法综述[J]. 导航定位与授时, 2022,9(5):28-37.

[59] Willner D, Chang C B, Dunn K P. Kalman filter algorithms for a multi-sensor system[J]. 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes, 1976:570-574.

[60] Bar-Shalom Y. On the track-to-track correlation problem[J]. IEEE Transactions on Automatic Control, 1981,26:571-572.

[61] Mhatre V P, Rosenberg C P. Homogeneous vs heterogeneous clustered sensor networks: a comparative study[J]. 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577), 2004,6:3646-3651.

[62] Zhou H, Taj M, Cavallaro A. Target Detection and Tracking With Heterogeneous Sensors[J]. IEEE Journal of Selected Topics in Signal Processing, 2008,2:503-513.

[63] Liang J, Wang Z. Adaptive sensor selection for multitarget detection in Heterogeneous Sensor Networks[J]. 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery, 2010,2:972-976.

[64] Yang C, Kaplan L M, Blasch E, et al. Optimal Placement of Heterogeneous Sensors for Targets with Gaussian Priors[J]. IEEE Transactions on Aerospace and Electronic Systems, 2013,49:1637-1653.

[65] 周志鑫. 船舶动力定位位置参考系统信息融合研究[D]. 大连海事大学控制工程, 2017.

[66] 李艳华, 房建成, 贾志凯. INS/CNS/GPS组合导航系统仿真研究[J]. 中国惯性技术学报, 2002(06):7-12.

[67] Hofmann-Wellenhof B, Lichtenegger H I M, Wasle E. GNSS - Global Navigation Satellite Systems, 2008[C].

[68] 王浩然, 王宏伟, 李正龙, 等. 基于捷联惯导与差速里程计的掘进机组合定位方法[J]. 工矿自动化, 2022,48(09):148-156.

[69] Bai X, Xu H, Li J, et al. Coal mine personnel positioning algorithm based on improved adaptive unscented Kalman filter with wireless channel fading and unknown noise statistics[J]. Transactions of the Institute of Measurement and Control, 2021,44:1217-1227.

[70] 吉晓冬, 瞿圆媛, 符世琛, 等. 基于SVD-Unscented卡尔曼滤波的掘进机行进调度纠偏研究[J]. 矿业科学学报, 2022,7(3):354-363.

[71] 崔玉明. 煤矿巷道掘进机视觉/惯性融合自主定位关键技术研究[D]. 中国矿业大学, 2021.

[72] 姚健. 基于SINS/WSN的掘进机定位定姿研究[D]. 中国矿业大学, 2021.

[73] Wu S, Wen W, Wang Y, et al. MVS-Pheno: A Portable and Low-Cost Phenotyping Platform for Maize Shoots Using Multiview Stereo 3D Reconstruction[J]. Plant Phenomics, 2020,2020.

[74] 柴宏红, 邵科, 于超, 等. 基于三维点云的甜菜根表型参数提取与根型判别[J]. 农业工程学报, 2020,36(10):181-188.

[75] 于合龙, 黄浦, 苏红宏, 等. 基于双目视觉的植物叶片三维形态与光照度同步测量[J]. 农业工程学报, 2016,32(10):149-156.

[76] Zennaro S, Munaro M, Milani S, et al. Performance evaluation of the 1st and 2nd generation Kinect for multimedia applications[J]. 2015 IEEE International Conference on Multimedia and Expo (ICME), 2015:1-6.

[77] Brown D. Close-Range Camera Calibration, 1971[C].

[78] Juarez-Salazar R, Zheng J, Díaz-Ramírez V H. Distorted pinhole camera modeling and calibration.[J]. Applied optics, 2020,59 36:11310-11318.

[79] 马科伟, 张锲石, 康宇航, 等. 移动机器人中视觉里程计技术综述[J]. 计算机工程, 2021,47(11):1-10.

[80] Wang Q, Cai M, Guo Z. An enhanced positioning technique for underground pipeline robot based on inertial Sensor/Wheel odometer[J]. Measurement, 2023.

[81] 张艳, 王宇. 基于视觉里程计的室内位姿测量技术研究[J]. 电子测量与仪器学报, 2022,36(6):73-81.

[82] 胡凯, 吴佳胜, 郑翡, 等. 视觉里程计研究综述[J]. 南京信息工程大学学报(自然科学版), 2021,13(03):269-280.

[83] Bay H, Tuytelaars T, Gool L V. SURF: Speeded Up Robust Features: European Conference on Computer Vision, 2006[C].

[84] Muja M, Lowe D G. Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration: International Conference on Computer Vision Theory and Applications, 2009[C].

[85] Kümmerle R, Grisetti G, Strasdat H M, et al. G2o: A general framework for graph optimization[J]. 2011 IEEE International Conference on Robotics and Automation, 2011:3607-3613.

[86] Burri M, Nikolic J, Gohl P, et al. The EuRoC micro aerial vehicle datasets[J]. The International Journal of Robotics Research, 2016,35:1157-1163.

[87] Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems[J]. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012:573-580.