煤炭作为我国重要传统能源行业，智能化发展是煤炭行业转型高质量发展的必由之路。为解决采掘失衡难题，掘进装备智能化水平亟待提升。虽然煤矿装备自动化、信息化取得很大进展，但传统井下设备位姿及巷道测量方法存在稳定性不足、全局定位困难等问题。论文提出一种单目视觉与激光雷达融合的巷道三维重建与装备定位总体方案，结合多传感器融合方法实现掘进装备位姿测量，构建二维栅格与三维点云一致的巷道地图，为掘进装备在煤矿复杂环境下的定位、定向提供丰富地图信息，完成巷道环境及装备位姿的可视化呈现，对实现井下少人甚至无人化智能高效生产具有重要意义和应用价值。

针对煤矿井下地图构建过程可能出现的点云运动畸变、计算实时性问题，建立基于激光雷达的巷道地图观测模型。采集激光雷达扫描数据并进行点云数据预处理，研究点云双边滤波算法和特征提取算法；通过PLICP点云匹配算法，建立最小化误差函数完成帧间位姿估计，获得机器人运动过程中不同视角下的多组统一点云，构建出坐标系一致的巷道二维栅格地图，保证激光雷达建图精度及效率，为联合地图构建奠定基础。

针对单目视觉测量过程遇到的特征不足、深度获取困难问题，提出基于单目视觉的掘进装备位姿测量方案。建立相机内参标定模型，求解相机内参数；借助ORB特征检测方法提取环境特征并进行特征点匹配，采用RANSAC算法完成误匹配点对的剔除。建立对极几何算法模型，求解优化之后的匹配点对，实现对相机帧间位姿估计。提出基于三角化深度计算方法的位姿解算模型，基于三维空间点坐标采用PNP解算方法实现对掘进装备位姿求解，以此提高视觉测量精度及稳定性，并为巷道三维重建奠定基础。

针对煤矿复杂环境下存在的视觉特征丢失、全局定位困难问题，提出基于单目视觉与激光雷达的松耦合SLAM算法框架，融合实现掘进装备同步定位与建图。建立单目视觉与激光雷达联合标定模型，求解其位姿转换矩阵；通过传感器时间软同步方法，借助线性插值法求解视觉对应时刻位姿，完成关键帧位姿匹配，实现传感器数据层融合；构建三维点云地图离线优化模型，通过最小化重投影误差优化三维点云地图。最后根据关键帧位姿匹配三维点云与二维栅格地图，完成巷道三维重建与掘进装备位姿解算。

最后，搭建实验平台，验证上述理论方法，对系统测量性能进行评价。论文分别对单目视觉和激光雷达SLAM方法的性能和组合SLAM方法的定位性能进行实验，结果显示组合的位姿解算方法可实现更好的定位精度。在楼道对整体巷道地图构建与装备定位技术进行实验，得到匹配的二维栅格地图、三维点云地图及掘进装备运动轨迹，验证巷道三维重建方法和装备定位技术的有效性。

As an important traditional energy industry in China, the demand of intelligent development of coal is directly related to the process of national economy and social intelligence. In order to solve the problem of mining imbalance, the intelligent level of tunneling equipment needs to be improved. Although automation and informatization of coal mine equipment made great progress. However, the traditional underground equipment posture and roadway measurement methods have problems such as insufficient stability and difficult global positioning. The paper proposed the overall plan of roadway 3D reconstruction and equipment positioning based on monocular vision and laser radar, use multi-sensor fusion method to realize tunneling equipment pose measurement, construction of roadway map with two-dimensional grid and three-dimensional point cloud, provide rich map information for positioning and orientation of tunneling equipment in complex environment of coal mine，complete the visualization of roadway environment and equipment pose, it is of great significance and application value to realize the intelligent and efficient production with fewer people even unmanned.

In view of the possible point cloud motion distortion and real-time calculation problems in the process of coal mine underground map construction, characteristic of roadway boundary, establish a model of roadway map observation for lidar. Gathering scanning data of laser radar and preprocess laser radar point cloud data, research on bilateral filtering algorithm and feature extraction algorithm. Through PLICP point cloud matching algorithm, establish minimizing error function to complete the inter-frame pose estimation, obtain unified point clouds from different perspectives in robot motion. Constructed 2D grid map of roadway with consistent coordinate system, ensure the accuracy and efficiency of laser radar mapping. lay the foundation for joint map building.

Aiming at the problems of insufficient features and difficult depth acquisition in monocular vision measurement process, proposed a tunneling equipment pose measurement scheme based on monocular vision. Establishing camera calibration model, calculate camera parameters. Extracting environmental features and matching feature points with ORB feature detection method, using RANSAC algorithm to eliminate mismatched points. Establishing The model of polar geometry algorithm to solve the optimized matching point pairs, realizing of pose estimation between camera frames. Proposing a pose solution model based on triangulation depth calculation method, based on the three-dimensional space point coordinates, use PNP solution method to solve the tunneling equipment pose, enhance visual measurement accuracy and stability, and lay the foundation for three-dimensional reconstruction of roadway.

Aiming at the problems of visual feature loss and positioning difficulty under global coordinate in complex environment of coal mine, proposed a framework of loosely coupled slam algorithm based on monocular vision and lidar, integration of tunneling equipment synchronous positioning and mapping system. Established a joint calibration model of monocular vision and laser radar and solved pose conversion matrix. Through sensor time soft synchronization method, with the help of the linear interpolation method, obtained the position and pose of the visual corresponding time, completing key frame pose matching, realizing sensor data lever fusion. Constructing 3D point cloud map offline optimization model, optimizing 3D point cloud map by minimizing reprojection error. According to the key frame pose matching, completing the matching of 3D point cloud and 2D grid map, and the 3D reconstruction of roadway and the solution of tunneling equipment pose are realized.

Finally, building the experimental platform to verify the theoretical method and evaluate the measurement performance of the system. The paper respectively verified the performance of monocular vision and lidar slam method, the positioning performance of combined salm method, the results show that the combined pose calculation method can achieve better positioning accuracy. Moving robot to verify the whole roadway map construction and equipment positioning technology in the corridor. Get aligned 2D grid maps and 3D point cloud maps and tunneling equipment motion trajectory, verify the effectiveness of 3D reconstruction method and equipment positioning technique.

[1] 刘峰,曹文君,张建明等.我国煤炭工业科技创新进展及“十四五”发展方向[J].煤炭学报,2021,46(01):1-15.

[2] 王国法,刘峰,孟祥军.煤矿智能化(初级阶段)研究与实践[J].煤炭科学技术,2019,47(08):1-36.

[3] 田子建,李宗伟,刘晓阳等. 基于电磁波及超声波联合测距的井下定位方法[J]. 北京理工大学学报2014,34(5):490-494.

[4] 刘永伟. 矿用悬臂式掘进机器人控制系统研制[D].西安:西安科技大学,2018.

[5] 叶伟.煤矿井下目标定位的研究现状与展望[J].中国矿业,2021,30(01):82-89+105.

[6] Reid P.B.Dunn M.T., Reid D.C., Ralston J.C. Real-World Automation: New Capabilities for Underground Longwall Mining[C]. In Proc. Australasian Conf. Robotics and Automation, 2010: 1-8.

[7] Amo I A D, Letamendia A, Diaux J. Nuevas resistencias comunicativas: la rebelión de los ACARP[J]. Revista Latina De Comunicación Social, 2014, 69:307-329.

[8] KHONZI HLOPHE,FRANCOIS DU PLESSIS.Implementation of an autonomous un-derground localization system[J]. Robotics and Mechatronics Conference.IEEE,2013,(6):1-34.

[9] 杨海,李威,罗成名等.基于捷联惯导的采煤机定位定姿技术实验研究[J].煤炭学报,2014,39(12)：2550-2556.

[10] 郭楠楠,申亮亮,邵会兵等.车载捷联惯导双里程计组合导航方法研究[J].导航定位与授时,2020,7(06):86-93.

[11] 薛光辉,张云飞,候称心等.基于激光靶向扫描的掘进机位姿测量方法[J].煤炭科学技术,2020,48(11):19-25.

[12] 刘超,符世琛,成龙等.基于TSOA定位原理混合算法的掘进机位姿检测方法[J].煤炭学报,2019,44(04):1255-1264.

[13] Yuxin Du.Visual measurement system for roadheaders pose detection in mines[J].Optical Engineering.2016,October.55(10):104107-1-104107-16.

[14] W. Yang, X. Zhang, H. Ma and G. Zhang, Laser Beams-Based Localization Methods for Boom-Type Roadheader Using Underground Camera Non-Uniform Blur Model, in IEEE Access, vol. 8, pp. 190327-190341, 2020, doi: 10.1109/ACCESS.2020.3032368.

[15] 薛旭升,张旭辉,毛清华等.基于双目视觉的掘进机器人定位定向方法研究[J].西安科技大学学报,2020,40(05):781-789.

[16] 朱信平,李睿,高娟等.基于全站仪的掘进机机身位姿参数测量方法[J].煤炭工程,2011(06):113-115.

[17] 张旭辉,刘博兴,张超等.掘进机全站仪与捷联惯导组合定位方法[J].工矿自动化,2020,46(09):1-7.

[18] 李传立,尚俊娜,李芳.里程计技术发展综述[J].软件导刊,2019,18(12):6-10.

[19] 杨文娟,张旭辉,马宏伟等.悬臂式掘进机机身及截割头位姿视觉测量系统研究[J].煤炭科学技术,2019,47(06):50-57.

[20] 赵顺顺. 基于线激光多目立体视觉的船舶曲板成形在位检测与自动调形系统开发[D].江苏:江苏大学,2020.

[21] Endres F, Hess J, Sturm J, et al. 2014, 3-Dmapping with an RGB-D camera[J]. IEEE transactions on robotics, 30(1): 177-187.

[22] 慈文彦,黄影平,胡兴.视觉里程计算法研究综述[J].计算机应用研究,2019,36(09):2561-2568.

[23] Paul R, Newman P. FAB-MAP 3D: Topological mapping with spatial and visual appearance [C]// Proc of IEEE International Conference on Robotics and Automation. Piscataway. Piscataway, NJ: IEEE Press, 2010: 2649-2656.

[24] 周朋博,刘晓峰,蔡国平.基于ORB-SLAM的低照度空间非合作目标的姿态估计[J].动力学与控制学报,2021,19(01):68-74.

[25] 赵玉琛,叶海峰,林靖宇.结合DBSCAN与PTAM算法的室内家居无标记增强现实系统[J].计算机应用研究,2020,37(S2):302-304.

[26] Yao Xiaoyong, Wu Pingdong, Huang Manling. Study on methods for synthesizing binocular stereoscopic images[C]. International Conference on Electrical and Control Engineering, 2011, 6127-6130.

[27] 卢亚兵. 基于双目视觉的实时定位与建图方法研究[D].哈尔滨:哈尔滨工业大学,2018.

[28] 吴东昱. RGBD-SLAM关键技术研究及应用[D].北京:北方工业大学,2019.

[29] Almeida J, Santos V M. Real time egomotion of a nonholonomic vehicle using L1DAR meas-urements[J]. Journal of Field Robotics, 2013, 30(1):129-141.

[30] Zhang J, Singh S. LOAM: Lidar Odornetry and Mapping in Real-time[C]// Robotics: Science and Systems Conference. 2014.

[31] Carlone, Luca,et al. "A Fast and Accurate Approximation for Planar Pose Graph Optimization.'' The International Journal of Robotics Research 33.7(2014):965-987.

[32] 晏小彬,彭道刚,戚尔江.基于地平面的单目视觉辅助激光雷达SLAM研究[J].光学学报,2020,40(24):173-183.

[33] 胡冬冬. 基于图像纹理信息的三维重建研究[D].武汉:华中科技大学,2017.

[34] 李在林. 基于多视图几何的三维重建技术研究[D].深圳:中国科学院大学(中国科学院深圳先进技术研究院),2020.

[35] R. Kataoka, R. Suzuki, Y. Ji, H. Fujii, H. Kono and K. Umeda, ICP-based SLAM Using LiDAR Intensity and Near-infrared Data, 2021 IEEE/SICE International Symposium on System Integration (SII), Iwaki, Fukushima, Japan, 2021, pp. 100-104.

[36] Senin N, Colosimo B M, Pacella M. Point set augmentation through fitting for enhanced ICP registration of point clouds in multisensor coordinate metrology[J]. Robotics and Computer-Integrated Manufacturing, 2013, 29(29).39-52.

[37] Soften Bouaziz, Andrea Tagliasacchi, Mark Pauly. Sparse Iterative Closest Point[J].Computer Graphics Forum, 2013, 32(5).113-123.

[38] 王浩,张生伟,徐恺.应用于图像匹配的实时自适应RANSAC算法研究[J/OL].电光与控制:2019,28(12):1-5.

[39] Acquiring 3D indoor environments with variability and repetition[J]. Young Min Kim,Niloy J. Mitra,Dong-Ming Yan,Leonidas Guibas. ACM Transactions on Graphics (TOG).2012 (6)

[40] 张伟.基于3D激光雷达的变电站巡检机器人定位与建图方法研究[D].华中科技大学,2018.

[41] Newcombe R A, Davison A J, lzadi S, et al. Kinect Fusion: Real-time dense surface mapping and tracking[C]. Mixed and augmented reality (ISMAR), 10th IEEE international symposium, 2011:127-136.

[42] Fuentes-Pacheco J. Visual simultaneous localization and mapping: a survey[M]. Kluwer Academic Publishers, 2015.

[43] D. Ferguson, A. Morris, D. Hahnel. An Autonomous Robotic System For Mapping Abandoned Mines. Proceedings of The 2003 IEEE International Conference on Robotics and Automation, Taipei, Taiwan, 2003, 9: 4270-427_5.

[44] Andreas Nuchter, Hartmut Surmann, Sebastian Thrun. 6D SLAM with an Application in Autonomous Mine Mapping. Proceedings of The 2004 IEEE International Conference on Robotics and Automation, New Orleans, LA, 2004, 4: 1998-2003.

[45] 陈学惠. 基于移动机器人双目视觉的井下三维场景重建方法研究[D].中国矿业大学(北京),2012.

[46] 陈凯,杨小聪,张达.采空区三维激光扫描变形监测系统[J].矿冶,2012,21(01):60-63.

[47] 叶瑾,许枫,杨娟,王佳维.一种基于多传感器的复合量测IMM-EKF数据融合算法[J].电子学报,2020,48(12):2326-2330.

[48] SOLOVIEV A, HAAG MLTD. Three-Dimensional Navigation with Scanning Ladars: Concept & Initial Verification[J]. IEEE Transactions on Aerospace and Electronic Systems, 2010,46(1): 1431.

[49] Zhang J,Singh S.Visual-lidar Odometry and Mapping: Low-drift, Robust, and Fast[C]. IEEE Conf on Robotics and Automation (ICRA). IEEE, 2018.

[50] 张毅,杜凡宇,罗元等.一种融合激光和深度视觉传感器的SLAM地图创建方法[J].计算机应用研究,2016,33(10):2970-2972+3006.

[51] 王光庭,曹凯,刘豪.基于激光雷达与视觉信息融合的SLAM方法[J].山东理工大学学报(自然科学版),2019,33(01):9-13+19.

[52] Zhang J, Singh S. Visual-lidar odometry and mapping: low-drift, robust, and fast[C]// IEEE International Conference on Robotics & Automation. IEEE, 2015.

[53] 吴伟坊. 基于机载LiDAR系统激光点云与图像点云的融合技术[D].广州:华南理工大学,2019.

[54] Wen Shuhuan, Othman Kamal Mohammed, Rad Ahmad B, etal. Indoor slam using laser and camera with closed-loop controller for nao humanoid robot[J]. Abstract and Applied Analysis, 2014,2014(1):1-8.

[55] Sun Fengchi, Zhou Yuan, Li Chao, etal. Research on activeslam with fusion of monocular vision and laser range data[A]. in the 8th world congress on intelligent control and automation[C]. 2010:6550-6554.

[56] Maitiniyazi Maimaitijiang,Abduwasit Ghulam. Unmanned Aerial System (UAS)-based phenotyping of soybean using multi-sensor data fusion and extreme learning machine[J]. ISPRS Journal of Photogrammetry and Remote Sensin. 2017.

[57] 周文文,万晓冬,李文.基于改进D-S证据理论的数据融合目标分类[J].半导体光电,2021,42(01):121-126.

[58] 沈志宏,赵子豪,王华进等.一种异构数据智能融合管理系统[J].软件学报,2021,32(03):763-780.

[59] Choi Dong Geol, Shim Inwook, Bok Yunsu, etal. Autonomous homing based on laser-camera fusion system [A]. in Ieee/rsj International Conference on Intelligent Robots andSystems [C]. Vilamoura, 2012:2512-2518.

[60] 高士岗,高登彦,欧阳一博等.中薄煤层智能开采技术及其装备[J].煤炭学报,2020,45(06):1997-2007.

[61] 梁倩倩. 三维点云滤波与特征点提取算法研究[D].长春:吉林大学,2018.

[62] 王子豪. 基于图像匹配的目标定位关键技术研究[D].成都:电子科技大学,2020.