~随着煤炭行业的发展，煤矿智能化已成为煤炭行业高质量发展的核心理念。近些年，在国家政策的鼓励下，煤矿智能化有了长足进步，但综掘工作面智能化水平仍然不足，安全事故频发。要实现综掘工作面智能化生产，必须要解决巷道三维重建、设备定位以及设备碰撞预警等技术难点，因此本文提出一种巷道视觉重建与设备碰撞预警方法。通过构建RTAB MAP三维稠密地图，并确定煤矿井下综掘设备的位姿，在Unity中建立综掘设备虚拟模型并同步地图数据与位姿数据，根据OBB包围盒距离关系，实现基于OBB包围盒碰撞预警方法，对实现煤矿智能化与无人化具有重要意义。
针对在煤矿井下复杂环境中巷道三维地图难以重构的问题，研究基于RGBD的RTAB MAP SLAM三维巷道重建方法。通过建立相机畸变模型并分析畸变对特征的影响，构建基于自适应天牛须搜索算法的相机内参优化模型，实现相机内参标定与畸变校正；通过直接法对相机进行跟踪，形成视觉里程计，利用Bundle Adjustment算法实现后端优化，根据相似度检测的方法完成RTAB MAP SLAM中回环检测，再通过点云拼接完成稠密三维点云地图构建，为三维巷道在Unity中的展示以及碰撞检测奠定了基础。
针对煤矿井下移动设备视觉位姿检测存在遮挡，且在粉尘浓度不同时会产生较大测量误差等问题，提出一种基于点、线特征的煤矿井下设备位姿测量方案。利用自适应阈值与最小边界矩形方法，通过椭圆拟合法获取光斑中心坐标，利用LSD梯度外接矩形和NAF方法确定激光束的中心线，结合最小二乘法提取出激光束直线特征；利用图像中的点、线特征信息，通过掘进设备位姿测量模型，解算出设备机身位姿，为碰撞检测提供数据支撑。
针对煤矿井下掘进设备机身视觉定位方法沿巷道掘进方向测量误差较大，难以满足碰撞检测要求的问题，研究基于标签的井下移动设备相对定位方法。通过Douglas-Peucker算法识别出贴标区域，根据k-NN特征模式识别算法获取设备名称，利用Hough直线拟合获取边界直线并求取交点，将其带入到基于三激光点特征的设备位姿检测模型中，计算出相邻设备间的相对位姿，为提高碰撞检测精度提供数据支撑。
针对煤矿井下设备碰撞检测效率低且实现困难等问题，提出一种基于OBB层次包围盒的碰撞检测方法。通过建立基于OBB层次包围盒碰撞检测框架，利用OBB包围盒三个轴向的最大投影值与最小投影值之差，计算出包围盒的边界，构建出OBB包围盒层次结构模型，根据包围盒之间的位置关系与设备碰撞预警阈值，初步实现基于OBB层次包围盒碰撞预警，为碰撞预警提供理论支撑，保证综掘工作面设备安全运行。
最后，搭建实验平台，验证上述方法的可行性。结果表明，RTAB MAP三维地图构建方法具有较好的地图重建精度，机身定位方法可以提供稳定有效的设备机身位姿数据，在Unity中能够对碰撞预警功能进行展示。
 

~With the development of coal industry, coal mine intelligence has become the core concept of high quality development of coal industry. In recent years, with the encouragement of national policies, the intelligentization of coal mines has made great progress, but the intelligent level of fully mechanized working face is still insufficient and safety accidents occur frequently. In order to realize the intelligent production of fully mechanized mining face, it is necessary to solve the technical difficulties of roadway three-dimensional reconstruction, equipment positioning and equipment collision warning. Therefore, this paper proposes a method of roadway visual reconstruction and equipment collision warning. By constructing RTAB MAP three-dimensional dense map and determining the pose of fully mechanized mining equipment in coal mine, the virtual model of fully mechanized mining equipment is established in Unity and the map data and pose data are synchronized. According to the distance relationship of OBB bounding box, the collision warning method based on OBB bounding box is realized, which is of great significance to realize intelligent and unmanned coal mine.
Aiming at the problem that 3D map of roadway is difficult to reconstruct in complex environment of coal mine, the 3D roadway reconstruction method of RTAB MAP SLAM based on RGBD is studied. By establishing the camera distortion model and analyzing the influence of distortion on the characteristics, the camera internal parameter optimization model based on the adaptive velvet search algorithm is constructed to achieve the camera internal parameter calibration and distortion correction. The camera is tracked directly to form a visual odometer, and the back-end optimization is realized by Bundle Adjustment algorithm. The loop detection in RTAB MAP SLAM is completed according to the similarity detection method, and then the dense 3D point cloud map is constructed by point cloud splicing, which lays a foundation for the display and collision detection of 3D roadway in Unity.
In view of the problems of occlusion in visual pose detection of underground mobile equipment in coal mines and large measurement error when dust concentration is different, a pose measurement scheme of underground equipment in coal mines based on point and line features is proposed. The adaptive threshold and the least boundary rectangle method are used to obtain the coordinates of the center of the spot by ellipse fitting method. The center line of the laser line is determined by the LSD gradient outer rectangle and NAF, and the linear characteristics of the laser beam are extracted by the least square method; Using the point and line feature information in the image, the pose measurement model of tunneling equipment is established, and the pose of the equipment fuselage is calculated, providing data support for collision detection.
Aiming at the problem that the measurement error of the visual positioning method of the fuselage of the underground tunneling equipment in coal mines along the tunneling direction is large, and it is difficult to meet the requirements of collision detection, the relative positioning method of underground mobile equipment based on tags is studied. The labeling area is identified by Douglas-Peucker algorithm, and the device name is obtained by k-NN feature pattern recognition algorithm. The Hough line fitting is used to obtain the boundary line and calculate the intersection point, which is substituted into the equipment pose detection model based on the characteristics of three laser points. The relative pose between adjacent equipment is calculated, which provides data support for improving the accuracy of collision detection.
Aiming at the problems of low efficiency and difficulty in realizing collision detection of underground equipment in coal mine, a collision detection method based on OBB hierarchical bounding box is proposed. By establishing the OBB-based hierarchical bounding box collision detection framework, the boundary of the bounding box is calculated by using the difference between the maximum projection value and the minimum projection value of the three axial directions of the OBB bounding box, and the hierarchical structure model of the OBB bounding box is constructed. By using the distance relationship between the bounding boxes, the equipment collision warning threshold is determined, and the collision warning of the OBB-based hierarchical bounding box is completed to ensure the safe operation of the equipment in the fully mechanized working face.
Finally, an experimental platform is built to verify the feasibility of the above method. The results show that the RTAB MAP three-dimensional map construction method has good map reconstruction accuracy. The fuselage positioning method can provide stable and effective posture data of the equipment fuselage, and realize the collision detection and early warning function in Unity.
 

[1] 钱鸣高, 许家林, 王家臣. 再论煤炭的科学开采[J]. 煤炭学报, 2018, 43(01): 1-13.

[2] 李梓佳. 中国的能源战略与能源金融分析[J]. 现代经济信息, 2020(07): 5+8.

[3] 王国法, 王虹, 任怀伟, 等. 智慧煤矿2025情景目标和发展路径[J]. 煤炭学报, 2018, 43(02): 295-305.

[4] 谭章禄, 马营营, 郝旭光. 智慧矿山标准发展现状及路径分析[J]. 煤炭科学技术, 2019, 47(03): 27-34.

[5] 武曌晗, 荣学文, 范永. 导盲机器人研究现状综述[J]. 计算机工程与应用, 2020, 56(14): 1-13.

[6] 李新春. 浅谈SLAM+AR眼镜在远程协助中的应用[J]. 中国设备工程, 2020(19): 251-252.

[7] 周路鹏, 王纪凯, 陈宗海. 融合环境语义信息的稠密三维场景重建研究综述[C]//第21届中国系统仿真技术及其应用学术年会论文集(CCSSTA21st 2020), Vol.21.中国云南昆明, 2020: 5.

[8] 施俊屹, 查富生, 孙立宁. 移动机器人视觉惯性SLAM研究进展[J]. 机器人, 2020, 42(06): 734-748.

[9] YUE Wang, HUANG Shou-dong, RONG Xiong, et al. A framework for multi-session RGBD SLAM in low dynamic workspace environment[J]. CAAI Transactions on Intelligence Technology, 2016, 1(1): 90-103.

[10] BŁAŻEJ C, ADAM S. Real-time RGBD SLAM system[J]. Warsaw Univ. of Technology (Poland), 2015, 9662(9): 96622B-966966.

[11] LIGOCKI A, JELÍNEK A. Fusing the RGBD SLAM with Wheel Odometry[J]. IFAC Papersonline, 2019, 52(27): 7-12.

[12] 段珊珊, 李昕. 基于RGBD传感器的场景自适应性视觉里程计算法[J]. 计算机与现代化, 2016(12): 73-77.

[13] 杨观赐, 王霄远, 蒋亚汶, 等. 视觉与惯性传感器融合的SLAM技术综述[J]. 贵州大学学报(自然科学版), 2020, 37(06): 1-12.

[14] 李卫成, 汪地, 宗殿栋, 等. 基于RGBD的机器人室内SLAM与路径规划系统[J]. 工业控制计算机, 2016, 29(03): 77-78+81.

[15] STEFAN H, STRAKA M, GERHARD R. Temporal coherence in image-based visual hull rendering[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(10): 1758-1767.

[16] 郑太雄, 黄帅, 李永福,等. 基于视觉的三维重建关键技术研究综述[J]. 自动化学报, 2020, 46(04): 631-652.

[17] 李兵. 基于彩色莫尔条纹的三维轮廓动态测量技术基础研究[J]. 机械工程学报, 2011, 47(02): 196.

[18] 侯飞, 郑福, 李国栋, 等.一种基于飞行时间相机的点云目标提取方法[J]. 激光与红外, 2019, 49(11): 1381-1387.

[19] 卢纯青, 宋玉志, 武延鹏, 等. 基于TOF计算成像的三维信息获取与误差分析[J]. 红外与激光工程, 2018, 47(10): 160-166.

[20] 周俞辰.基于激光三角测距法的激光雷达原理综述[J].电子技术与软件工程, 2016(19): 94-95.

[21] 徐超, 李乔. 基于计算机视觉的三维重建技术综述[J]. 数字技术与应用, 2017(01): 54-56.

[22] WEN Shu-huan, LIU Xin, ZHANG Hong, et al. Dense point cloud map construction based on stereo VINS for Mobile vehicles[J]. Isprs Journal of Photogrammetry and Remote Sensing, 2021, 178(9): 328-344.

[23] LI Qing-qing, JORGE P Q, GIA T N, et al. Multi-Sensor fusion for navigation and mapping in autonomous vehicles: accurate localization in urban environments[J]. Unmanned Systems, 2020, 08(03): 9.

[24] 高兴波, 史旭华, 葛群峰, 等. 面向动态物体场景的视觉SLAM综述[J]. 机器人, 2020(7): 1-18.

[25] 赵锐, 万旺根. 基于多尺度空间特征融合的三维重建[J]. 工业控制计算机, 2021, 34(11): 86-87+90.

[26] 邓仕超, 陈艺海, 黄扬, 等. 基于结构光三维重建系统的改进相位研究[J]. 组合机床与自动化加工技术, 2021(11): 31-34+38.

[27] 赵冬青, 畅雅雯, 单彦虎, 等.双目测量系统的室内定位与重建[J]. 激光杂志: 1-5.

[28] 李婧, 韩贤权. 激光三维虚拟技术的室内环境设计图像重建研究[J]. 激光杂志, 2021, 42(09): 124-128.

[29] 崔海华, 姜涛, 杜坤鹏, 等. 基于深度学习位姿估计的多视结构光三维成像方法[J]. 光学学报, 2021, 41(17): 72-81.

[30] 吉白冰, 曹其新. 一种用于机器人抓取的单目实时三维重建系统[J]. 机械设计与制造, 2021(09): 287-290.

[31] 杨俊哲, 姜龙飞, 李梅, 等. 基于激光点云的掘进工作面三维场景重建技术研究[J].煤炭科学技术, 2021, 49(S1): 40-45.

[32] 余兆凯, 彭晓峰, 邱昌杰. 基于Kinect三维重建的地下井室可视化方法研究[J]. 导航定位与授时, 2021, 8(04): 112-119.

[33] 吴永林. 基于大数据和RGBD图像的虚拟实验室场景三维重建[J]. 宁夏师范学院学报, 2021, 42(07): 96-103.

[34] 宋婉婷, 江文松, 罗哉. 基于多标签分类的点云快速批量三维重建[J]. 激光与光电子学进展, 2021, 58(12): 75-85.

[35] 李磊磊, 黄海霞, 郭阳, 等. 基于红外辐射偏振成像的目标三维重建方法[J]. 红外与毫米波学报, 2021, 40(03): 413-419.

[36] 杨永兆, 张玉金, 张立军. 由形状结构和位姿特征学习的稠密点云重建[J]. 计算机科学与探索失:1-12[2022-03-17].http://kns.cnki.net/kcms/detail/11.5602.TP.20210401.1024.002.html.

[37] 陈寂驰, 魏国华, 郭聪隆, 等. 一种基于红外图像序列的深度学习三维重建仿真方法初探[J]. 空天防御, 2020, 3(04): 21-29.

[38] 信寄遥, 陈成军, 李东年. 基于RGBD相机的多视角机械零件三维重建[J]. 计算技术与自动化, 2020, 39(03): 147-152.

[39] 王开鑫, 王世峰, 孙琪, 等. 点云分割匹配的三维重建算法[J]. 长春理工大学学报(自然科学版), 2020, 43(04): 49-56.

[40] 葛媛媛, 张宏基. 基于立体视觉的煤矿巷道三维重建方法研究[J]. 计算机与数字工程, 2013, 41(07): 1169-1171.

[41] MAO Ming-yuan, ZHANG He-wei, LI Si-meng, et al. SEMANTIC-RTAB-MAP (SRM): A semantic SLAM system with CNNs on depth images[J]. Mathematical Foundations of Computing, 2019, 2(1): 29-41.

[42] CHEN M. Bionic SLAM based on MEMS pose measurement module and RTAB-Map closed loop detection algorithm[J]. Cluster Computing, 2019, 22(3): 5367-5378.

[43] ÁDÁM W, PÉTER T, STEFAN R, et al. A benchmark of popular indoor 3D Reconstruction technologies: comparison of arcore and RTAB-Map[J]. Electronics, 2020, 9(12): 1.

[44] 马宏伟, 王世斌, 毛清华, 等. 煤矿巷道智能掘进关键共性技术[J]. 煤炭学报, 2021, 46(01): 310-320.

[45] 石泉, 孙常军, 郑洪涛, 等. 掘进机机器人化的关键技术研究[J]. 煤炭科学技术, 2020, 48(S2): 199-204.

[46] 张旭辉, 刘博兴, 张超, 等. 掘进机全站仪与捷联惯导组合定位方法[J]. 工矿自动化, 2020, 46(09): 1-7.

[47] 马源, 符世琛, 张子悦, 等. 悬臂式掘进机位姿检测方法研究现状[J]. 工矿自动化, 2020, 46(08): 15-20.

[48] CHEN Qi-wei, TANG Lai-ying, YANG Zai-hua, et al. Study and application of lunar rover tracking measurement technology with laser radar and iGPS[J]. IOP Conference Series: Materials Science and Engineering, 2019, 592(1): 012147.

[49] 张佩华, 吕红丽. iGPS导航技术的探索分析研究[J]. 现代导航, 2018, 9(02): 88-93.

[50] 雷孟宇, 张旭辉, 杨文娟, 等. 煤矿掘进设备视觉位姿检测与控制研究现状与趋势[J].煤炭学报: 1-14[2022-03-17].DOI:10.13225/j.cnki.jccs.JJ21.0534.

[51] 张凯, 田原, 贾曲. 机器视觉在煤机设备中的应用现状与趋势[J]. 煤矿机械, 2020, 41(12): 123-125.

[52] 王洋洋. UWB技术在煤矿精确定位中的应用[J]. 煤炭技术, 2020, 39(05): 186-188.

[53] KRASNOV T V, GARIFULLIN V F, FEOKTISTOV D S, et al. Research of data transmission system based on UWB technology[J]. Journal of Physics: Conference Series, 2020, 1515(2): 022063.

[54] 李丽, 张晓亮. 基于机器视觉的采摘机器人果实三维模型重构与识别[J]. 农机化研究, 2021, 43(10): 231-235.

[55] 郗厚印, 张栋, 周涛, 等. 采摘机器人识别抓取重叠番茄果实的方法研究[J]. 农机化研究, 2021, 43(12): 17-23+50.

[56] ROMAN M, JOSIF B, NIKOLAY D. Development of position system of a roadheader on a base of active IR-sensor[J]. Procedia Engineering, 2015, 100: 617-621.

[57] CHRISTOPH G, HARKER M, O'LEARY P, et al. An algebraic framework for the Real-Time solution of inverse problems on embedded systems[C]. CORR, 2014, abs/1406.0380: 1097-1102.

[58] CHRISTOPH G, O'LEARY P, HARKER M. Analytic multisource data fusion and its application to a Large-Scale optical PSD[J]. IEEE Trans. Instrumentation and Measurement, 2014, 63(5): 1116-1126.

[59] 田原. 悬臂式掘进机惯性定位技术研究与试验[J]. 煤矿机电, 2020, 41(01): 9-12.

[60] 杜雨馨. 矿井悬臂式掘进机位姿感知及定位方法研究[D]. 北京,中国矿业大学,2019.

[61] 张旭辉, 刘永伟, 毛清华, 等. 煤矿悬臂式掘进机智能控制技术研究及进展[J]. 重型机械, 2018(02): 22-27.

[62] 张旭辉, 赵建勋, 张超. 悬臂式掘进机视觉伺服截割控制系统研究[J]. 煤炭科学技术: 1-8.[2022-03-17].http://kns.cnki.net/kcms/detail/11.2402.TD.20200227.1657.034.html

[63] 张旭辉, 张楷鑫, 张超, 等. 悬臂式掘进机视觉位姿检测系统外参标定方法[J]. 机械科学与技术: 1-8[2022-03-17].DOI:10.13433/j.cnki.1003-8728.20200534.

[64] 杨文娟, 张旭辉, 张超, 等. 悬臂式掘进机器人巷道成形智能截割控制系统研究[J].工矿自动化, 2019, 45(09): 40-46.

[65] 杨文娟, 张旭辉, 马宏伟, 等. 悬臂式掘进机机身及截割头位姿视觉测量系统研究[J].煤炭科学技术, 2019, 47(06): 50-57.

[66] 包建军. 煤矿井下设备接近探测方法研究[J]. 工矿自动化, 2017, 43(01): 1-4.

[67] 邢勇锋. 煤矿井下综掘工作面设备协同控制方案设计[J]. 自动化应用, 2020(09): 142-143.

[68] 杜群, 甄成刚, 郝悍勇. 基于量子蚁群的快速碰撞检测算法研究[J]. 计算机仿真, 2019, 36(12): 209-213+26.

[69] 卢江, 钱德英, 周伟中, 等. 基于观察坐标与混合包围盒的装配碰撞检测方法[J]. 船舶工程, 2019, 41(09): 12-16,51.

[70] NIU Hao-tian, MA Cun-bao, HAN Pei. A decentralized method for collision detection and avoidance applied to civil aircraft[J]. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 2021, 235(6): 621-645.

[71] CHANDRAMOHAN K, KAMALAKKANNAN P. An efficient neighbor discovery and aloha based collision detection and correction in vanet[J]. Indian Journal of Science and Technology, 2015, 8(35): 1-9.

[72] ZHANG Fei-wang, CHUN Yang-liu, XIN Sui, et al. Three-dimensional point cloud object segmentation and collision detection based on depth projection[J]. Optics and Precision Engineering, 2020, 28(7): 1600-1608.

[73] MELERO F J, AGUILERA Á, FEITO F R. Fast collision detection between high resolution polygonal models[J]. Computers &Amp; Graphics, 2019, 83: 97-106.

[74] 方梦娟, 徐巧玉, 李坤鹏, 等. 多臂凿岩机器人的碰撞检测方法研究[J]. 河南科技大学学报(自然科学版), 2022, 43(01): 19-25+5.

[75] 王妙云, 张旭辉, 马宏伟, 等. 远程控制综采设备碰撞检测与预警方法[J]. 煤炭科学技术, 2021, 49(09): 110-116.

[76] 魏畅, 程岳, 王建生. OSG下碰撞检测算法的设计与实现[J]. 信息技术与信息化, 2021(03): 100-102.

[77] 靳雁霞, 程琦甫, 张晋瑞, 等. 融合DNN与AABB—圆形包围盒自碰撞检测[J]. 中国图象图形学报, 2020, 25(08): 1674-1683.

[78] 邓新敏. 输电铁塔试装仿真碰撞检测技术研究与应用[J]. 新技术新工艺, 2021(07): 43-47.

[79] 袁野, 白瑞林, 邢晓凡. 未知模型机器人的碰撞检测算法[J]. 制造业自动化, 2021, 43(07): 34-38+100.

[80] 刘纯键, 高红星, 蒋林, 等. 基于一种碰撞检测算法的液压机械臂柔顺性分析[J]. 武汉科技大学学报, 2021, 44(05): 366-373.

[81] 廖伟东, 李俊渊, 黄昕. 机器人仿真系统碰撞检测技术研究[J]. 机床与液压, 2021, 49(09): 67-70.

[82] 徐涵, 李建兵, 许涛, 等. 基于EEMD-SVM的行驶碰撞检测方法[J]. 信息工程大学学报, 2021, 22(02): 142-146.

[83] 肖世龙, 张德育, 刘源, 等. 基于量子群智能优化检测虚拟现实中物体碰撞[J]. 济南大学学报(自然科学版), 2021, 35(05): 423-427+43.

[84] 王飞. 矿用精确定位系统防碰撞方法研究与实现[J]. 煤炭技术, 2016, 35(07): 108-109.

[85] 王道累,孙昊,胡松, 等. 基于天牛须搜索算法的单目相机标定方法[J]. 济南大学学报(自然科学版),2020,34(06):568-574.