瓦斯钻孔深度测量是防治井下瓦斯灾害的重要措施，而通过统计钻机打入的钻杆数量可以间接计算钻孔深度。目前采用人工数钻杆的方法需要耗费大量人力资源，并且人在长时间劳作下容易产生疲惫，导致效率低下。钻机作业时需要工人进行装卸钻杆，因而可以借助视觉技术来识别工人的装卸钻杆动作，达到自动数钻杆的目的。人体骨架序列是一种简单且能够清楚反映肢体动作的数据，因此论文提出一种基于人体骨架序列的井下钻杆计数方法，主要的工作包括：

（1）提出一种用于井下视频流的多人目标跟踪模型。该模型以目标跟踪模型DeepSORT为基础框架，通过YOLOv5s模型来提高跟踪所需的检测目标质量，通过改进DeepSORT-CNN模型来提高跟踪所需的代价矩阵质量，从而提升井下多人目标的跟踪效果。实验结果表明，所提跟踪模型在复杂环境中的应用结果优于原DeepSORT模型，其中YOLOv5s的mAP精度为 92.1%、改进DeepSORT-CNN的AUC精度为 96.0%。

（2）为获取装卸钻杆工人的骨架序列，提出一种用于分离多人骨架序列的方法。首先，利用上述多人目标跟踪模型在视频流中对工人进行连续检测及跟踪，得到工人的目标图像和ID信息；其次，使用AlphaPose模型对目标图像上的人体关键点进行检测，得到单帧骨架数据；最后，根据工人的ID信息对连续检测到的骨架数据进行关联，进而分离出装卸钻杆工人的骨架序列。

（3）提出一种用于识别工人装卸钻杆动作的改进ST-GCN⁺⁺模型。首先，使用人体骨架序列作为动作表征数据，减少图片背景和人体表观颜色对动作的干扰；其次，通过人体骨架图分区策略与空间特征融合机制构建出改进模型；最后，建立装卸钻杆动作数据集来完成动作识别。实验结果表明，改进ST-GCN⁺⁺的准确率较ST-GCN提高 8.9%，同时它对装、卸钻杆动作的识别精度明显优于C3D、ResNet101-LSTM等模型。

（4）根据实际需求，研发一套用于井下钻机的钻杆计数系统。首先，利用已有打钻监控系统及RTSP协议完成井下视频采集；其次，搭建智能视频分析服务器，并设计人机交互界面；最后，通过智能算法识别工人的每一次卸钻杆动作，完成钻杆数量统计。

Gas drilling depth measurement is an important measure to prevent and control underground gas disasters, and the drilling depth can be indirectly calculated by counting the number of drill pipes drilled by the drilling rig. At present, the method of manually counting drill pipes requires a lot of human resources, and people are prone to fatigue under long-term labor, resulting in low efficiency. The drilling rig needs workers to load and unload the drill pipe, so the visual technology can be used to identify the workers' action of loading and unloading the drill pipe, so as to achieve the purpose of automatically counting the drill pipe. The human skeleton sequence is a simple data that can clearly reflect the body movements. Therefore, the thesis proposes a downhole drill pipe counting method based on human skeleton sequence. The main work includes:

(1) A multi-person target tracking model for underground video stream is proposed. The model takes the target tracking model DeepSORT as the basic framework, improves the quality of the detection target required for tracking through the YOLOv5s model, and improves the quality of the cost matrix required for tracking by improving the DeepSORT-CNN model, thereby improving the tracking effect of multiple underground targets. The experimental results show that the proposed tracking model is superior to the original DeepSORT model in complex environments. The mAP accuracy of YOLOv5s is 92.1%, and the AUC accuracy of the improved DeepSORT-CNN is 96.0%.

(2) In order to obtain the skeleton sequence of loading and unloading drill pipe workers, a method for separating multi-person skeleton sequence is proposed. Firstly, the above multi-person target tracking model is used to continuously detect and track workers in the video stream, and the target image and ID information of workers are obtained. Secondly, the AlphaPose model is used to detect the key points of the human body on the target image, and the single frame skeleton data is obtained. Finally, according to the worker 's ID information, the continuously detected skeleton data is associated, and then the skeleton sequence of the loading and unloading drill pipe worker is separated.

(3) An improved ST-GCN⁺⁺ model for identifying workers ' loading and unloading drill pipe movements is proposed. Firstly, the human skeleton sequence is used as the action representation data to reduce the interference of image background and human apparent color on the action. Secondly, an improved model is constructed through the human skeleton map partitioning strategy and spatial feature fusion mechanism. Finally, the loading and unloading drill pipe action data set is established to complete the action recognition. The experimental results show that the accuracy of improved ST-GCN⁺⁺ is 8.9% higher than that of ST-GCN, and its recognition accuracy of loading and unloading drill pipe action is obviously better than that of C3D, ResNet101-LSTM and other models.

(4) According to the actual demand, a set of drill pipe counting system for downhole drilling rig is developed. Firstly, the downhole video acquisition is completed by using the existing drilling monitoring system and RTSP protocol. Secondly, build an intelligent video analysis server and design a human-computer interaction interface; Finally, the intelligent algorithm is used to identify each drill pipe unloading action of the worker, and the number of drill pipes is counted.

[1]中华人民共和国2021年国民经济和社会发展统计公报[J]. 中国统计, 2022(03): 9-26.

[2]王双明. 对我国煤炭主体能源地位与绿色开采的思考[J]. 中国煤炭, 2020, 46(02): 11-16.

[3]张建民, 李全生, 张勇, 等. 煤炭深部开采界定及采动响应分析[J]. 煤炭学报, 2019, 44(05): 1314-1325.

[4]Chen X, Li L, Wang L, et al. The current situation and prevention and control countermeasures for typical dynamic disasters in kilometer-deep mines in China[J]. Safety Science, 2019, 115: 229-236.

[5]Pan X, Cheng H, Chen J, et al. An experimental study of the mechanism of coal and gas outbursts in the tectonic regions[J]. Engineering Geology, 2020, 279:105883.

[6]王恩元, 张国锐, 张超林, 等. 我国煤与瓦斯突出防治理论技术研究进展与展望[J]. 煤炭学报, 2022, 47(01): 297-322.

[7]王耀锋. 中国煤矿瓦斯抽采技术装备现状与展望[J]. 煤矿安全, 2020, 51(10): 67-77.

[8]李东前. 煤矿瓦斯防治技术研究[J]. 当代化工研究, 2021(10): 99-100.

[9]李树刚, 包若羽, 张天军, 等. 本煤层瓦斯抽采钻孔合理密封深度确定[J]. 西安科技大学学报, 2019, 39(02): 183-188+216.

[10]郑磊. 煤矿井下钻场打钻监控视频系统应用智能分析技术研究[J]. 能源与环保,2019,41(09):107-110.

[11]赵朗月, 吴一全. 基于机器视觉的表面缺陷检测方法研究进展[J]. 仪器仪表学报, 2022, 43(01): 198-219.

[12]孙志飞, 吴银成, 胡云. 钻杆长度测量方法[J]. 工矿自动化, 2015, 41(03): 51-53.

[13]彭业勋. 煤矿井下钻杆计数方法研究[D]. 西安科技大学, 2019.

[14]杜京义, 党梦珂, 乔磊, 等. 基于改进时空图卷积神经网络的钻杆计数方法[J]. 工矿自动化, 2023, 49(01): 90-98.

[15]董立红, 王杰, 厍向阳. 基于改进Camshift算法的钻杆计数方法[J]. 工矿自动化, 2015, 41(01): 71-76.

[16]吴鑫艺. 基于改进RFB和孪生网络的煤矿钻杆计数方法研究[D]. 西安科技大学, 2021.

[17]张国华. 基于视觉跟踪算法的井下钻杆计数研究及应用[D]. 中国矿业大学, 2021.

[18]张栋, 姜媛媛. 融合注意力机制与逆残差结构的轻量级钻机目标检测方法[J]. 电子测量与仪器学报, 2022, 36(11): 201-210.

[19]高瑞, 郝乐, 刘宝, 等. 基于改进ResNet网络的井下钻杆计数方法[J]. 工矿自动化, 2020, 46(10): 32-37.

[20]张栋, 姜媛媛. 基于改进MobileNetV2的钻杆计数方法[J]. 工矿自动化, 2022, 48(10): 69-75.

[21]党伟超, 姚远, 白尚旺, 等.煤矿探水卸杆动作识别研究[J]. 工矿自动化, 2020, 46(7): 107-112.

[22]付苗苗, 邓淼磊, 张德贤. 深度神经网络图像目标检测算法综述[J]. 计算机系统应用, 2022, 31(07): 35-45.

[23]Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6):1137-1149.

[24]杜京义, 史志芒, 等. 轻量化煤矸目标检测方法研究[J]. 工矿自动化, 2021, 47(11): 119-125.

[25]Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.

[26]Redmon J, Farhadi A. YOLO9000: better, faster, stronger[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 7263-7271.

[27]Redmon J, Farhadi A. YOLOv3: An incremental improvement[EB/OL]. https://arxiv.org/abs/1804.02767.pdf. 2023-04-29.

[28]谭宇璇, 樊绍胜. 基于图像增强与深度学习的变电设备红外热像识别方法[J]. 中国电机工程学报, 2021, 41(23): 7990-7998.

[29]Bochkovskiy A, Wang C Y, Liao H. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. https://arxiv.org/pdf/2004.10934.pdf. 2023-04-29.

[30]沈科, 季亮, 张袁浩, 等. 基于改进YOLOv5s模型的煤矸目标检测[J]. 工矿自动化, 2021, 47(11): 107-111+118.

[31]Ge Z, Liu S， Wang F, et al. YOLOX: Exceeding YOLO Series in 2021[EB/OL]. https://arxiv.org/pdf/2107.08430.pdf. 2023-05-26.

[32]Wang C , Bochkovskiy A, Liao H. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. https://arxiv.org/pdf/2207.02696.pdf. 2023-05-26.

[33]金沙沙, 龙伟, 胡灵犀, 等.多目标检测与跟踪算法在智能交通监控系统中的研究进展[J]. 控制与决策, 2023, 38(04): 890-901.

[34]李玺, 查宇飞, 等. 深度学习的目标跟踪算法综述[J]. 中国图象图形学报, 2019, 24(12): 2057-2080.

[35]Hornakova A, Henschel R, Rosenhahn B, et al. Lifted disjoint paths with application in multiple object tracking[C]//Proceedings of the International conference on machine learning. 2020: 4364-4375.

[36]Bewley A, Ge Z, Ott L, et al. Simple online and realtime tracking[C]//Proceedings of the IEEE international conference on image processing. 2016: 3464-3468.

[37]Wojke N, Bewley A, Paulus D. Simple online and realtime tracking with a deep association metric[C]//Proceedings of the IEEE international conference on image processing. 2017: 3645-3649.

[38]张旭辉, 闫建星, 张超, 等. 基于改进YOLOv5s+DeepSORT的煤块行为异常识别[J]. 工矿自动化, 2022, 48(06): 77-86+117.

[39]田皓宇, 马昕, 李贻斌. 基于骨架信息的异常步态识别方法[J]. 吉林大学学报(工学版), 2022, 52(04): 725-737.

[40]张宇, 温光照, 米思娅, 等. 基于深度学习的二维人体姿态估计综述[J].软件学报, 2022, 33(11): 4173-4191.

[41]Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation[C]//Proceedings of the Computer Vision–ECCV. 2016: 483-499.

[42]马双双, 王佳, 曹少中, 等. 基于深度学习的二维人体姿态估计算法综述[J]. 计算机系统应用, 2022, 31(10): 36-43.

[43]Cao Z, Simon T, Wei S, et al. Realtime multi-person 2d pose estimation using part affinity fields[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 7291-7299.

[44]Fang H, Xie S, Tai Y, et al. RMPE: Regional multi-person pose estimation[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2334-2343.

[45]Papandreou G, Zhu T, Chen L, et al. Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model[C]//Proceedings of the European conference on computer vision. 2018: 269-286.

[46]Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5693-5703.

[47]张冰冰, 李培华, 孙秋乐. 基于局部约束仿射子空间编码的时空特征聚合卷积网络模型[J]. 计算机学报, 2020, 43(09): 1589-1603.

[48]Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the IEEE international conference on computer vision. 2015: 4489-4497.

[49]Feichtenhofer C, Pinz A, Zisserman A. Convolutional two-stream network fusion for video action recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 1933-1941.

[50]Li C, Zhong Q, Xie D, et al. Skeleton-based action recognition with convolutional neural networks[C]//Proceedings of the IEEE International Conference on Multimedia & Expo Workshops. 2017: 597-600.

[51]Yan S, Xiong Y, Lin D. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the AAAI conference on artificial intelligence. 2018, 32(1).

[52]Li X, Meng F, Zhao F, et al. Two-Stream adaptive-attentional subgraph convolution networks for skeleton-based action recognition[J]. Multimedia Tools and Applications, 2022, 81(4): 4821-4838.

[53]陈佳倩, 金晅宏, 王文远, 等. 基于YOLOv3和DeepSort的车流量检测[J]. 计量学报, 2021, 42(06): 718-723.

[54]Sun B, Luo A, Rong B, et al. One-Shot Face Recognition through a Region Inception ResNet with Modified Triplet Loss[C]//Proceedings of the Journal of Physics Conference Series. 2022, 2320(1): 012022.

[55]卢健, 王航英, 陈旭, 等. 基于多尺度特征表示的行人再识别[J]. 控制与决策, 2021, 36(12): 3015-3022.

[56]Kumar R. Image Classification Using Network Inception-Architecture Appications[J]. International Journal of Innovative Research in Computer and Communication Engineering, 2021, 10(1): 339-342.

[57]Howard A, Zhu M, Chen B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. https://arxiv.org/pdf/1704. 04861.pdf. 2023-04-29.

[58]Huang S, Gong M, Tao D. A coarse-fine network for keypoint localization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 3028-3037.

[59]Xiao B, Wu H, Wei Y. Simple baselines for human pose estimation and tracking[C]//Proceedings of the European conference on computer vision. 2018: 466-481.

[60]杨世强, 李卓, 王金华, 等. 基于新分区策略的ST-GCN人体动作识别[J/OL]. 计算机集成制造系统: 1-16[2023-04-29]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211022.1500.014.html

[61]He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.

[62]Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018:7132-7141.

[63]张传雷, 武大硕, 向启怀, 等. 基于ResNet-LSTM的具有注意力机制的办公人员行为视频识别[J]. 天津科技大学学报, 2020, 35(06): 72-80.