论文中文题名: |
改进 YOLOv5+DeepSort 的矿工个人防护装备检测与跟踪研究
|
姓名: |
张科
|
学号: |
20206043039
|
保密级别: |
公开
|
论文语种: |
chi
|
学科代码: |
081104
|
学科名称: |
工学 - 控制科学与工程 - 模式识别与智能系统
|
学生类型: |
硕士
|
学位级别: |
工学硕士
|
学位年度: |
2023
|
培养单位: |
西安科技大学
|
院系: |
电气与控制工程学院
|
专业: |
控制科学与工程
|
研究方向: |
图像处理与深度学习
|
第一导师姓名: |
王征
|
第一导师单位: |
西安科技大学
|
论文提交日期: |
2023-06-14
|
论文答辩日期: |
2023-06-02
|
论文外文题名: |
Research on Detection and Tracking of Miners' Personal Protective Equipment based on improved YOLOv5+DeepSort
|
论文中文关键词: |
个人防护装备 ; 图像增强 ; 目标检测 ; YOLOv5 算法 ; DeepSort 算法
|
论文外文关键词: |
Personal protective equipment ; Image enhancement ; Object detection ; YO LOv5 algorithm ; DeepSort algorithm
|
论文中文摘要: |
︿
矿井安全一直以来是国家和社会重点关注的安全问题。为避免矿工在作业过程中遭受伤害,矿工在下井作业时需要监测个人防护装备是否佩戴完整。当前对矿工个人防护装备的监测方法主要以人工监管与目标检测为主,存在人工成本高、检测效率低等问题。本文基于 YOLOv5 与 DeepSort 算法构建矿工个人防护装备规范佩戴检测与多目标跟踪方法,实现矿工个人防护装备佩戴的有效监测。本文的主要研究工作如下:
(1)针对深度学习需要大量样本作为训练数据集,而矿井下存在环境复杂、光照强度低及粉尘影响严重导致监控摄像头捕捉到的图像质量不高的问题,本文基于深度学习的矿井图像增强方法构建改进 KinD 矿井图像增强方法,对比传统矿井图像增强方法与其他基于深度学习的矿井图像增强方法。实验结果表明本文改进 KinD 矿井图像增强算法能够有效增强图像中矿工的边缘细节信息,提高图像整体对比度,算法的峰值信噪比(PSNR)、结构相似度(SSIM)及算法运行速度较 KinD 矿井增强算法分别提升 6.92%,4.05%及 0.2s。
(2)针对当前目标检测算法对矿工个人防护装备检测精度低、速度较慢,以及对小目标误检、漏检等问题,本文在 YOLOv5 目标检测模型的基础上提出轻量化的YOLOv5 目标检测模型(XGF_YOLOv5)。首先构建轻量化主干特征提取网络 Xception,其次在主干网络嵌入改进的 Fused_MBConv 模块、Ghost 模块及注意力机制模块 CBAM,最后在多尺度特征层之后融入自适应空间特征融合(ASFF)提升多尺度特征融合能力。通过客观对比、主观对比及消融实验验证本文改进模型的有效性与合理性。实验结果表明本文构建的 XGF_YOLOv5 目标检测模型相较于 YOLOv5 模型更加轻量化,参数量减少 12%。改进模型对矿工个人防护装备检测的准确率相较于 YOLOv5 模型提升约2%。改进方法的平均均值精度(mAP)、F1 分数(F1-score)、精确度(Precision)及召回率(Recall)相较于 YOLOv5 模型分别提升 3.10%、1.83%、3.01%和 2.10%,具有较优的检测性能。
(3)将本文构建的 XGF_YOLOv5 检测模型与基于重识别模型 OSNet 的 DeepSort算法结合构建视频多目标跟踪算法。应用卡尔曼滤波更新跟踪轨迹,采用 CIoU 匹配实现矿工个人防护装备规范佩戴的检测与跟踪。实验结果表明,本文改进的矿工个人防护装备跟踪算法的准确率及精度较 YOLOv5+DeepSort 算法分别提升 3.9%和 1.8%,ID错误切换次数降低 9 次。通过对矿工个人防护装备类别计数及跟踪结果进行判定,可实现快速监测矿工个人防护装备是否规范佩戴的目的。
﹀
|
论文外文摘要: |
︿
Mine safety has always been a key concern of the state and society. In order to preventminers from being injured in the process of operation, miners need to check whether their personal protective equipment is sufficiently worn when working in the well. Current monitoring methods for miners' personal protective equipment mainly focus on manual supervision and object detection, which suffer from high labor costs and low detection efficiency. In this paper, based on YOLOv5 and DeepSort algorithms, the detection and mutli-object tracking methods of miners' personal protective equipment (PPE) standard wearing is constructed to realize effective monitoring of miners' PPE wearing. The main research work of this paper is the following:
(1) As deep learning requires a large number of samples as training data set, and there are problems such as complex environment, low light intensity and serious dust influence under the mine, leading to poor image quality captured by surveillance cameras, this paper builds an improved KinD mine image enhancement method based on deep learning mine image enhancement method. Contrast traditional mine image enhancement methods with additional deep learning based mine image enhancement methods. Experimental results show that the modified KinD mine image enhancement algorithm can effectively enhance the edge details of miners in the image and improve the overall image contrast. Compared with the KinD mine image enhancement algorithm, the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and running speed of the algorithm are improved by 6.92%, 4.05% and 0.2s, respectively.
(2) In view of the low accuracy and moderate speed of the current object detection algorithm for miners' personal protective equipment detection, as well as the problems of detection and missing detection of tiny targets, a lightweight YOLOv5 object detection model (XGF_YOLOv5) is proposed in this paper based on the YOLOv5 object detection model.
Firstly,constructing Xception, a lightweight backbone feature extraction network. Secondly, the modified Fused_MBConv module, Ghost module and attention mechanism module CBAM are embedded in the backbone network. Finally, the adaptive spatial feature fusion (ASFF) is integrated after the multi-scale feature layer to improve the multi-scale feature fusion capability.
The effectiveness and plausibility of the modified model are verified by objective comparisons,subjective comparisons and ablation experiments. Experimental results show that the
XGF_YOLOv5 object detection model constructed in this paper is lighter and has 12 percent fewer parameters compared to the YOLOv5 model. Compared to the YOLOv5 model, the improved model improves the accuracy of the detection of miners' personal protective equipment by about 2 percent. Compared to the YOLOv5 model, the mAP, F1 score, Precision and Recall of the modified method are improved by 3.10%, 1.83%, 3.01%% and 2.10%,respectively, showing better detection performance.
(3) The proposed video mutli-object tracking algorithm is constructed by combining the modified YOLOv5 object detection model constructed in this paper with the DeepSort algorithm based on OSNet, a re-identification model. The Kalman filter is applied to update the tracking trajectory and CIoU matching is applied to achieve the detection and tracking of the miner's PPE wearing standards. Experimental results show that compared to
YOLOv5+DeepSort, the proposed algorithm improves the accuracy and precision by 3.9% and 1.8%, reducing the number of ID error switches by a factor of 9. The purpose of quickly monitoring whether a miner's personal protective equipment is worn accurately can be achieved by judging the results of counting and tracking the categories of the miner's personal protective equipment.
﹀
|
参考文献: |
︿
[1] 秦容军. 我国煤炭开采现状及政策研究[J]. 煤炭经济研究, 2019, 39(01): 57-61. [2] GB/T 29510-2013, 个体防护装备配备基本要求[S]. [3] 孙鑫. 煤矿井下工作人员安全防护装备改进研究[D]. 湖北: 湖北工业大学, 2011. [4] 曾明荣. 全力以赴防范化解重大安全风险——《“十四五”国家安全生产规划》解 读[J]. 中国安全生产, 2022, 17(05): 7-9. [5] 邵小强, 杨涛, 卫晋阳, 等. 改进同态滤波的矿井监控视频图像增强算法[J]. 西安科技大学学报, 2022,42(6): 1205-1213. [6] 龚云, 颉昕宇. 基于同态滤波方法的煤矿井下图像增强技术研究[J]. 煤炭科学技术,2023, 51(03): 241-250. [7] Tian F, Chen T, Zhang J. Research on Improved Retinex-Based Image Enhancement Method for Mine Monitoring[J]. Applied Sciences, 2023, 13(4): 2672. [8] Hua G, Jiang D. A new method of image denoising for underground coal mine based on the visual characteristics[J]. Journal of applied mathematics, 2014, 2014. [9] 李晓宇, 吕进来, 郝晓丽. 一种改进的 Retinex 矿井图像增强算法[J]. 科学技术与工程, 2020, 20(29): 12028-12034. [10] 李藤, 刘凯雷. 基于深度学习的矿井下低照度图像增强算法研究[J]. 电子测试, 2022, 36(09): 51-53+134. [11] 王满利, 张航, 李佳悦, 张长森. 基于深度神经网络的矿井下低光照图像增强算法[J/OL]. 煤炭科学技术, 2023: 1-13. [12] Song H. Multi-Scale Safety Helmet Detection Based on RSSE-YOLOv3[J]. Sensors, 2022,22(16): 6061. [13] Cheng R, He X, Zheng Z, et al. Multi-scale safety helmet detection based on SAS-YOLOv3-tiny[J]. Applied Sciences, 2021, 11(8): 3652. [14] 邱浩然. 基于改进 YOLOv3 的安全帽检测算法研究与实现[D]. 成都: 西南交通大学,2020. [15] Chen J, Deng S, Wang P, et al. Lightweight Helmet Detection Algorithm Using an Improved YOLOv4[J]. Sensors, 2023, 23(3): 1256. [16] 徐守坤, 王雅如, 顾玉宛, 李宁, 庄丽华, 石林. 基于改进 Faster-RCNN 的安全帽佩戴检测研究[J]. 计算机应用研究, 2020, 37(03): 901-905. [17] Zhang Y, Xiao F, Lu Z. Helmet Wearing State Detection Based on Improved YOLOv5s[J].Sensors, 2022, 22(24): 9843. [18] 杨永波, 李栋. 改进 YOLOv5 的轻量级安全帽佩戴检测算法[J/OL]. 计算机工程与应用, 2022: 1-8. [19] 赵敏,杨国亮,王吉祥,龚志鹏.改进 YOLOv7-tiny 的安全帽实时检测算法[J/OL].无线电工程:1-11[2023-05-18]. [20] Nath N D, Behzadan A H, Paal S G. Deep learning for site safety: Real-time detection of personal protective equipment[J]. Automation in Construction, 2020, 112: 103085: 1-20. [21] 赵恩铭, 杨松, 罗创, 等. 基于 YOLOv5s 的施工人员安全防护设备佩戴检测研究[J].大理大学学报, 2022, 7(12): 37-42. [22] Ma L, Li X, Dai X, et al. A Combined Detection Algorithm for Personal Protective Equipment Based on Lightweight YOLOv4 Model[J]. Wireless Communications and Mobile Computing, 2022, 2022. [23] Wang Z, Wu Y, Yang L, et al. Fast personal protective equipment detection for real construction sites using deep learning approaches[J]. Sensors, 2021, 21(10): 3478. [24] 杜雪瑞. 基于深度学习的加油站卸油防护设备检测研究[D]. 大庆: 东北石油大学,2022. [25] Lo J H, Lin L K, Hung C C. Real-Time Personal Protective Equipment Compliance Detection Based on Deep Learning Algorithm[J]. Sustainability, 2022, 15(1): 391. [26] 李晓宇, 陈伟, 杨维, 等. 基于超像素特征与 SVM 分类的人员个人防护帽分割方法[J]. 煤炭学报, 2021, 46(06): 2009-2022. [27] 崔铁军, 王凌霄. YOLOv4 目标检测算法在煤矿工人口罩佩戴监测工作中的应用研 究[J].中国安全生产科学技术,2021,17(10):66-71. [28] 管利聪. 基于深度学习的下矿人员安保穿戴设备检测系统设计[D]. 青岛: 青岛科技大学, 2020. [29] 李熙尉, 孙志鹏, 王鹏, 陶虹京. 基于 YOLOv5s 改进的井下人员和安全帽检测算法研究[J]. 煤, 2023, 32(03): 22-25. [30] 代少升, 曾奇, 黄炼, 等. 基于 S3-YOLOv5s 的矿井人员防护设备检测算法研究[J].半导体光电, 2023, 44(01): 153-160. [31] Song H, Zhang X, Song J, et al. Detection and tracking of safety helmet based on DeepSort and YOLOv5[J]. Multimedia Tools and Applications, 2022, 82(7): 10781-10794. [32] Li F, Chen Y, Hu M, et al. Helmet Wearing Tracking Detection Based on StrongSORT[J].Sensors, 2023, 23(3): 1682. [33] 何超. 基于改进 YOLOv3 的安全帽检测系统研究[D]. 武汉: 华中科技大学, 2019. [34] 陈伟, 任鹏, 田子建, 等. 基于注意力机制的无监督矿井人员跟踪[J]. 煤炭学报,2021, 46(S1): 601-608. [35] 冉险生, 张之云, 陈卓, 等. 基于改进 DeepSORT 算法的摩托车头盔佩戴检测[J/OL].计算机工程与应用,2023:1-13. [36] 孙宏伟, 曹雪虹, 焦良葆, 等. 改进的 DeepSORT 发电厂人员跟踪算法[J]. 南京工程学院学报(自然科学版), 2022, 20( 3): 1-6. [37] 张旭辉, 闫建星, 张超, 等. 基于改进 YOLOv5s+DeepSORT 的煤块行为异常识别[J].工矿自动化, 2022, 48(6): 77-86, 117. [38] 邵小强, 李鑫, 杨涛, 等. 改进 YOLOv5s 和 DeepSORT 的井下人员检测及跟踪算法[J/OL]. 煤炭科学技术, 2023: 1-12. [39] Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems. 2015: 91-99. [40] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 779-788. [41] Farhadi A, Redmon J. Yolov3: An incremental improvement[C]//Computer vision and pattern recognition. Berlin/Heidelberg, Germany: Springer, 2018, 1804: 1-6. [42] Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection[OL]. [2021-09-09]. https://arxiv.org/abs/2004.10934. [43] Ge Z, Liu S T, WANG F, et al. YOLOX: Exceeding YOLO seriesin 2021[J]. arXiv: 2107.08430, 2021. [44] Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]//European Conference on Computer Vision, 2016: 21-37. [45] 张瑶, 卢焕章, 张路平, 等. 基于深度学习的视觉多目标跟踪算法综述[J]. 计算机工程与应用, 2021, 57(13): 55-66. [46] Ciaparrone G, Sánchez F L, Tabik S, et al. Deep learning in video multi-object tracking:A survey[J]. Neurocomputing, 2020, 381: 61-88. [47] Bewley A, Ge Z, Ott L, et al. Simple online and realtime tracking[C]//2016 IEEE international conference on image processing (ICIP). IEEE, 2016: 3464-3468. [48] 司垒, 王忠宾, 熊祥祥, 等. 基于改进 U-net 网络模型的综采工作面煤岩识别方法[J]. 煤炭学报, 2021, 46 (S1): 578-589. [49] Wang W, Wu X, Yuan X, et al. An experiment-based review of low-light image enhancement methods[J]. IEEE Access, 2020, 8: 87884-87917. [50] Li C, Guo C, Han L, et al. Low-light image and video enhancement using deep learning:A survey[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(12):9396-9416. [51] Wei C, Wang W, Yang W, et al. Deep retinex decomposition for low-light enhancement[J].2018: 1-12. [52] Zhang Y, Zhang J, Guo X. Kindling the darkness: A practical low-light image enhancer[C]//Proceedings of the 27th ACM international conference on multimedia. 2019:1632-1640. [53] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1-9. [54] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016: 2818-2826. [55] Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4510-4520. [56] Naik S V, Majjigudda S K, Naik S, et al. Survey on comparative study of pruning mechanism on MobileNetV3 model[C]//2021 International Conference on Intelligent Technologies (CONIT). IEEE, 2021: 1-8. [57] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7132-7141. [58] Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 13713-13722. [59] 杨永波, 李栋. 改进 YOLOv5 的轻量级安全帽佩戴检测算法[J]. 计算机工程与应用,2022, 58(9): 201-207. [60] 江泽涛, 肖芸, 张少钦, 等. 基于 Dark-YOLO 的低照度目标检测方法[J]. 计算机辅助设计与图形学学报, 2023, 35(03): 441-451. [61] Ma N, Zhang X, Zheng H T, et al. ShuffleNetV2: Practical guidelines for efficient CNN architecture design[C]//Proceedings of the 15th European Conference on Computer Vision(ECCV). 2018:116–131. [62] Han K, Wang Y, Tian Q, et al. Ghostnet: More features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 1580-1589. [63] Tan M, Le Q. Efficientnetv2: Smaller models and faster training[C]//International conference on machine learning. PMLR, 2021: 10096-10106. [64] Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1251-1258. [65] Hassanin M, Anwar S, Radwan I, et al. Visual attention methods in deep learning: An in-depth survey[J]. 2022: 1-20. [66] 王玲敏, 段军, 辛立伟. 引入注意力机制的 YOLOv5 安全帽佩戴检测方法[J]. 计算机工程与应用, 2022, 58(9): 303-312. [67] 刘彦清. 基于 YOLO 系列的目标检测改进算法[D]. 长春: 吉林大学, 2021. [68] 李红光, 于若男, 丁文锐. 基于深度学习的小目标检测研究进展[J]. 航空学报, 2021,42(7): 100-118. [69] Liu S, Huang D, Wang Y. Learning spatial fusion for single-shot object detection[J]. 2019:1-10. [70] 裴云成. 基于 DeepSORT 的行人多目标跟踪算法研究与应用[D]. 济南: 齐鲁工业大学, 2022. [71] 贾豆豆. 基于 YOLOv5+DeepSort 的小目标跟踪方法研究[D]. 太原: 中北大学, 2022. [72] 罗丽洁, 韩华, 金婕, 等. 基于轻量多分支网络的行人重识别方法[J]. 智能计算机与应用, 2022, 12(11): 103-110. [73] Zheng Z, Wang P, Liu W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[C]//Proceedings of the AAAI conference on artificial intelligence. 2020,34(07): 12993-13000. [74] 洪宇. 基于改进 DeepSORT 的水下鱼类计数方法研究[D]. 上海: 上海海洋大学, 2022. [75] 魏超宇. 基于深度学习与多目标跟踪的小番茄果实计数研究[D]. 杭州: 中国计量大学, 2021
﹀
|
中图分类号: |
TP302.7
|
开放日期: |
2023-06-15
|