煤矿井下环境昏暗，空气中弥漫着大量煤灰，导致监控画面模糊不清晰，在视频监测时会存在遮挡以及误检率高等问题，为保障井下人员安全，基于视频监控信息的矿井人员检测与跟踪对于煤矿安全生产具有重要意义。本文具体工作如下：

     (1)针对矿井视频监控中低照度且图像模糊问题，提出一种基于引导滤波与Retinex理论融合的矿井图像增强算法。首先将RGB颜色模型转换至HSV模型中，采用引导滤波器作为Retinex算法的中心环绕函数，在亮度通道获取不同分量信息。然后，对低频光照分量进行Gamma校正，对高频反射分量用引导滤波去噪，根据亮度变化对饱和度进行校正，最后将图像由HSV模型转换至RGB模型。实验结果表明，本文算法增强了图像细节轮廓信息，人员与环境可以清晰分辨。

     (2) 针对矿井下多尺度、小目标矿工检测不佳的问题，选用Faster RCNN网络模型对矿井人员进行检测。首先，通过对RPN网络结构进行改进，在VGG16网络最后一层使用1×1、3×3、5×5的滑动窗口生成候选区域，然后使用12种不同尺度的Anchor来实现对多尺度矿工的候选区域提取，最后将改进的RPN网络结构和特征融合技术进行级联。实验结果表明，采用改进的Faster RCNN算法在矿井数据集上得到0.08s/frame的检测速度和91.35%的准确率，比Faster RCNN算法分别提高0.06s/frame和5.72%。

     (3) 针对SimaFC跟踪方法对矿井人员特征提取不充分，导致跟踪效果不佳的问题，提出了一种轻量级网络的混合注意力孪生神经网络跟踪算法。首先，将改进后的轻量级网络MobileNetV3作为特征提取骨干网络，提取到更具有表达能力的特征。然后，设计了混合注意力模型，对提取的矿工特征进行修饰，提高网络的判别能力。最后为了获得更准确的跟踪结果，使用不同部分的特征向量做互相关的结果做加权平均，以提高模型的性能。在不同矿井场景下进行矿井人员跟踪实验，与其他主流跟踪算法对比。实验结果表明，本文跟踪算法可以对井下矿工目标实时有效跟踪，在矿井数据集上平均精确度和成功率达到75.7%和65.3%，比SimaFC方法分别提高了6.2%和6.6%。

     本文方法对矿井视频图像增强和井下人员定位有一定参考价值，为煤矿的安全生产提供了有益支撑。

关 键 词：矿井图像增强；目标检测；Faster RCNN；目标跟踪；孪生神经网络

研究类型：应用研究

  Due to uneven illumination, low illumination and large dust, video imaging is mixed with noise in underground coal mine, there will be problems of occlusion and high error detection rate in video monitoring. In order to ensure the safety of underground personnel, the detection and tracking of mine personnel based on video monitoring information is of great significance for the safety of coal mine production. The specific work of this paper is as follows:

  (1)Aiming at the problem of low illumination and image blur in mine video monitoring, a fusion method of guided filtering and Retinex algorithm was proposed under the condition of HSV space transformation. Firstly, the RGB color model was transformed into HSV model, and the improved Retinex algorithm was used to obtain different component information in the brightness channel. Then, the low frequency illumination component is Gamma corrected, the high frequency reflection component is denoised by guided filtering, and the saturation is corrected according to the brightness variation. Finally, the image is converted from HSV model to RGB model. Simulation results show that the improved algorithm is superior to the traditional enhancement algorithm in terms of standard deviation, information entropy and average gradient.

  (2)To solve the problem of poor detection of multi-scale and small-target miners in mines, improved the Faster RCNN detection algorithm. Firstly, by improving the RPN network structure, candidate regions are generated by using 1×1, 3×3 and 5×5 sliding Windows in the last layer of VGG16 network. Then, Anchor of 12 different scales is used to extract candidate regions of multi-scale miners. Finally, the improved RPN network structure and feature fusion technology are cascated. The experimental results show that the improved Faster RCNN algorithm can obtain the detection speed of 0.08s/frame and the accuracy of 91.35% on the mine data set, which are 0.06s/frame and 5.72% higher than the improved Faster RCNN algorithm.

  (3)Aiming at the problems of the traditional mine personnel target tracking algorithm, such as slow modeling and updating speed, large computation and unable to meet the real-time effective tracking, a lightweight network mixed attention twin neural network is proposed. Firstly, the improved lightweight network MobileNetV3 is used as the backbone network for feature extraction to extract more expressive features. Finally, in order to obtain more accurate tracking results, feature vectors of different parts are used to make cross-correlation results for weighted average in the generation of similarity score response graph, so as to improve the performance of the model. Mine personnel tracking experiments are carried out in different mine scenes and compared with other target tracking algorithm models. Experimental results show that the proposed algorithm can effectively track the targets of underground miners in real time, and the average accuracy and success rate of mine data sets reach 75.7% and 65.3%.

  The method presented in this paper has certain reference value for video image enhancement and underground personnel positioning, which lays a foundation for safety production of coal mine.

Key words: Mine Image Enhancement; Target Detection; Faster RCNN; Target Tracking; Siamese Network

Thesis       : Application Research

[1]王国法, 赵国瑞, 任怀伟. 智慧煤矿与智能化开采关键核心技术分析[J]. 煤炭学报, 2019, 44(1): 34-41.

[2]Ashish K, Handari B. A logarithmic law based histogram modification scheme for naturalness image contrast enhancement[J]. Journal of Ambient Intelligence and Humanized Computing, 2020, 11(4): 308-314.

[3]Galdran A, Vazquezcorral J, Pardo D, et al. Fusion-Based Variational Image Dehazing[J]. IEEE Signal Processing Letters, 2017, 24(2): 151-155.

[4]Subramani B, Veluchamy M. Fuzzy Gray Level Difference Histogram Equalization for Medical Image Enhancement[J]. Journal of medical systems, 2020, 6(44): 1568-1569.

[5]戴声奎, 钟峥. 基于最大熵模型的双直方图算法[J]. 电子学报, 2019, 3: 678-685.

[6]Kalaivani S, Shantharajah S. Agricultural leaf blight disease segmentation using indices based histogram intensity segmentation approach[J]. Multimedia tools and applications, 2020, 79(13): 9145-9159.

[7]Fu Q, Celenk M, Wu A. An improved algorithm based on CLAHE for ultrasonic well logging image enhancement[J]. Cluster Computing, 2019, 22(2): 12609-12618.

[8]Kou F, Wei Z, Chen W, et al. Intelligent detail enhancement for exposure fusion[J]. IEEE Transactions on Multimedia , 2018, 20(2): 484-495.

[9]Wang B, Li C Y. A visual hierarchical frame work based model for underwater image enhancement [J]. Frontiers of Computer Science, 2019, 13(3): 665-667.

[10]Subramani B, Veluchamy M. Fuzzy Gray Level Difference Histogram Equalization for Medical Image Enhancements[J]. Journal of medical system, 2020, 6(44): 1568-1569.

[11]Jiao T Y, Zhang J S, Wang Y. Wheel image enhancement based on wave let analysis and pseudo-color processings[J]. Automation and Instrumentation, 2020, 35(1): 47-51.

[12]范伟强, 刘毅. 基于自适应小波变换的煤矿降质图像模糊增强算法[J]. 煤炭学报, 2020, 45(12): 4248-4260.

[13]王满利, 田子健. 基于非下采样轮廓波变换的矿井图像增强算法[J]. 煤炭学报, 2020, 45(9): 3351-3362.

[14]张立亚, 郝博南, 孟庆勇, 等. 基于HSV空间改进融合Retinex算法的井下图像增强方法[J]. 煤炭学报, 2020, 45(S1): 532-540.

[15]Kaur P, Khehra B, Pharwhah A P. Color Image Enhancement based on Gamma Encoding and Histogram Equalization[J]. Materials Today, 2021, 46(9): 4025-4030.

[16]Kotoula E, Robinson D W, Bedford C. Interactive relighting, digital image enhancement and inclusive diagrammatic representations for the analysis of rock art superimposition [J]. Journal of Archaeological Science, 2018, 93(5): 26-41.

[17]Long M Z, Lan Z, Xie X, et al. Adaptive Image Enhancement Based on Guide Image and Fraction-Power Transformation for Wireless Capsule Endoscopy [J]. IEEE transactions on biomedical circuits and systems, 2018, 12(5): 993-1003.

[18]Zhang L, Xu Q, Zhu G, et al. Improved colour-to-grey method using image segmentation and colour difference model for colour vision deficiency[J]. IET Image Processing, 2018, 12(3): 314-319.

[19]许凤麟, 苗玉彬. 基于彩色加权引导滤波-Retinex算法的导航图像增强[J]. 上海交通大学学报, 2019, 53(8): 921-927.

[20]SHQ H, JIE X, KE S. Image haze removal based on rolling deep learning and Retinex theory[J]. IET Image Processing, 2021, 16(2): 485-498.

[21]吕建中. 基于图像的井下人员检测算法研究[D]. 重庆: 重庆大学, 2015.

[22]智宁, 毛善君, 李梅. 基于照度调整的矿井非均匀照度视频图像增强算[J]. 煤炭学报, 2017(8): 2190-2197.

[23]Bhandari A K, Kumar A, Singh G K. Improved transfer function and gamma correction based method for contrast and brightness enhancement of satellite image[J]. AEU-International Journal of Electronics and Communications, 2015, 69(2): 579-589.

[24]唐守峰, 史可. 一种矿井低照度增强算法[J]. 工矿自动化, 2021, 47(10): 32-36.

[25]朱礼仪. 矿井图像增强和井下人员检测算法的研究[D]. 徐州：中国矿业大学，2019.

[26]Baoping W, Jianjun M, Zhaoxuan H, et al. Adaptive image enhancement algorithm based on fuzzy entropy and human visual characteristics[J]. Journal of Systems Engineering and Electronics. 2018, 29(5): 1079-1088

[27]Park S, Yu S, Kim M, et al. Dual Autoencoder Network for Retinex Based Low-Light Image Enhancement[J]. IEEE Access. 2018: 22084-22093.

[28]陈啸宇. 多目标检测理论与方法研究[D]. 成都: 电子科技大学, 2021.

[29]Viola P, Jones M. Rapid Object Detection using a Boosted Cascade of Simple Features[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2003: 1-511.

[30]Dalal N, Triggs B. Histograms of Oriented Gradients for Human Detection[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2005: 886-893.

[31]Felzenszwalb P F, Huttenlocher D P. Pictorial Structures for Object Recognition[J]. International Journal of Computer Vision, 2005, 61(1): 55-79.

[32]Dollar P, Tu Z, Perona P, et al. Integral Channel Features[C]// British Machine Vision Conference(BMVC), 2009: 556-567.

[33]Dollar P, Wojek C, Schiele B, et al. Pedestrian detection: A benchmark[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2010: 304-311.

[34]Mathias M, Benenson R, Timofte R, et al. Handling Occlusions with Franken Classifiers[C]// IEEE International Conference on Computer Vision(ICCV), 2014: 1505-1512.

[35]Girshick R, Donahue J, Darrell T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2014: 580-587.

[36]He K, Zhang X, Ren S, et al. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2015, 37(9): 1904-1916.

[37]夏胡云, 叶学义, 罗霄晗, 等. 多尺度空间金字塔池化PCANet的行人检测[J]. 计算机工程, 2019, 45(2): 270-277.

[38]Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.

[39]任少卿. 基于特征共享的高效物体检测[D]. 北京: 中国科学技术大学, 2016.

[40]Dai J, Li Y, He K, et al. R-FCN: Object Detection via Region-based Fully Convolutional Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 35(9): 1604-1612.

[41]Sengar S S, Mukhopadhyay S. Detection of moving objects based on enhancement of optical flow[J]. Optik International Journal for Light and Electron Optics, 2017, 145: 130-141.

[42]徐守坤, 邱亮, 李宁, 等. 基于HOG-CSLBP及YOLOv2的行人检测[J]. 计算机工程与设计, 2019, 40(10): 2964-2968.

[43]周纪强. 监控视频中多累目标检测与多目标跟踪算法研究[D]. 哈尔滨: 哈尔滨工业大学, 2017.

[44]孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 045(7): 1244-1260.

[45]王旭东, 王屹炜, 闫贺. 背景抑制直方图模型的连续自适应均值漂移跟踪算法[J]. 电子与信息学报, 2019, 41(6): 1480-1487.

[46]高毅. 基于步态识别的跨场景多目标跟踪方法[J]. 控制工程, 2021, 28(7): 1375-1381.

[47]Lukezic A, Vojir T, Cehovin L, et al. Discriminative correlation filter with channel and spatial reliability[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2017: 4847-4856.

[48]Fan H, Ling H. SANet: Structure-aware network for visual tracking[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2017: 42-49.

[49]房立金, 米晓晗. 基于快速判别尺度空间的TLD跟踪算法[J]. 华中科技大学学报(自然科学版), 2020, 48(1): 26-31.

[50]Henriques J F, Caseiro R, Martins P, et al. High-Speed Tracking with Kernelized Correlation Filters[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2015, 37(3): 583-596.

[51]Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 24(5): 610-619.

[52]包本刚. 融合多特征的目标检测与跟踪方法[J].电子测量与仪器学报, 2019, 33(9): 93-99.

[53]Zhang N, Xu A, Wang K, et al. Remaining useful life prediction of lithium batteries based on extended Kalman particle filter[J]. IEEJ Transactions on Electrical and Electronic Engineering, 2021, 16(2): 206-214.

[54]Havangi R. Intelligent adaptive unscented particle filter with application in target tracking[J]. Signal Image and Video Processing, 2020, 14(2): 1-9.

[55]Liu S, Huang D, Wang Y. Adaptive nms: Refining pedestrian detection in a crowd[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 6459­6468.

[56]昝孟恩, 周航, 韩丹, 等. 粒子滤波目标跟踪算法综述[J]. 计算机工程与应用, 2019, 55(5): 8-17.

[57]Zhang S, Wen L, Bian X, et al. Single­shot refinement neural network for object detection[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 4203­4212.

[58]Tian Z, Shen C, Chen H, et al. Fcos: Fully convolutional one­stage object detection[C]// IEEE/CVF International Conference on Computer Vision (ICCV), 2019: 9627­9636.

[59]Bertinetto L, Valmadre J, Henriques F, et al. Fully convolutional siamese networks for object tracking[C]// Procceedings of the European Conference on Computer Vision (ECCV), 2016: 850-865

[60]Sun P, Zhang R, Jiang Y, et al. Sparse r­cnn: End­to­end object detection with learnable proposals[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 14454­14463.

[61]Jeong J, Verma V, Hyun M, et al. Interpolation based semi­supervised learning for object detection.[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 11602­11611.

[62]Yang Q, Wei X, Wang B, et al. Interactive self­training with mean teachers for semi­supervised object detection.[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), 2021: 5941­5950.

[63]田朗, 黄平牧, 吕铁军. SA-Siam++: 基于双分支孪生网络的目标跟踪算法[J]. 北京邮电大学学报, 2019, 42(6): 105-110.

[64]Wang X, Huang T, Gonzalez J, et al. Frustratingly simple few­shot object detection[C]// International Conference on Machine Learning (ICML), 2020: 9919­9928.

[65]Chao Y, Vijayanar S, Seybold B, et al. Rethinking the Faster RCNN architecture for temporal action localization[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), 2018: 1130­1139.

[66]Liu L, Ouyang W, Wang X, et al. Deep learning for generic object detection: A survey[J]. International journal of computer vision, 2020, 128 (2): 261-318.

[67]Jie H, Li S, Gang S, et al. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2018: 7132-7141.

[68]任欢, 王旭光. 注意力机制综述[J]. 计算机应用. 2021, 41(S1): 1-6.

[69]Gao Y, Xu J, Lin S, et al. Gcnet: Non-local networks meet squeeze-excitation networks and beyond [C]// IEEE International Conference on Computer Vision Workshop(ICCVW), 2019: 1971-1980.

[70]Wu Y, Lim J, Yang M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848.