综采工作面生产过程中，采煤机向前割煤时，如果滚筒附近的液压支架护帮板未正常收回，两者将会发生碰撞产生截割干涉，导致综采设备损伤，甚至威胁人员安全。同时，由于片帮等原因导致有大块煤落在刮板输送机上，可能会造成刮板输送机堵塞停机，降低综采工作效率。随着煤矿智能化水平的提升，煤矿井下常采用机器视觉手段对工作面进行监测。本文采用机器视觉方法，研究液压支架护帮板异常状态识别方法及刮板输送机大块煤图像识别方法，但是生产现场常会产生严重的粉尘，影响井下监控视频图像的清晰度，因此同时研究工作面低照度雾尘图像清晰化方法，提高综采工作面监控视频图像质量，本文研究对煤矿安全、高效开采具有重要意义。
       针对综采工作面监控图像雾尘严重、亮度低的问题，建立了雾尘图像清晰化模型对透射率函数进行求解，引入对数变换对透射率函数进行优化以提高图像亮度。同时，为了避免对数变换造成图像中矿灯等光源出现过亮的问题，提出了根据原始图像亮度峰值来选择对数变换倍数的方法。与多种目前广泛使用的图像去雾算法进行对比，结果表明提出的方法在视觉上和客观评价指标上都具有良好的效果，适用于综采工作面低照度、高雾尘环境，为后续液压支架护帮板异常状态的识别和刮板输送机大块煤识别提供高质量的图像。
       针对液压支架护帮板处于未收回异常状态的识别问题，提出了一种基于改进Faster R-CNN深度迁移学习的护帮板异常状态识别方法。该方法以Faster R-CNN深度学习方法作为主要框架，采用VGG16网络作为特征提取网络。为了提高识别准确率，将softmax分类器替换为Adaboost分类器，引入soft-NMS方法代替NMS方法，同时将已在ImageNet数据集训练好的VGG16网络的参数和权重进行迁移，得到护帮板异常状态识别模型，实现对液压支架护帮板异常状态准确识别。
       针对刮板输送机大块煤的识别问题，采用基于核模糊C均值聚类的方法对大块煤图像进行分割，结合图像形态学操作开运算和闭运算实现对煤块的重叠区域进行重新填充，然后通过外接矩形标注方法对大块煤的像素尺寸进行统计，最后根据相机标定投影原理求出比例系数，将大块煤外接矩形的像素尺寸转换为实际尺寸，实现了大块煤准确识别。
       为了验证本文提出的方法，运用黄陵二号煤矿综采工作面监控视频对液压支架护帮板异常状态识别方法和刮板输送机大块煤识别方法进行了试验验证，结果表明：液压支架护帮板异常状态识别模型在现场视频中的平均识别精度为90.02%，平均识别耗时为89.92ms，刮板输送机大块煤识别的准确率为90.86%。同时，基于上述理论方法设计了综采工作面异常状态图像识别软件系统，利用QT设计软件系统界面，与Python、OpenCV进行交互，实现了综采工作面雾尘视频图像的清晰化、液压支架护帮板异常状态检测、大块煤图像识别等功能。
 

    In the production process of fully mechanized mining face, when the shearer cuts coal forward, if the guard plate of hydraulic support near the drum is not normally retracted, the two will collide and produce cutting interference, resulting in damage to fully mechanized mining equipment and even threatening personnel safety. At the same time, large pieces of coal fall on the scraper conveyor due to wall and other reasons, which may block and shut down the scraper conveyor and reduce the working efficiency. With the improvement of intelligent level of coal mine, machine vision is often used to monitor the working face in coal mine. In this paper, the machine vision method is used to study the abnormal state recognition method of hydraulic support side guard and the image recognition method of scraper conveyor large coal. However, serious dust is often produced on the production site, which affects the definition of underground monitoring video image. Therefore, at the same time, the method of clarifying low illumination fog dust image of working face is studied to improve the quality of monitoring video image of fully mechanized mining face. This paper studies the impact on coal mine safety efficient mining is of great significance.
    Aiming at the problems of serious fog and dust and low brightness in the monitoring image of fully mechanized mining face, a fog and dust image clarity model is established to solve the transmittance function, and the logarithmic transformation is introduced to optimize the transmittance function to improve the image brightness. At the same time, in order to avoid the problem of over illumination of miner's lamp and other light sources in the image caused by logarithmic transformation, a method of selecting logarithmic transformation multiple according to the brightness peak of the original image is proposed. Compared with a variety of image defogging algorithms widely used at present, the results show that the proposed method has good results in vision and objective evaluation indexes. It is suitable for the low illumination and high fog and dust environment of fully mechanized mining face, and provides high-quality images for the subsequent identification of abnormal states of hydraulic support guard plate and the identification of large coal of scraper conveyor.
    Aiming at the problem of identifying the abnormal state of the guard plate of hydraulic support, an abnormal state identification method of the guard plate based on deep transfer learning is proposed. This method takes Faster R-CNN deep learning method as the main framework and VGG16 network as the feature extraction network. In order to improve the recognition accuracy, the softmax classifier is replaced by AdaBoost classifier, and the soft NMS method is introduced to replace the NMS method. The parameters and weights of VGG16 network trained in ImageNet data set are transfered to obtain the abnormal state recognition model of the upper guard plate, so as to realize the accurate recognition of the abnormal state of the guard plate of hydraulic support.
    Aiming at the recognition of large coal of scraper conveyor, the method based on Kernel Fuzzy C-means clustering is used to segment the image of large coal, and the overlapping area of coal is refilled by combining the open operation and closed operation of image morphology operation. Then the pixel size of large coal is counted by the external rectangle marking method, and finally the proportion coefficient is calculated according to the camera calibration projection principle, the pixel size of the external rectangle of large coal is converted into the actual size, and the accurate recognition of large coal is realized.
    In order to verify the method, the monitoring video of the fully mechanized mining face of Huangling No. 2 coal mine is used to test and verify the abnormal state recognition method of the hydraulic support side guard and the large coal recognition method of the scraper conveyor. The results show that the average recognition accuracy of the abnormal state recognition model of the hydraulic support side guard in the field video is 90.02%, the average recognition time is 89.92ms, and the accuracy of the large coal recognition of the scraper conveyor is 90.86%. At the same time, based on the above theoretical methods, the abnormal state image recognition software system of fully mechanized mining face is designed. Using the QT design software system interface, it interacts with Python and OpenCV to realize the functions of clarity of fog and dust video image of fully mechanized mining face, abnormal state detection of hydraulic support sideboard, image recognition of large coal and so on.
 

[1] 钱鸣高，许家林，王家臣. 再论煤炭的科学开采[J]. 煤炭学报，2018,(01): 1-13.

[2] 宋秋爽，袁智，宋振铎，等.综采成套装备试验平台关键技术研究[J]. 煤炭科学技术，2017,45(08):194-199.

[3] 李霞，崔涛.我国煤炭资源可持续发展的保障分析[J]. 中国煤炭，2019, 45(01): 33-37.

[4] 王国法.煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术，2022,50(01):1-27.

[5] 王国法.工作面支护与液压支架技术理论体系[J]. 煤炭学报，2014,39(08):1593-1601.

[6] 赵巧芝.我国刮板输送机发展现状、趋势及关键技术[J]. 煤炭工程，2020,52(08):183-187.

[7] 蒋华伟，杨震，张鑫，等.图像去雾算法研究进展[J]. 吉林大学学报(工学版)，2021,51(04):1169-1181.

[8] Oakley J P, Bu H. Correction of simple contrast loss in clor images[J]. IEEE Transactions on Image Processing,2012,16(2):511-522.

[9] Zhang Y, Ding L, Sharma G. Hazerd: an outdoor scene dataset and benchmark for single image dehazing[C]. 2017 IEEE international conference on image processing (ICIP). IEEE, 2017: 3205-3209.

[10] Yuan K, Wei J, Lu W, et al. Single Image Dehazing via NIN-DehazeNet[J]. IEEE Access, 2019, 7: 181348-181356.

[11] Ren W, Liu S, Zhang H, et al. Single image dehazing via multi-scale convolutional neural networks[C]. European conference on computer vision(ECCV). Springer, Cham, 2016: 154-169.

[12] Ni W, Gao X, Wang Y. Single satellite image dehazing via linear intensity transformation and local property analysis[J]. Neurocomputing, 2016, 175: 25-39.

[13] Tan R T. Visibility in bad weather from a single image[C].2008 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2008: 1-8.

[14] Fang F, Wang T, Wang Y, et al. Variational Single Image Dehazing for Enhanced Visualization[J]. IEEE Transactions on Multimedia, 2019.

[15] Fattal R. Single image dehazing[J]. ACM transactions on graphics, 2008, 27(3): 1-9.

[16] Yang M, Liu J, Li Z, et al. Pre-processing for single image dehazing[J]. Signal Processing: Image Communication, 2020: 115777.

[17] He K, Sun J, Tang X. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 33(12): 2341-2353.

[18] Zhang X. Research on remote sensing image dehaze based on GAN[J]. Journal of Signal Processing Systems, 2022, 94(3): 305-313.

[19] Huang S C, Chen B H, Wang W J. Visibility restoration of single hazyimages captured in real-world weather conditions[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(10): 1814-1824.

[20] He K M, Sun J, T X O. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409.

[21] Hassan N, Ullah S, Bhatti N, et al. A cascaded approach for image defogging based on physical and enhancement models[J]. Signal, Image and Video Processing, 1-9.

[22] Ju M, Ding C, Guo Y J, et al. Image Dehazing Based on Gamma Correction Prior[J]. IEEE Transactions on Image Processing, 2019.

[23] Rong Z, Jun W L. Improved wavelet transform algorithm for single image dehazing[J]. Optik, 2014, 125(13): 3064-3066.

[24] Zhang L, Wang S, Wang X. Saliency-based dark channel prior model for single image haze removal[J]. IET Image processing, 2018, 12(6): 1049-1055.

[25] Salazar S, Ramos J M, Echeverri C J O, et al. Image dehazing using morphological opening, dilation and Gaussian filtering[J]. Signal, Image and Video Processing, 2018, 12(7): 1329-1335.

[26] Z. Wang,Y. Feng. Fast single haze image enhancement[J]. Computers and Electrical Engineering, 2014,40(3):785-795.

[27] Zhang W, Dong L, Pan X, et al. Single image defogging based on multi-channel convolutional MSRCR[J]. IEEE Access, 2019, 7: 72492-72504.

[28] 智宁，毛善君，李梅，等.基于深度融合网络的煤矿图像尘雾清晰化算法[J]. 煤炭学报，2019,44(02):655-666.

[29] 智宁,毛善君,李梅.基于照度调整的矿井非均匀照度视频图像增强算法[J]. 煤炭学报，2017,42(08):2190-2197.

[30] 智宁，毛善君，李梅.基于双伽马函数的煤矿井下低亮度图像增强算法[J]. 辽宁工程技术大学学报(自然科学版)，2018,37(01):191-197.

[31] 张谢华，张申，方帅，等.煤矿智能视频监控中雾尘图像的清晰化研究[J]. 煤炭学报，2014,39(01):198-204.

[32] 任怀伟，李帅帅，赵国瑞，等.基于深度视觉原理的工作面液压支架支撑高度与顶梁姿态角测量方法研究[J]. 采矿与安全工程学报，2022,39(01):72-81+93.

[33] 张坤.基于信息融合技术的液压支架姿态监测方法研究[D]. 太原：太原理工大学，2018.

[34] 张坤，廉自生.液压支架姿态角度测量系统[J].工矿自动化，2017, 43(5): 40-45.

[35] 张旭辉，王冬曼，杨文娟.基于视觉测量的液压支架位姿检测方法[J].工矿自动化，2019,45(3):56-60.

[36] 王渊，李红卫，郭卫，等.基于图像识别的液压支架护帮板收回状态监测方法[J].工矿自动化，2019,45（2）:47-53.

[37] 满溢桥.液压支架护帮板与采煤机滚筒截割干涉监测技术研究[D]. 北京：中国矿业大学（北京），2019.

[38] OJALA T, P IETIKAINEN M, HARWOOD D. A comparative study of texture measures with classification based on featured distributions[J]. Pattern recognition, 1996, 29(1):51-59.

[39] 方承志，袁海峰. 基于 CMYK-H-CbCr 肤色检测和改进型 AdaBoost 算法的人脸检测[J]. 计算机应用与软件，2017, 34(8): 167-172.

[40] 田仙仙，鲍泓，徐成. 一种改进HOG特征的行人检测算法[J]. 计算机科学，2014,41(09):320-324.

[41] 文学志，方巍，郑钰辉，等. 一种基于类 Haar 特征和改进 AdaBoost 分类器的车辆识别算法[J]. 电子学报，2011, 39(5): 1121-1126.

[42] 林英男. 基于标识的双目视觉室内定位算法研究[D]. 哈尔滨：哈尔滨工业大学, 2017.

[43] Pednault E P D. Statistical Learning Theory[J]. MIT Encyclopedia of the Cognitive Sciences, 1997, 1-4.

[44] 刘兵，孙超，王旭艳，等. 基于模糊对向传播网络的水中目标分类识别[J]. 计算机工程与应用，2007，43(28): 227-229.

[45] Breiman L. Random forests[J]. Machine learning, 2001, 45: 5-32.

[46] Li J, Liang X, Shen S M, et al. Scale-aware fast R-CNN for pedestrian detection [J]. IEEE t-ransactions on Multimedia, 2017, 20(4): 985-996.

[47] Li Y,He K,Sun J.R-FCN: object detection via region-based fully convolutional networks[C].Advances in Neural Information Processing Systems.2016:379-387.

[48] Lecun Y,Boser B,Denker J S,et al.Backpropagation Applied to Handwritten Zip Code Recognition[J]. Neural Computation,2014,1(4):541-551.

[49] Girshick R.Fast R-CNN[C].Proc of IEEE Intemational Conference on Computer Vision(ICCV). 2015:1440-1448.

[50] Liu W,Anguelov D,Erhan D,et al.SSD:Single Shot MultiBox Detector[C].European Conference on Computer Vision(ECCV).2016:21-37.

[51] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C].Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.

[52] Kumar A, Zhang Z J, Lyu H. Object detection in real time based on improved single shot multi-box detector algorithm[J]. EURASIP Journal on Wireless Communications and Networking, 2020, 2020(1): 1-18.

[53] 唐士宇，朱艾春，张赛，等.基于深度卷积神经网络的井下人员目标检测[J]. 工矿自动化，2018,44(11):32-36.

[54] 杜京义，陈瑞，郝乐，等.煤矿带式输送机异物检测[J]. 工矿自动化，2021,47(08):77-83.

[55] Wang Y, Wang Y, Dang L. Video detection of foreign objects on the surface of belt conveyor underground coal mine based on improved SSD[J]. Journal of Ambient Intelligence and Humanized Computing, 2020: 1-10.

[56] 王家臣，潘卫东，张国英，等.图像识别智能放煤技术原理与应用[J]. 煤炭学报，2022,47(01):87-101.

[57] 鞠默然，罗海波，王仲博，等.改进的YOLO V3算法及其在小目标检测中的应用[J]. 光学学报，2019,39(07):253-260.

[58] 唐聪，凌永顺，郑科栋，等.基于深度学习的多视窗SSD目标检测方法[J]. 红外与激光工程，2018,47(01):302-310.

[59] 李阳．矿用核子皮带秤在线监测系统[D]．西安：西安科技大学，2017．

[60] 杜京义，郝乐，王悦阳，等.一种煤矿井下输煤大块物检测方法[J]. 工矿自动化，2020,46(05):63-68.

[61] 许军，吕俊杰，杨娟利，等.基于图像处理的溜槽堆煤预警研究[J]. 煤炭技术，2017,36(12):232-234.

[62] 鲍勇豪.基于视觉信息识别的煤流控制系统[J]. 电子技术与软件工程，2019(10):55.

[63] 刘丽霞，李宝文，王阳萍，等.改进Canny边缘检测的遥感影像分割[J]. 计算机工程与应用，2019,55(12):54-58+180.

[64] 王健生，郭鹏，郝震，等.基于视觉测量的电厂输煤胶带堆煤检测[J]. 工矿自动化,2016,42(08):43-47.

[65] 袁姮，王志宏，姜文涛. 熵能量堆煤图像的定位与识别[J]. 中国图象图形学报, 2015,20(08):1062-1069.

[66] 余乐.一种煤和煤矸石图像识别的新方法[J]. 现代计算机(专业版)，2017(17):66-70.

[67] 吴开兴，宋剑.基于灰度共生矩阵的煤与矸石自动识别研究[J]. 煤炭工程，2016，48（02）：98-101.

[68] 程健，王东伟，杨凌凯，等.一种改进的高斯混合模型煤矸石视频检测方法[J]. 中南大学学报（自然科学版），2018,49(1):118-123.

[69] Zhao Y，Sun M. Image Processing and Recognition System Based on DaVinci Technology for Coal and Gangue[J]. Applied Mechanics&Materials，2011，130-134：2107-2110.

[70] 李铁莲，罗昌绩.矿井综采工作面自动化系统方案研究[J]. 能源与节能，2014(8):70-71.

[71] 李艳伟，宋建成，雷志鹏，等.工作面“三机”状态监测装置改进设计[J]. 工矿自动化，2013,39(11):12-16.

[72] 张健，苗向民.放顶煤液压支架护帮板结构改进分析[J]. 煤矿机械，2014(12):194-196

[73] 秦艳华.液压支架护帮装置的结构形式与选型分析[J]. 煤矿机械，2014(12):185-186

[74] 宋高峰. 大采高工作面煤壁稳定性分析及控制分析[D]. 北京：中国矿业大学(北京)，2017.

[75] 曹正坤，赵小东.综采工作面大块煤堵塞问题的解决措施[J]. 煤矿机械，2019,40(09)：171-173.

[76] Zhang W, Dong L, Pan X, et al. Single image defogging based on multi-channel convolutional MSRCR[J]. IEEE Access, 2019, 7: 72492-72504.

[77] Meng G, Wang Y, Duan J, et al. Efficient image dehazing with boundary constraint and contextual regularization[C]. Proceedings of the IEEE international conference on computer vision. 2013: 617-624.

[78] Galdran A. Image dehazing by artificial multiple-exposure image fusion[J]. Signal Processing, 2018, 149: 135-147.

[79] Ehsan S M, Imran M, Ullah A, et al. A Single Image Dehazing Technique Using the Dual Transmission Maps Strategy and Gradient-Domain Guided Image Filtering[J]. IEEE Access, 2021, 9: 89055-89063..

[80] Berman D, Treibitz T, Avidan S. Air-light estimation using haze-lines[C]. 2017 IEEE Internati-onal Conference on Computational Photography (ICCP). IEEE, 2017: 1-9.

[81] Trinh H-H,Ngo M-D,Dinh V-T.HOG and geometrical model based moving vehicle detection[C]. 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA).IEEE,2017:1899-1904.

[82] Neumann D,Langner T,Ulbrich F,et al.Online vehicle detection using Haar-like,LBP and HOG feature based image classifiers with stereo vision preselection[C]. 2017 IEEE Intelligent Vehicles Symposium (IV).IEEE,2017:773-778.

[83] 周飞燕，金林鹏，董军. 卷积神经网络研究综述[J]. 计算机学报，2017, 40(6): 1229-1251

[84] M. D. Zeiler, R. Fergus. Stochastic pooling for regularization of deep convolutional neural net-works[J]. Proceedings of the International Conference on Learning Representations (ICLR),2013, 1: 1-8

[85] A. Mikolov, S. Kombrink, L. Burget, et al. Extensions of recurrent neural network language model[C]. Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, 2011, 5528-5531

[86] Bodla N, Singh B, Chellappa R, et al. Soft-NMS — Improving Object Detection with One Line of Code[C]. 2017 IEEE International Conference on Computer Vision (ICCV), August 8, 2017: 5562-5570.

[87] 汤勃，孔建益，伍世虔. 机器视觉表面缺陷检测综述[J]. 中国图象图形学报，2017,22(12): 1640-1663.

[88] 赵文昌，李忠木. 融合改进人工蜂群和K均值聚类的图像分割[J]. 液晶与显示，2017,32(9): 726-735.

[89] 隋心怡, 王瑞刚, 张鸿翔. 一种改进的K-均值聚类算法[J]. 计算机与数字工程，2018, 46(4): 682-685.

[90] 左进，陈泽茂. 基于改进 K 均值聚类的异常检测算法[J]. 计算机科学，2016, 43(8): 258-261.

[91] 金日. 基于模糊c均值聚类算法的主动轮廓模型研究[D]. 苏州：苏州大学，2020.

[92] 赵海峰，陈书海.模糊隶属度加权的KFCM脑MRI的组织分割方法[J]. 计算机辅助设计与图形学学报，2018, 30(11):80-87.

[93] 金大䑐（韩），张红艳译.QT5开发实战[M]. 北京：人民邮电出版社，2015.