近年来，虽然我国煤炭行业的安全生产情况持续好转，但是易导致煤矿安全生产事故的不安全行为时有发生，而不安全行为的前提是不安全状态。因此，及时检测煤矿从业人员岗前不安全状态，从根源上杜绝不安全行为，可以在一定程度上预防煤矿安全事故。本研究选取不安全状态中对煤矿从业人员影响较大的疲劳状态，通过提取人脸与姿态的相关检测指标，运用计算机视觉技术，构建煤矿从业人员不安全状态检测系统，以期达到预防和减少煤矿安全事故发生的目的。论文的主要研究内容和结论如下：

（1）开展了不安全行为、不安全状态与疲劳的理论研究。分析三者关系，确定不安全状态的内涵，明确不安全状态的分级。通过对比不同疲劳程度与不同等级不安全状态可能导致的后果，得出疲劳与不安全状态的映射关系：非疲劳与轻度疲劳对应Ⅰ级安全状态，中度疲劳对应Ⅱ级较不安全状态，重度疲劳对应Ⅲ级完全不安全状态。

（2）设计了人脸与姿态不安全状态检测算法。运用Media Pipe多媒体机器学习框架设计人脸不安全状态检测算法，识别睁闭眼、张闭嘴状态，检测眨眼频率、打哈欠频率；运用Open Pose算法设计姿态不安全状态检测算法，识别头部与背部的异常动作，检测头部左右旋转角Roll、背部胸腔弯曲度Bend。

（3）利用疲劳诱发实验测定了人脸检测阈值与姿态检测范围。实验时间为连续的24小时，将该时间段平均划分为12次进行测试，依次计为T₁、T₂、T₃、T₄、T₅、T₆、T₇、T₈、T₉、T₁₀、T₁₁、T₁₂。选取20个男生做为被试，每隔2小时对所有被试进行疲劳测定以及反应准确率的判断。经过实验发现，被试在T₁时间段处于非疲劳程度，T₂时间段处于轻度疲劳程度，T₄时间段达到中度疲劳程度，T₆时间段达到重度疲劳程度。并分别测定被试在T₁、T₂、T₄、T₆时间段的人脸与姿态数据，最终得到Ⅰ级、Ⅱ级、Ⅲ级不安全状态的人脸检测阈值与姿态检测范围。

（4）构建了系统融合模型检测不安全状态。一种融合模型是将眼部、嘴部、头部、背部特征分别做为单个向量进行检测，另一种融合模型是将眼部、嘴部特征组合成向量一，头部、背部特征组合成向量二，这两种模型都是先通过SVM训练后再进行D-S融合。最终训练得到模型一的检测准确率为85.23%，模型二的检测准确率为93.65%，可知组合特征模型的检测效果更好。

In recent years, although the safety production situation of our country's coal industry has continued to improve, unsafe behaviors that easily lead to safety production accidents in coal mines have occurred from time to time, and the premise of unsafe behavior is unsafe state. Therefore, timely detection of the pre-job unsafe state of coal mine employees and the root cause of unsafe behavior can prevent coal mine safety accidents to a certain extent. This paper selected fatigue that has a greater impact on coal mine employees in the unsafe state, extracted detection indicators of face and posture, and useed computer vision technology to construct an unsafe state detection system for coal mine employees, in order to prevent and reduce coal mine safety accidents. The main research contents and conclusions of the paper are as follows:

(1) Theoretical research on unsafe behavior, unsafe state and fatigue was carried out. Analyzed their relationship, determined and clarified unsafe state. By comparing the possible consequences of different fatigue degrees and different levels of unsafe states, obtained the mapping relationship between fatigue and unsafe states: non-fatigue and mild fatigue correspond to level I safe state, moderate fatigue corresponds to level II less safe state, Severe fatigue corresponds to a level III completely unsafe state.

(2) A face and posture unsafe state detection algorithm was designed. Used the Media Pipe to design a face unsafe state detection algorithm, identified the state of open and closed eyes, open and closed mouth, and detected the frequency of blinking and yawning; used the Open Pose to design a posture unsafe state detection algorithm, identified abnormal movement of the head and back, and detected the head rotation angle-Roll, and the back curvature-Bend.

(3) The threshold of face detection and the range of posture detection were determined by fatigue-induced experiments. The experimental time was 24 consecutive hours, and it was equally divided into 12 times to test, which were sequentially counted as T₁, T₂, T₃, T₄, T₅, T₆, T₇, T₈, T₉, T₁₀, T₁₁, and T₁₂. Twenty male students were selected as subjects, and all subjects were tested for fatigue and the judgment of response accuracy every 2 hours. After the experiment, it was found that the subjects were in the non-fatigue level in the T₁, mild fatigue in the T₂, moderate fatigue in the T₄, and severe fatigue in the T₆. The face and posture data of T₁, T₂, T₄, and T₆ were measured respectively, and finally the face detection threshold and posture detection range of the unsafe state of I, II and III were obtained.

(4) A system fusion model was constructed to detect the unsafe state. One fusion model was to detect the features of eyes, mouth, head and back separately, and the other fusion model was to combine the features of the eyes and mouth into vector one, and the features of the head and back into vector two, both models were first trained by SVM and then D-S fusion. In the final training, the detection accuracy of model 1 was 85.23%, and the detection accuracy of model 2 was 93.65%. It can be seen that the detection effect of the combined feature model is better.

[1] Chen L, Qi H, Long R Y, et al. Research on 10-year tendency of China coal mine accidents and the characteristics of human factors[J]. Safety Science, 2012, 50(4): 745-750.

[2] 姚明亮, 祁神军, 成家磊, 等. 工作负荷与建筑工人不安全行为的结构关系研究[J]. 建筑经济, 2019, 40(1): 30-35.

[3] Mahdinia M, Mohammadfam I, Aliabadi M M, et al. The mediating effect of workers' situation awareness on the relationship between work-related factors and human error: a path analysis approach[J]. International journal of occupational safety and ergonomics : JOSE, 2021: 11-19.

[4] You X D , Zhu L , Liu Z G , et al. Experimental Study on the Relationship between Fatigue and Unsafe Behavior of Urban Rail Transit Drivers[J]. Transportation Research Record Journal of the Transportation Research Board, 2021, 2675(10): 1151-1160.

[5] 张孔. 基于多特征的驾驶员不安全行为检测的研究[D]. 天津: 天津大学, 2016.

[6] Tayibnapis I R , Koo D Y , Choi M K , et al. A novel driver fatigue monitoring using optical imaging of face on safe driving system[C]//International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC). IEEE, 2016.

[7] 田水承, 孙璐瑶, 唐艺璇, 等. 矿工不安全状态评价指标体系的构建与分析[J]. 西安科技大学学报, 2021, 41(3): 402-409.

[8] 李红霞, 薛建文, 杨妍. 疲劳对矿工不安全行为影响的ERP实验研究[J]. 西安科技大学学报, 2015, 35(3): 376-380.

[9] Wang C , Wang J , Wang X , et al. Exploring the impacts of factors contributing to unsafe behavior of coal miners[J]. Safety Science, 2019(115): 339-348.

[10] Chen Z , Qiao G , Zeng J . Study on the Relationship between Worker States and Unsafe Behaviours in Coal Mine Accidents Based on a Bayesian Networks Model[J]. Sustainability, 2019, 11(18): 5021-5021.

[11] 吴红玉. 基于生理测量的矿工不安全行为实验研究[D]. 焦作: 河南理工大学, 2016.

[12] 王武宏. 道路交通系统驾驶行为理论与方法[M]. 科学出版社, 2001.

[13] 陈建武, 毕春波, 廖海江, 等. 作业疲劳测量方法对比研究[J]. 中国安全生产科学技术, 2011, 7(5): 65-68.

[14] 王天芳, 薛晓琳. 疲劳自评量表[J]. 中华中医药杂志, 2009, 24(3): 348-349.

[15] 黄河, 耿东. 日本《自觉症状调查表》(2002版)在中国制造业工人的应用性研究[J]. 人类工效学, 2009, 15(3): 26-28.

[16] 吴绍斌, 高利, 王刘安. 基于脑电信号的驾驶疲劳检测研究[J]. 北京理工大学学报, 2009, 29(12): 1072-1075.

[17] 徐琳. 铁路坐位作业女工腰背痛流行病学调查[J]. 铁道劳动安全卫生与环保, 2002, 29(6): 287-287.

[18] 徐凯宏, 王述洋, 宋春明. 合理构建视频显示终端(VDT)作业疲劳工间休息制度[J]. 中国安全科学学报, 2009, 19(4): 26-31.

[19] 张连成, 张力为, 阳海英, 等. 运动性心理疲劳的测量与评价研究进展[J]. 中国运动医学杂志, 2012, 31(2): 180-185.

[20] Kecklund G , T Akerstedt. Sleepiness in long distance truck driving: an ambulatory EEG study of night driving[J]. Ergonomics, 1993, 36(9): 1007-1017.

[21] Katsis C D , Ntouvas N E , Bafas C G , et al. Assessment of Muscle Fatigue During Driving Using Surface EMG[J]. Biomedical engineering, 2004, 15(3): 266-271.

[22] Pinto J R, Cardoso J S, Lourenco A, et al. Towards a continuous biometric system based on ECG signals acquired on the steering wheel[J]. Sensors, 2017, 17(10): 2228-2228.

[23] 郭孜政, 刘仙, 陈瑞雅, 等. 基于ERP技术的疲劳对驾驶人行为监控能力影响研究[J]. 中国公路学报, 2018, 31(8): 173-180.

[24] 项天. 基于生理测量的建筑工人疲劳与不安全行为关系研究[D]. 北京: 清华大学, 2019.

[25] 徐明伟, 金龙哲, 张驎, 等. 基于心率监测的建筑工人生理疲劳分析[J]. 北京科技大学学报, 2018, 40(6): 760-765.

[26] Dinges D F , Grace R . PERCLOS: A Valid Psychophysiological Measure of Alertness as Assessed by Psychomotor Vigilance[J]. Tech Brief, 1998.

[27] Heitmann A , Guttkuhn R , Aguirre A , et al. Technologies for the Monitoring and Prevention of Driver Fatigue[C]//Driver Assessment, 2001.

[28] Ito T , Mita S , Kozuka K , et al. Driver blink measurement by the motion picture processing and its application to drowsiness detection[C]//The IEEE 5th International Conference on Intelligent Transportation Systems. IEEE, 2002: 168-173.

[29] Kithil P W , Jones R D , Mccuish J . Driver Alertness Detection Research Using Capacitive Sensor Array[C]//Driving Assessment: the First International Driving Symposium on Human Factors in Driver Assessment, 2001.

[30] 张利芳. 基于PCA算法的人脸识别系统研究[D]. 太原: 中北大学, 2015.

[31] 高修峰, 张培仁, 李子青. 人脸图像质量评估标准[J]. 小型微型计算机系统, 2009, 30(1): 95-99.

[32] 叶子龙. 人脸识别算法研究与嵌入式加速实现[D]. 广州: 华南理工大学, 2020.

[33] Li H , Gang H , Shen X , et al. Eigen-PEP for Video Face Recognition[C]//Asian Conference on Computer Vision. Springer, Cham, 2014.

[34] Ojala T , Pietikainen M , Maenpaa T . Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns[C]//Transactions on Pattern Analysis and Machine Intelligence. IEEE, 2002: 971-987.

[35] Belhumeur P N , Hespanha J P , Kriegman D J . Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 711-720.

[36] Senthilkumar R , Gnanamurthy R K . A Comparative Study of 2D PCA Face Recognition Method with Other Statistically Based Face Recognition Methods[J]. Journal of the Institution of Engineers, 2015, 97(3): 1-6.

[37] Chow T , Wang P , Ma E . A New Feature Selection Scheme Using a Data Distribution Factor for Unsupervised Nominal Data[J]. IEEE Transactions on Systems Man & Cybernetics Part B, 2008, 38(2): 499-509.

[38] 刘斌, 徐岩, 米强, 等. LBP特征和改进Fisher准则的人脸识别[J]. 计算机工程与应用, 2017, 53(16): 155-160.

[39] 孙文荣, 周先春, 嵇亚婷. 基于直方图均衡化、PCA和SVM算法的人脸识别[J]. 软件, 2014, 35(8): 11-15.

[40] 樊颖军. 局部二值模式分类器耦合特征脸的人脸识别算法[J]. 计算机与数字工程, 2016, 44(8): 1576-1580.

[41] Bengio, Yoshua, Courville, et al. Representation Learning: A Review and New Perspectives[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013, 35(8): 1798-1828.

[42] 余凯, 贾磊, 陈雨强, 等. 深度学习的昨天、今天和明天[J]. 计算机研究与发展, 2013, 50(9): 1799-1804.

[43] 祁星晨, 卓旭升. 基于改进MTCNN算法的低功耗边缘人脸检测跟踪系统[J]. 电子技术应用, 2021, 47(5): 40-44.

[44] Yang, Zhang, Peihua, et al. Face detection and alignment method for driver on highroad based on improved multi-task cascaded convolutional networks[J]. Multimedia tools and applications, 2019, 78(18): 26661–26679.

[45] 刘斌, 徐岩, 米强, 等. 融合Gabor和Gist特征的人脸识别[J]. 计算机工程与应用, 2017, 53(15): 217-221+238.

[46] Zhou S B , Jiang Z B , Xie X Z . Object Tracking Using Mean-Shift Based on Comentropy[J]. Computer Systems & Applications, 2010, 19(12): 49-53.

[47] 丁建. 人体姿态估计方法综述[J]. 福建质量管理, 2019(22): 285.

[48] Tsang E, Sun H. An efficient posture recognition method using fuzzy logic[J]. Virtual Reality, 1998, 3(2): 112-119.

[49] Bernard Boulay, Francois Brémond, Monique Thonnat. Applying 3D human model in a posture recognition system[J]. Pattern Recognition Letters, 2006, 27(15): 1788-1796.

[50] 许书平. 基于CNN和特征点分块的人脸检测与识别[D]. 西安: 西安电子科技大学, 2019.

[51] 仝泽友, 丁恩杰. 矿井皮带区矿工违规行为识别方法[J]. 河南科技大学学报:自然科学版, 2020, 41(2): 40-46+6-7.

[52] 佟瑞鹏, 张艳伟. 人工智能技术在矿工不安全行为识别中的融合应用[J]. 中国安全科学学报, 2019, 29(1): 7-12.

[53] 陈天, 闫雨寒, 徐达伟, 等. 基于改进双流算法的矿工行为识别方法研究[J]. 河南科技大学学报(自然科学版), 2021, 42(4): 47-53+7.

[54] 邓益侬, 罗健欣, 金凤林. 基于深度学习的人体姿态估计方法综述[J]. 计算机工程与应用, 2019, 55(19): 22-42.

[55] Wei S E, Ramakrishna V, Kanade T, et.al. Convolutional Pose Machines[C]//IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016.

[56] Cao Z, Simon T, Wei S E, et al. Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields[C]//IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017.

[57] Hossein R, Ajmal M, Mubarak S. Learning a Deep Model for Human Action Recognition from Novel Viewpoints[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3): 667-681.

[58] Abdelbaky A, Aly S. Two-stream spatiotemporal feature fusion for human action recognition[J]. The Visual Computer, 2021, 37(7): 1821-1835.

[59] Papandreou G , Zhu T , Kanazawa N , et al. Towards Accurate Multi-person Pose Estimation in the Wild[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017.

[60] Iqbal U , Milan A , Gall J . PoseTrack: Joint Multi-Person Pose Estimation and Tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017.

[61] Insafutdinov E , Andriluka M , Pishchulin L , et al. ArtTrack: Articulated Multi-Person Tracking in the Wild[C]//IEEE Conference on Computer Vision & Pattern Recognition, IEEE, 2017.

[62] 全泽林. 基于行为安全管理在煤矿安全生产管理中的应用研究[J]. 内蒙古煤炭经济, 2021(8): 131-132.

[63] 林泽炎, 石伟. 预防事故的行为干预技术及应用[J]. 中国安全科学学报, 1998, 8(2): 30-34.

[64] 何刚, 余保华, 朱艳娜, 等. 基于网络结构模型的矿工不安全行为影响因素研究[J]. 煤矿安全, 2017, 48(3): 227-229+233.

[65] 刘劲松. 安全生产管理严细实概论[M]. 白山出版社, 2007.

[66] 罗春红, 谢贤平. 事故致因理论的比较分析[J]. 中国安全生产科学技术, 2007, 3(5): 111-115.

[67] 付汇琪, 贾琳, 扈天保, 等. 危险源与事故原因、隐患之间关系的再讨论——基于国际标准和地勘事故案例的危险源含义分析[J]. 安全, 2021, 42(12): 14-19.

[68] 刘媛媛. 煤矿操作者状态对不安全行为的影响研究[D]. 太原: 太原科技大学, 2015.

[69] 杨小朋, 闫瑞兵, 彭莉莉. 大型煤炭企业事故致因模型及对策分析[J]. 中国安全科学学报, 2021, 31(S1): 73-79.

[70] 王莉, 刘致君, 李磊, 等. 注意力分配方式对煤矿安全监控行为影响的眼动研究[J]. 煤矿安全, 2022, 53(3): 243-248.

[71] 陈铁华, 刘景品, 李红霞, 等.矿长安全意识影响因素及影响路径研究[J]. 矿业安全与环保, 2022, 49(1): 109-113+120.

[72] 陈铁华, 刘景品, 李红霞. 新生代矿工安全认知对安全行为的影响[J]. 煤矿安全, 2021, 52(11): 256-260.

[73] 李祎琼. 心理疲劳的研究综述[J]. 黑河学刊, 2009(8): 35-36+43.

[74] 朱祖祥, 葛列众, 张智军. 工程心理学[M]. 人民教育出版社, 2000: 290-302.

[75] De Vries J, Michielsen H J, Van Heck G L. Assessment of fatigue among working people: comparison of six questionnaires[J]. Occupational & Environmental Medicine, 2003, 60(11): 110-115.

[76] Chalder T, Berelowitz G, Pawlikowska T, et al. Development of a Fatigue Scale[J]. Journal of Psychosomatic Research, 1993, 37(2): 147-153.

[77] Adam J J, Teeken J C, Ypelaar P J C, et al. Exercise-induced arousal and information processing[J]. International Journal of Sport Psychology, 1997, 28(3): 217-226.

[78] Cian C, Barraud P A, Melin B, et al. Effects of fluid ingestion on cognitive function after heat stress or exercise-induced dehydration[J]. International Journal of Psychophysiology, 2001, 42(3): 243-251.

[79] Robert G, Hockey J. Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework[J]. Biological Psychology, 1997, 45(1–3): 73-93.

[80] 高晓旭, 董丁稳, 杨日丽. BP神经网络在煤矿本质安全评价模型中的应用[J]. 西安科技大学学报, 2011, 31(6): 780-785.

[81] 曾宇航. 我国煤矿安全监控系统研究综述[J]. 软件导刊, 2010, 9(7): 140-142.

[82] 叶旭庆. 基于3D卷积神经网络的人体行为识别[D]. 西安: 西安电子科技大学, 2015.

[83] 温廷新, 王贵通, 孔祥博, 等. 基于迁移学习与残差网络的矿工不安全行为识别[J]. 中国安全科学学报, 2020, 30(3): 41-46.

[84] 卢伟, 胡海阳, 王家鹏, 等. 基于卷积神经网络面部图像识别的拖拉机驾驶员疲劳检测[J]. 农业工程学报, 2018, 34(7): 192-199.

[85] Omidyeganeh M , Shirmohammadi S , Abtahi S , et al. Yawning Detection Using Embedded Smart Cameras[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(3): 1-13.

[86] Qiao Y, Kai Z, Xu L, et al. A smartphone-based driver fatigue detection using fusion of multiple real-time facial features[C]//Consumer Communications & Networking Conference. IEEE, 2016.

[87] Mandal B , Li L , Gang S W , et al. Towards Detection of Bus Driver Fatigue Based on Robust Visual Analysis of Eye State[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(3): 1-13.

[88] 耿磊, 袁菲, 肖志涛, 等. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44(1): 274-279.

[89] 邬敏杰, 穆平安, 张彩艳. 基于眼睛和嘴巴状态的驾驶员疲劳检测算法[J]. 计算机应用与软件, 2013, 30(3): 25-27+54.

[90] 张敏飞. 基于视觉的人体行为识别研究与系统设计实现[D]. 杭州: 浙江工业大学, 2014.

[91] 王聪, 刘明光, 齐飞. 智能视频监控系统动态目标检测与识别算法综述[J]. 电气技术, 2018, 19(9): 6-11.

[92] 孙宁, 邹采荣, 赵力. 人脸检测综述[J]. 电路与系统学报, 2006, 11(6): 101-108+113.

[93] Bin F , Shuo X , Xiaofeng F . A Fatigue Driving Detection Method Based on Multi Facial Features Fusion[C]//11th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). 2019.

[94] Wenbo Z , Huicheng Y , Yi J , et al. A Method for Recognizing Fatigue Driving Based on Dempster-Shafer Theory and Fuzzy Neural Network[J]. Mathematical Problems in Engineering, 2017, 2(5): 1-10.

[95] 张军, 涂国平. 对D-S证据理论中冲突证据的一种新的处理方法[J]. 统计与决策, 2007, 2(13): 23-24.

[96] 由东媛. D-S证据理论研究及其在汽车火灾预警中的应用[D]. 青岛: 青岛科技大学, 2019.

[97] Hong X , Shuofu Y , Bin X , et al. Research on a Successively Increasing Feature Selection Algorithm of EEG Signal for Driving Fatigue Based on SVM[J]. Journal of Biomedical Engineering, 2013, 30(6): 1321-1326.

[98] 楚文慧, 吴超仲, 张晖, 等. 基于个性化行为模型的驾驶疲劳识别方法[J]. 中国安全科学学报, 2018, 28(6): 43-48.

[99] 贾海江, 刘志强, 汪彭, 等. 基于人-车特征的疲劳驾驶辨识方法研究[J]. 中国科技论文, 2016, 11(7): 751-753+759.

[100] 王海玉, 王映龙, 闵建亮, 等. 基于小波变换与多项指标的疲劳驾驶检测应用[J]. 计算机与现代化, 2018(10): 32-35+43.

[101] 王维. 基于视频报警终端检测驾驶员疲劳监测系统[J]. 中国新通信, 2019, 21(5): 162-162.

[102] 黎雪. 基于混合核函数的间歇过程统计建模和监测方法研究[D]. 沈阳: 东北大学, 2014.