随着煤矿开采变的更加代化和智能化，人们开始关注人为因素对煤矿安全事故造成的影响。为了保障井下工作人员在工作时拥有积极的工作精神状态，避免人为因素引起煤矿安全事故。提出了基于脑电信号的情绪状态识别和疲劳状态识别方法，从情绪状态和疲劳状态两方面对矿工做出主观指标和客观指标的精神状态评估。具体内容包括：

(1) 针对情绪状态识别中单特征缺乏其它信息的问题，提出了一种具有时域和空间域的时空熵特征。将每一时刻的熵拼接成熵序列，配合拥有时序处理能力的双向LSTM神经网络模型进行训练和识别。基于时空熵的特征提取不仅提取了更加有效的信息特征，而且减少了模型训练时间。通过实验对比了时空近似熵和时空样本熵，测得基于时空近似熵的Bi-LSTM情绪三分类中效价和唤醒度的准确率达到了83.3%和82.1%，对情绪四分类的准确率达到了85.2%。结果表明时空近似熵作为特征拥有更好的准确率。

(2) 针对疲劳状态识别中识别率低的问题，提出了脑功能连接与功率谱联合识别疲劳状态的方法。具体表现为，将脑功能连接应用于多通道功率谱的权值归一化环节，实现不同区域的功率谱按照区域关系进行调整重要性分量。然后对比了十种功率谱判断指标，选出效果最好的指标作为疲劳状态的研究指标。对疲劳状态的平均识别准确率达到82.1%，对正常状态的平均识别准确率达到83.2%。相比于DCPM和脑连接疲劳识别方法，该方法结合了两种方法论的优点，表现出了更好的效果和更高的准确率。

(3) 针对情绪状态与疲劳状态两个任务的决策融合问题，提出了基于田口正交法的决策融合。根据多参数因素与精神状态指标的相关影响程度关系，将田口正交法用于试验因素与试验指标关系的趋势预测，实现情绪状态和疲劳状态的融合。最后，依据田口正交法的参数模型提出了主观和客观精神指标的双维度精神状态评估方法。

实验结果表明，本课题所提出的基于时空熵特征的情绪和疲劳状态融合的精神状态评估方法，能够完成对矿工的情绪状态和疲劳状态的识别任务，基于主观指标和客观指标的方法用于矿工精神状态评估是可行的。对矿工的井下工作和安全生产有一定的参考价值。

As coal mining becomes more modern and intelligent, people begin to pay attention to the impact of human factors on coal mine safety accidents. In order to ensure that underground workers have a positive mental state at work and avoid coal mine safety accidents caused by human factors. An emotional state recognition and fatigue state recognition method based on EEG signals is proposed to evaluate the mental state of miners using both subjective and objective indicators. The specific contents include:

(1) Aiming at the problem that a single feature lacks other information in emotional state recognition, a spatio-temporal entropy feature with a time domain and a spatial domain is proposed. The entropy of each moment is spliced into an entropy sequence, and the bidirectional LSTM neural network model with time series processing ability is trained and identified. Feature extraction based on spatio-temporal entropy not only extracts more effective information features but also reduces model training time. The spatio-temporal approximate entropy and the spatio-temporal sample entropy are compared by experiments. The accuracy of valence and arousal in Bi-LSTM emotion three classification based on spatio-temporal approximate entropy is 83.3% and 82.1%, and the accuracy of emotion four classification is 85.2%. The results show that the spatio-temporal approximate entropy has better accuracy as a feature.

(2) Aiming at the problem of low recognition rates in fatigue state recognition, a method combining brain functional connectivity and the power spectrum is proposed to identify fatigue state. Specifically, the brain functional connection is applied to the weight normalization link of the multi-channel power spectrum, so that the power spectrum of different regions can be adjusted according to the regional relationship. At the same time, ten power spectrum judgment indexes are compared, and the best index is selected as the research index of fatigue state. The average recognition accuracy of a fatigued state is 82.1%, and the average recognition accuracy of a normal state is 83.2%. Compared with the DCPM and the brain connection fatigue recognition methods, this method combines the advantages of the two methodologies, showing better results and higher accuracy.

(3) Aiming at the problem of decision fusion between emotional state and fatigue state, a decision fusion based on the Taguchi orthogonal method is proposed. According to the relationship between multi-parameter factors and mental state indicators, the Taguchi orthogonal method is used to predict the trend of the relationship between experimental factors and experimental indicators, so as to realize the fusion of emotional state and fatigue state. Finally, according to the parameter model of the Taguchi orthogonal method, a two-dimensional mental state evaluation method of a subjective mental index and an objective mental index is proposed.

The experimental results show that the proposed method of fusion of emotional and fatigue states for mental state evaluation can complete the task of identifying the emotional state and fatigue state of miners. It is feasible to evaluate the mental state of miners based on subjective and objective indicators. It has a certain reference value for the underground work and safety production of miners.

[1] 王军胜. 新形势下煤矿安全常态化管理的思考[J]. 中国煤炭工业, 2022(04): 58-59.

[2] 孙云川, 刘雄. 2021年全国煤矿事故特征及规律分析[J]. 采矿技术, 2022, 22(06): 78-81.

[3] 孟亦月, 李敬兆, 张梅. 基于 LSTM 边缘计算与随机森林雾决策的矿工状态监测设备[J]. 煤矿机械, 2018, 39(11): 144-148.

[4] Noroozi F, Corneanu C A, Kamińska D, et al. Survey on emotional body gesture recognition[J]. IEEE Transactions on Affective Computing, 2018, 12(2): 505-523.

[5] Tzirakis P, Trigeorgis G, Nicolaou M A, et al. End-to-end multimodal emotion recognition using deep neural networks[J]. IEEE Journal of Selected Topics in Signal Processing, 2017, 11(8): 1301-1309.

[6] 王露, 唐韬, 卿粼波, 等. 基于公共空间视频的人脸情绪识别[J]. 安徽工业大学学报 (自然科学版), 2019, 36(01):68-73+79.

[7] Yang D, Alsadoon A, Prasad P W C, et al. An emotion recognition model based on facial recognition in virtual learning environment[J]. Procedia Computer Science, 2018, 125:2-10.

[8] Yang H, Han J, Min K. A multi-column CNN model for emotion recognition from EEG signals[J]. Sensors, 2019, 19(21): 4736.

[9] Dar M N, Akram M U, Khawaja S G, et al. CNN and LSTM-based emotion charting using physiological signals[J]. Sensors, 2020, 20(16): 4551.

[10] 邱金平. 基于GoogLeNet的脑电情绪识别研究[D]. 长春. 长春大学, 2021.

[11] Liu Y, Ding Y, Li C, et al. Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network[J]. Computers in Biology and Medicine, 2020, 123: 103927.

[12] 张克军, 韩娜, 陈欣然, 等. 基于脑电图的新型情绪识别特征提取方法[J]. 华南师范大学学报: 自然科学版, 2019, 51(5): 6-11.

[13] 卢官明, 丛文康, 魏金生, 等. 基于 CNN 和 LSTM 的脑电信号情感识别[J]. 南京邮电大学学报 (自然科学版), 2021.

[14] Yin Y, Zheng X, Hu B, et al. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM[J]. Applied Soft Computing, 2021, 100: 106954.

[15] Iyer A, Das S S, Teotia R, et al. CNN and LSTM based ensemble learning for human emotion recognition using EEG recordings[J]. Multimedia Tools and Applications, 2023, 82(4): 4883-4896.

[16] Salvati L, d'Amore M, Fiorentino A, et al. On-road detection of driver fatigue and drowsiness during medium-distance journeys[J]. Entropy, 2021, 23(2): 135.

[17] Canadian Centre for Occupational Health and Safety, 2012. Fatigue fact sheet.

[18] Bauerle T, Dugdale Z, Poplin G. Mineworker fatigue: A review of what we know and future decisions[J]. Mining Engineering, 2018, 70(3): 33.

[19] Friedman L S, Almberg K S, Cohen R A. Injuries associated with long working hours among employees in the US mining industry: risk factors and adverse outcomes[J]. Occupational and Environmental Medicine, 2019, 76(6): 389-395.

[20] Chen S, Xu K, Zheng X, et al. Linear and nonlinear analyses of normal and fatigue heart rate variability signals for miners in high-altitude and cold areas[J]. Computer Methods and Programs in Biomedicine, 2020, 196: 105667.

[21] Chen S, Xu K, Yao X, et al. Psychophysiological data-driven multi-feature information fusion and recognition of miner fatigue in high-altitude and cold areas[J]. Computers in Biology and Medicine, 2021, 133: 104413.

[22] 田水承, 乌力吉, 寇猛, 范永斌. 基于生理实验的矿工不安全行为与疲劳关系研究[J]. 西安科技大学学报, 2016, 36(03): 324-330.

[23] 李诚信, 赵伟, 李千等. 疲劳监测技术在智慧矿山安全管理中的应用[J].煤炭科学技术, 2021, 49(S1): 131-137.

[24] 李红霞, 黄已芯, 田水承, 等. 小脑神经网络在矿工疲劳监测控制系统中的应用[J]. 西安科技大学学报, 2018, 3.

[25] 徐宝国, 彭思, 宋爱国. 基于运动想象脑电的上肢康复机器人[J]. 机器人, 2011, 33(3): 307-313.

[26] 聂双喜. 基于运动想象脑电信号的机械臂控制系统研究[D]. 成都. 电子科技大学, 2017.

[27] 王朴, 郭毅, 张君梅, 等. 运动想象疗法对脑卒中后患者上肢运动功能康复效果的系统评价[J]. 中国循证医学杂志, 2011, 11(05): 529-539.

[28] 钱一润, 王从庆, 张闻锐, 等. 基于SSVEP的空间机械臂脑机交互方法[J/OL]. 计算机测量与控制, 2022, 03(04): 1-8.

[29] 王萌. 面向手部感觉运动的侵入式脑机接口研究[D]. 上海. 上海交通大学, 2019.

[30] 陈兆权. 具有高阶曲率补偿的低压开关电容式带隙基准源研究与设计[D]. 广州. 华南理工大学, 2018.

[31] Bourdillon P, Hermann B, Guénot M, et al. Brain-scale cortico-cortical functional connectivity in the delta-theta band is a robust signature of conscious states: an intracranial and scalp EEG study[J]. Scientific Reports, 2020, 10(1): 14037.

[32] (美) 施塔, (美) 卡拉特. 情绪心理学: 中国轻工业出版社, 2015.03

[33] Ekman P. An argument for basic emotions[J]. Cognition & Emotion, 1992, 6(3-4): 169-200.

[34] Keltner D, Sauter D, Tracy J, et al. Emotional expression: Advances in basic emotion theory[J]. Journal of Nonverbal Behavior, 2019, 43: 133-160.

[35] 孙颖, 胡艳香, 张雪英, 段淑斐. 面向情感语音识别的情感维度 PAD 预测[J]. 浙江大学学报(工学版), 2019, 53(10): 2041-2048.

[36] Russell J A. A circumplex model of affect[J]. Journal of Personality and Social Psychology, 1980, 39(6): 1161.

[37] Ahn J, Gobron S, Silvestre Q, et al. Asymmetrical facial expressions based on an advanced interpretation of two-dimensional russell’s emotional model[J]. Proceedings of ENGAGE, 2010, 2.

[38] Koelstra S, Muhl C, Soleymani M, et al. Deap: A database for emotion analysis; using physiological signals[J]. IEEE Transactions on Affective Computing, 2011, 3(1): 18-31.

[39] Min Jianliang, Wang Ping, Hu Jianfeng., 2017. The original EEG data for driver fatigue detection. figshare. Dataset.

[40] 侯利君. 基于交叉验证的模型选择中投票和平均方法的对照[J]. 数学的实践与认识, 2019 (9): 188-192.

[41] 李锦瑶, 杜肖兵, 朱志亮, 等. 脑电情绪识别的深度学习研究综述[J]. 软件学报, 2023, 34(01): 255-276.

[42] Qiao W, Bi X. Ternary-task convolutional bidirectional neural turing machine for assessment of EEG-based cognitive workload[J]. Biomedical Signal Processing and Control, 2020, 57: 101745.

[43] 蔡冬丽, 钟清华, 朱永升, 等. 基于混合神经网络的脑电情感识别[J]. 华南师范大学学报(自然科学版), 2021, 53(01): 109-118.

[44] Cincotta P M, Giordano C M, Silva R A, et al. The Shannon entropy: an efficient indicator of dynamical stability[J]. Physica D: Nonlinear Phenomena, 2021, 417: 132816.

[45] 杨少康, 田泽金, 吕跃进. 基于区间值直觉模糊集新模糊度的模糊熵及其应用[J]. 广西大学学报 (自然科学版), 2018.

[46] Delgado-Bonal A, Marshak A. Approximate entropy and sample entropy: A comprehensive tutorial[J]. Entropy, 2019, 21(6): 541.

[47] Wang F, Wang H, Fu R. Real-time ECG-based detection of fatigue driving using sample entropy[J]. Entropy, 2018, 20(3): 196.

[48] 魏炘, 石强, 符文熹, 等. 考虑 CEEMDAN 样本熵和 SVR 的短期风速预测[J]. 水电能源科学, 2020, 38(11): 207-210.

[49] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.

[50] 何雯, 王凯. 脑卒中后上肢功能康复研究进展[J]. 中国康复理论与实践, 2014, 20(04): 334-339.

[51] 张林川. 精神分裂症幻听患者脑连接研究[D]. 西安. 第四军医大学, 2017.

[52] Tsai Y H, Yuan R, Patel D, et al. Altered structure and functional connection in patients with classical trigeminal neuralgia[J]. Human brain mapping, 2018, 39(2): 609-621.

[53] Okamoto M, Dan H, Sakamoto K, et al. Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10–20 system oriented for transcranial functional brain mapping[J]. Neuroimage, 2004, 21(1): 99-111.

[54] Koessler L, Maillard L, Benhadid A, et al. Automated cortical projection of EEG sensors: anatomical correlation via the international 10–10 system[J]. Neuroimage, 2009, 46(1): 64-72.

[55] 朱淮烽, 刘东俭, 张榕梁. 基于功率谱密度的用户车辆振动响应趋势分析[J]. 机电工程技术, 2022, 51(12): 95-98.

[56] 姚彦贵, 熊珍琴, 唐力晨, 等. 抗震分析中关于反应谱与功率谱密度函数之间关系的解析[J]. 核安全, 2022, 21(01): 93-103.

[57] 刘卫健, 黎淑娟, 黄际元, 等. 储能参与风电并网系统的波动平抑控制研究[J]. 电器与能效管理技术, 2018(16): 41-47.

[58] 薛奋勤, 李华, 刘丽娜, 等. 基于功率谱密度频谱分析的小鼠睡眠分期方法的研究[J]. 中国医疗器械信息, 2020, 26(17): 34-36.

[59] Secco A, Tonin A, Rana A, et al. EEG power spectral density in locked-in and completely locked-in state patients: a longitudinal study[J]. Cognitive Neurodynamics, 2021, 15(3): 473-480.

[60] Wang R, Wang J, Yu H, et al. Power spectral density and coherence analysis of Alzheimer’s EEG[J]. Cognitive Neurodynamics, 2015, 9(3): 291-304.

[61] 王鹏博, 何雷, 冯丽媛等. 基于EEG功率谱探究电针改善卒中相关失眠患者日间嗜睡的即刻及累积效应[J]. 针灸临床杂志, 2022, 38(07): 24-31.

[62] Iwasaki A. Deriving the variance of the discrete Fourier transform test using Parseval’s theorem[J]. IEEE Transactions on Information Theory, 2019, 66(2): 1164-1170.

[63] Demuru M, La Cava S M, Pani S M, et al. A comparison between power spectral density and network metrics: an EEG study[J]. Biomedical Signal Processing and Control, 2020, 57: 101760.

[64] Deng P Y, Qiu X Y, Tang Z, et al. Detecting fatigue status of pilots based on deep learning network using EEG signals[J]. IEEE Transactions on Cognitive and Developmental Systems, 2020, 13(3): 575-585.

[65] Subasi A. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients[J]. Expert Systems with Applications, 2005, 28(4): 701-711.

[66] Deng P Y, Qiu X Y, Tang Z, et al. Detecting fatigue status of pilots based on deep learning network using EEG signals[J]. IEEE Transactions on Cognitive and Developmental Systems, 2020, 13(3): 575-585.

[67] Xiao X, Xu M, Jin J, et al. Discriminative canonical pattern matching for single-trial classification of ERP components[J]. IEEE Transactions on Biomedical Engineering, 2019, 67 (8): 2266-2275.

[68] 许敏鹏, 程秀敏, 明东. 不同视觉注意状态调制稳态视觉诱发电位特征的可分性研究[J]. 生物医学工程学杂志, 2019, 36(05): 705-710.

[69] Lu Y, Wang M, Wu W, et al. Dynamic entropy-based pattern learning to identify emotions from EEG signals across individuals[J]. Measurement, 2019, 150: 107003.

[70] Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning[J]. Journal of Big Data, 2016, 3(1): 1-40.

[71] Apparao K C, Birru A K. Optimization of Die casting process based on Taguchi approach[J]. Materials Today: Proceedings, 2017, 4(2): 1852-1859.