微表情是一种当人们试图控制和隐藏自身强烈的情感时，快速呈现出的表情，可以准确反映人们内心想法的真实写照。基于这种性质，微表情研究在临床治疗、犯罪调查、商业谈判等领域都有着巨大的应用潜力。微表情识别属于细粒度视频分类的范畴，与一般的图像分类相比，视频分类更为复杂，还需要考虑时间信息等因素，再加上微表情的脸部肌肉运动幅度非常小且持续时间很短(0.04s-0.2s)，使得准确识别微表情变得非常困难。本文将微表情识别分为两个方面，结合上述问题做了如下研究：

在连续、快速微表情识别方面，例如商业谈判、课题教学中，微表情的出现通常间隔比较短，要求算法在确保一定准确率的前提下，还要有较低的识别耗时。所以本文提出了一种基于双流三维卷积神经网络的微表情识别算法。算法采用C3D作为主干网络，并将其第3、4、5层的双层卷积结构改为单层卷积结构，加快模型的推理速度；构建并行双流网络，同时传入微表情人脸图片和微表情光流图片，兼顾人脸空间属性和光流运动信息，在不改变网络深度的情况下获得了一定的识别精度提升；加入轻量级的3D时空注意力机制增强模型细节感知能力。在公开数据集SMIC和CASMEⅡ上进行算法性能评估，结果证明所提出的算法在保证准确率的同时识别速度提审升了约15%，可以有效解决快速微表情识别问题。

在更为严谨的、高精度的微表情识别方面，例如临床心理治疗、刑事审讯中，微表情识别有较为充裕的判断时间，不需要有很快的识别速度，但是识别错误的代价会比较大，要求算法有较高的识别精度。所以本文提出了一种基于多尺度三维残差卷积神经网络的微表情识别算法。算法采用3D-ResNet50作为主干网络，为了提高算法的特征提取能力，空间上，在网络中加入不同尺寸的多尺度卷积模块，整合全局信息和局部信息；时间上，将模型不同层数得到的特征图使用注意力特征模块进行融合，增强模型的上下文感知能力，再将融合结果分层输出，选择准确率最高的结果作为模型的输出。在公开数据集SMIC和CASMEⅡ上进行算法性能评估，均取得最高准确率，在高精度微表情识别方面可以作为重要参考依据。

开发设计微表情识别系统，该系统包括用户模块和微表情模块，用户模块主要有用户登录和用户数据管理两个功能，微表情模块主要有微表情识别和微表情数据管理两个功能，微表情识别功能集成上述两种微表情识别算法，面对不同场景，用户可以选择不同的微表情识别算法。

Micro-expression is a kind of expression that appears quickly when people try to control and hide their strong emotions, which can accurately reflect the true reflection of people's inner thoughts. Based on this nature, microexpression research has great potential in clinical treatment, crime investigation, business negotiation and other fields. Micro-expression recognition belongs to the category of fine-grained video classification. compared with general image classification, video classification is more complex, and factors such as time information need to be taken into account. in addition, the range of facial muscle movement of micro-expression is very small and the duration is very short (0.04s-0.2s), which makes it very difficult to identify micro-expression accurately. In this paper, microexpression recognition is divided into two aspects, combined with the above problems to do the following research:

In the aspect of continuous and fast micro-expression recognition, such as business negotiation and project teaching, the interval between the occurrence of micro-expression is usually relatively short, which requires the algorithm to have a lower recognition time on the premise of ensuring a certain accuracy. Therefore, a microexpression recognition algorithm based on double-flow 3D convolution neural network is proposed in this paper. In the algorithm, C3D is used as the backbone network, and the double-layer convolution structure of layer 3, 4 and 5 is changed into single-layer convolution structure to speed up the reasoning speed of the model. At the same time, micro-expression facial images and micro-expression optical flow images are introduced, and the recognition accuracy is improved without changing the depth of the network. A lightweight 3D spatio-temporal attention mechanism is added to enhance the detail perception of the model. The performance of the algorithm is evaluated on the open data sets SMIC and CASME Ⅱ, and the results show that the proposed algorithm improves the recognition speed by about 15% while ensuring the accuracy, which can effectively solve the problem of fast micro-expression recognition.

In the aspect of more rigorous and high-precision micro-expression recognition, such as clinical psychotherapy and criminal interrogation, micro-expression recognition has plenty of judgment time and does not need to have fast recognition speed, but the cost of recognition errors will be high. the algorithm is required to have high recognition accuracy. Therefore, a microexpression recognition algorithm based on multi-scale 3D residual convolution neural network is proposed in this paper. The algorithm uses 3D-ResNet50 as the backbone network. In order to improve the feature extraction ability of the algorithm, multi-scale convolution modules of different sizes are added to the network to integrate global information and local information. In terms of time, the feature images obtained from different layers of the model are fused with the attention feature module to enhance the context awareness of the model, and then output the fusion results layer by layer, and select the result with the highest accuracy as the output of the model. The performance of the algorithm is evaluated on the open data sets SMIC and CASME II, and the highest accuracy is achieved, which can be used as an important reference in high-precision microexpression recognition.

Develop and design a micro-expression recognition system. The system includes a user module and a micro-expression module. The user module mainly has two functions of user login and user management. The micro-expression module mainly has two functions of micro-expression recognition and micro-expression data management. Micro-expression recognition function Integrating the above two micro-expression recognition algorithms, users can choose the corresponding micro-expression recognition algorithm for different scenarios.

[1] Wang H. Facial expression decomposition[C]//Proceedings ninth IEEE international conference on computer vision. IEEE, 2003: 958-965.

[2] Peng M, Wu Z, Zhang Z, et al. From macro to micro expression recognition: Deep learning on small datasets using transfer learning[C]//2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 2018: 657-661.

[3] Liong S T, See J, Wong K S, et al. Automatic micro-expression recognition from long video using a single spotted apex[C]//Computer Vision–ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13. Springer International Publishing, 2017: 345-360.

[4] Ma H, An G, Wu S, et al. A region histogram of oriented optical flow (RHOOF) feature for apex frame spotting in micro-expression[C]//2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). IEEE, 2017: 281-286.

[5] Verburg M, Menkovski V. Micro-expression detection in long videos using optical flow and recurrent neural networks[C]//2019 14th IEEE International conference on automatic face & gesture recognition (FG 2019). IEEE, 2019: 1-6.

[6] Shen X, Wu Q, Fu X. Effects of the duration of expressions on the recognition of micro expressions[J]. Journal of Zhejiang University Science B, 2012, 13: 221-230.

[7] Porter S, Ten Brinke L. Reading between the lies: How do facial expressions reveal concealed and fabricated emotions?[J]. Psychological Science, 2008, 19(5): 508-514.

[8] RUSSELL T A,CHU E,PHILLIPSM L. A Pilot Study to Investigate the Effectiveness of Emotion Recognition Remediation in Schizophrenia Using the Micro-expression Training Tool[ J] . British Journal of Clinical Psychology, 2006,45(4) :579-583.

[9] Ekman P. Lie catching and microexpressions[J]. The philosophy of deception, 2009, 1(2): 5-5.

[10] 何景琳，梁正友，孙宇，等. 结合C3D与光流法的微表情自动识别［J］.计算机系统应用，2021，30（1）：221-227.

[11] 吴奇，申寻兵，傅小兰. 微表情研究及其应用[J]. 心理科学进展，2010，18（9）: 1359-1368.

[12] 魏文辉. 基于代表性 AU 区域的微表情识别算法研究 ［D］. 济南：山东大学，2021

[13] 张学森，贾静平．基于三维卷积神经网络和峰值帧光流的微表情识别算法[J].模式识别与人工智能，2021,34(5) : 423-433.

[14] Ekman P. MicroExpression Training Tool (METT). University of California, San Francisco[J]. 2002,8(2):145-161

[15] Frank M G, Maccario C J, Govindaraju V. Protecting airline passengers in the age of terrorism[J]. ABC-CLIO, Santa Barbara, 2009,45(22):19-25

[16] 周伟航,肖正清,钱育蓉,等.微表情自动分析方法研究综述[J]. 计算机应用研究，2022， 39(07):1921-1932

[17] Li X, Hong X, Moilanen A, et al. Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods[J]. IEEE transactions on affective computing, 2017, 9(4): 563-577.

[18] Polikovsky S, Kameda Y, Ohta Y. Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor[J]. 2009,14(6):12-16

[19] Li X, Pfister T, Huang X. A spontaneous micro-expression database: Inducement, collection and baseline[C]. In Proceedings of the 10th IEEE International Conference Automatic Face and Gesture Recognition. 2013: 1-6.

[20] 许诗琪. 基于深度学习的微表情识别算法研究[D].北方工业大学,2022.

[21] Zeng Z, Pantic M, Roisman G I, et al. A survey of affect recognition methods: audio, visual and spontaneous expressions[C]//Proceedings of the 9th international conference on Multimodal interfaces. 2007: 126-133.

[22] Fasel B, Luettin J. Automatic facial expression analysis: a survey[J]. Pattern recognition, 2003, 36(1): 259-275.

[23] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on pattern analysis and machine intelligence, 2002, 24(7): 971-987.

[24] Zhao G, Pietikainen M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE transactions on pattern analysis and machine intelligence, 2007, 29(6): 915-928.

[25] Li X, Pfister T, Huang X, et al. A spontaneous micro-expression database: Inducement, collection and baseline[C]//2013 10th IEEE International Conference and Workshops on Automatic face and gesture recognition (fg). IEEE, 2013: 1-6.

[26] Guo Y, Tian Y, Gao X, et al. Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method[C]//2014 international joint conference on neural networks (IJCNN). IEEE, 2014: 3473-3479.

[27] Le Ngo A C, Phan R C W, See J. Spontaneous subtle expression recognition: Imbalanced databases and solutions[C]//Computer Vision-ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Revised Selected Papers, Part IV 12. Springer International Publishing, 2015: 33-48.

[28] Yan W J, Wu Q, Liu Y J, et al. CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces[C]//2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, 2013: 1-7.

[29] Davison A K, Lansley C, Costen N, et al. Samm: A spontaneous micro-facial movement dataset[J]. IEEE transactions on affective computing, 2016, 9(1): 116-129.

[30] Wang Y, See J, Phan R C W, et al. Lbp with six intersection points: Reducing redundant information in lbp-top for micro-expression recognition[C]//Computer Vision–ACCV 2014: 12th Asian Conference on Computer Vision, Singapore, Singapore, November 1-5, 2014, Revised Selected Papers, Part I 12. Springer International Publishing, 2015: 525-537.

[31] Wang Y, See J, Phan R C W, et al. Efficient spatio-temporal local binary patterns for spontaneous facial micro-expression recognition[J]. PloS one, 2015, 10(5): 124-134.

[32] 马浩原, 安高云, 阮秋琦. 平均光流方向直方图描述的微表情识别[J]. 信号处理, 2018, 34(3): 279-288

[33] Liu Y J, Zhang J K, Yan W J, et al. A main directional mean optical flow feature for spontaneous micro-expression recognition[J]. IEEE Transactions on Affective Computing, 2015, 7(4): 299-310.

[34] Liong S T, Phan R C W, See J, et al. Optical strain based recognition of subtle emotions[C]//2014 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS). IEEE, 2014: 180-184.

[35] Pfister T, Li X, Zhao G, et al. Recognising spontaneous facial micro-expressions[C]//2011 international conference on computer vision. IEEE, 2011: 1449-1456.

[36] Cortes, C., and V apnik, V. Support-vector networks[J]. Mach. Learn. 1995:20, 273– 297.

[37] Khor H Q, See J, Phan R, et al. Enriched long-term recurrent convolutional network for facial micro-expression recognition [C] IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, 2018: 667-674.

[38] Peng M, Wang C Y, Bi T, et al. A novel apex-time network for cross-dataset micro-expression recognition[C] //Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction. Los Alamitos: IEEE Computer Society Press, 2019: 1-6

[39] Peng M, Wang C, Chen T, et al. Dual temporal scale convolutional neural network for micro-expression recognition[J]. Frontiers in psychology, 2017, 8: 1745-1750.

[40] Reddy S P T, Karri S T, Dubey S R, et al. Spontaneous facial micro-expression recognition using 3D spatiotemporal convolutional neural networks[C]//2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019: 1-8.

[41] GAN Y S, LIONG S T, YAU W C, et al. OFF-ApexNet on micro-expression recognition system[J]. Signal Processing: Image Communication, 2019, 74: 129-139.

[42] Zheng S, Guo J, Cui X, et al. Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection[J]. IEEE transactions on medical imaging, 2019, 39(3): 797-805.

[43] Carreira J，Zisserman A. Quo vadis，action recognition？ a new model and the kinetics dataset［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：6299-6308.

[44] 李星燃,张立言,姚树婧.结合特征融合和注意力机制的微表情识别方法[J].计算机科 学,2022,49(02):4-11

[45] 林宇凌, 金晓宏, 王中任. 基于LK光流法的微流控芯片中流体速度检测[J]. 激光与 红外, 2020(008):50-63.

[46] 刘子琦. 基于计算机视觉的高铁桥梁结构位移测量方法研究[D].哈尔滨工业大 学,2020.

[47] Asthana A, Zafeiriou S, Cheng S, et al. Robust discriminative response map fitting with constrained local models[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2013: 3444-3451.

[48] Wu H Y, Rubinstein M, Shih E, et al. Eulerian video magnification for revealing subtle changes in the world[J]. ACM transactions on graphics (TOG), 2012, 31(4): 1-8.

[49] LIONG S T, GAN Y, SEE J, et al. Shallow Triple Stream Three-dimensional Cnn (ststnet) for Micro-expression Recognition[C]. 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), 2019, 1-5.

[50] MILBORROW S, NICOLLS F. Active Shape Models with SIFT Descriptors and MARS[C]. 2014 International Conference on Computer Vision Theory and Applications (VISAPP), 2014, 2: 380-387.

[51] Zhou Z, Zhao G, Pietikäinen M. Towards a practical lipreading system[C]//CVPR 2011. IEEE, 2011: 137-144.

[52] LIONG S T, GAN Y S, ZHENG Danna, et al. Evaluation of the spatio-temporal features and GAN for micro-expression recognition system[J]. Journal of Signal Processing Systems, 2020, 92(7): 705-725.

[53] Feichtenhofer C, Pinz A, Zisserman A. Convolutional two-stream network fusion for video action recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 1933-1941.

[54] Fasel B, Luettin J. Automatic facial expression analysis: a survey[J]. Pattern recognition, 2003, 36(1): 259-275.

[55] 朱威，屈景怡，吴仁彪.结合批归一化的直通卷积神经网络图像分类算法[J].计算机 辅助设计与图形学学报,2017，29(9)：1650-1657

[56] IOFFE S,SZEGEDY C. Batch normalization: Accelerating deep network training by reducting internal covariate shift[C] //Published as a conference paper at ICLR，2015:1-11.

[57] 刘建伟，赵会丹，罗雄麟.深度学习批归一化及其相关算法研究进展[J].自动化学 报，2020，46(6)：1090-1120.

[58] Li X, Pfister T, Huang X, et al. A spontaneous micro-expression database: Inducement, collection and baseline[C]//2013 10th IEEE International Conference and Workshops on Automatic face and gesture recognition (fg). IEEE, 2013: 1-6.

[59] Yan W J, Li X, Wang S J, et al. CASME II: An improved spontaneous micro-expression database and the baseline evaluation[J]. PloS one, 2014, 9(1): e86041.

[60] [60]Patel D,Hong XP, Zhao GY. Selective deep features for micro-expression recognition. Proceedings of 2016 23rd International Conference on Pattern Recognition[J].Cancun, Mexico. 2016. 2258–2263