步态作为一种识别人的复杂生物特征，在视频监控、疾病诊断以及人体行为分析中都有着广泛应用。但针对人在行走过程中衣着、携带物等一些干扰因素而导致步态识别率显著下降的问题，本文对步态识别开展研究，提出了一种加入了残差连接和注意力机制的图卷积网络与双路网络结合的步态识别方法，改善了在外观干扰因素下的步态识别率。

首先为了能够提高在衣着、携带物等一些遮挡物下提取关键点的精度，在姿态估计算法HRNet中加了CPN网络，通过利用CPN中含有GlobalNet和RefineNet两个网络的优势，对所提取的关键点特征进行了二次特征提取输出最终的骨架序列信息。通过可视化结果表明，加入CPN网络后，遮挡下的关键点提取有所改善。

其次，为了能够同时提取到步态骨架序列中的时间和空间特征，采用了图卷积网络，并针对图卷积网络在步态识别下的相关问题进行改进——加入了残差连接和注意力机制。通过加入残差连接可以对骨架数据的时空信息提取出更加精细的步态特征，扩大了感受野，提高了步态特征的表征能力，然后加入注意力机制对各个关节的重要程度进行建模，增大显著区域特征的权重，最后输出步态特征。实验表明，在CASIA-B数据集下，该网络在NM、BG、CL三种状态下的识别率分别达到了84.5%、76.2%和72.1%。

最后设计了一种双路网络结构，分别用来提取行人的动态和静态特征。其主要思想是：将人体骨架信息和改进的图卷积网络作为双路网络中的动态分支网络；静态分支网络则采用卷积神经网络，把所提取的关键点信息进行计算处理，得到关节角度和肢体长度一并作为静态分支的输入，之后采用通道注意力机制对双路网络拼接融合后的特征进行处理。最终获取到的双路网络的最优模型在NM、BG、CL三种状态下的识别率分别达到了93.5%、88.52%和87.2%，与改进后的图卷积网络相比有较大提升，尤其是在BG和CL状态下，分别提高了12.32%和15.1%，证明了双路网络在提高步态识别率方面的优越性，也说明了该算法的有效性和鲁棒性。

Gait, as a complex biometric feature, has broad applications in video surveillance, disease diagnosis, and human behavior analysis. However, the recognition accuracy of gait can be significantly reduced due to factors such as clothing and carried objects during walking. In this paper, a gait recognition method that combines residual connections and attention mechanisms with graph convolutional networks and dual-path networks is proposed to improve the gait recognition accuracy under appearance interference factors.

Firstly, in order to improve the accuracy of key point extraction under occlusions such as clothing and carried objects, the Cascaded Pyramid network(CPN) was added to the High-Resolution Net pose estimation algorithm. By leveraging the advantages of the GlobalNet and RefineNet networks in CPN, the features of the extracted key points were further extracted and the final skeleton sequence information was output. Visualization results showed that the addition of the CPN network improved the key point extraction under occlusions.

Next, in order to extract both temporal and spatial features from the gait skeleton sequence, a Graph Convolutional Network (GCN) was used, and improvements were made to address the issues of GCN in gait recognition by adding residual connections and attention mechanisms. The addition of residual connections allows for more refined extraction of temporal and spatial information from the skeleton data, expands the receptive field, and enhances the representation capability of gait features. Then, the attention mechanism is incorporated to model the importance of each joint, increasing the weight of significant region features, and finally outputting the gait features. Experimental results show that on the CASIA-B dataset, the recognition rates of the proposed network for three different states (NM, BG, CL) reached 84.5%, 76.2%, and 72.1% respectively.

Finally, a dual-path network structure was designed to extract both dynamic and static features of pedestrians. The main idea is to use the human skeleton information and the improved GCN as the dynamic branch network in the dual-path network. The static branch network uses a convolutional neural network (CNN) to process the extracted key point information and obtain joint angles and limb lengths, which are then used as input for the static branch. The channel attention mechanism is then used to process the fused features of the dual-path network. The optimal model of the dual-path network achieved recognition rates of 93.5%, 88.52%, and 87.2% for NM, BG, and CL states respectively, which is a significant improvement compared to the improved GCN, especially for BG and CL states with an increase of 12.32% and 15.1% respectively. This demonstrates the superiority of the dual-path network in improving gait recognition accuracy and validates the effectiveness and robustness of the proposed algorithm.

[1] 孙哲南,赫然,王亮,阚美娜,冯建江,郑方,郑伟诗,左旺孟,康文雄,邓伟洪,张杰,韩琥,山世光,王云龙,茹一伟,朱宇豪,刘云帆,何勇.生物特征识别学科发展报告[J].中国图象图形学报,2021,26(06):1254-1329.

[2] 刘琦,于汉超,蔡剑成,韩琥.大数据生物特征识别技术研究进展[J].科技导报,2021,39 (19):74-82.

[3] Wu Zifeng, Huang Yongzhen, Wang Liang, Wang Xiaogang, Tan Tieniu. A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs.[J]. IEEE transac- ions on pattern analysis and machine-intelligence,2017,39(2).

[4] Mao M, Song Y. Gait Recognition Based on 3D Skeleton Data and Graph Convolutional Network[C]// 2020 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 2020.

[5] Book A,Costello K, Camilleri J A. Psychopathy and victim selection: The use of gait as acue to vulnerability[J].Journal of interpersonal violence, 2013,28(11):2368-2383.

[6] Wolff C. A psychology of gesture[M]. Routledge, 2015.

[7] Venture Gentiane. Human characterization and emotion characterization from gait.[J]. Conference proceedings : Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference,2010,2010.

[8] 刘晓芳,周航,韩权等.基于视觉的步态识别研究综述[J].小型微型计算机系统,2018,39(08):1685-1692.

[9] 柴艳妹,夏天,韩文英,李洋.多特征融合的步态识别算法[J].小型微型计算机系统,2014,35(03):636-641.

[10] 陈玲,杨天奇.基于质心和轮廓关键点的步态识别[J].计算机工程与应用,2015,51(19): 173-177+188.

[11] Wolf T, Babaee, M Rigoll, G:Multi-view gait recognition using 3d convolutional neural networks[J],IEEE international conference on image processing ,2016,4165-4169.

[12] Takemura Noriko, Makihara Yasushi, Muramatsu Daigo, Echigo Tomio, Yagi Yasushi. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition[J]. IPSJ Transactions on Computer Vision and Applications,2018,10(1).

[13] Chao Hanqing, Wang Kun, He Yiwei, Zhang Junping, Feng Jianfeng. GaitSet: Cross-view Gait Recognition through Utilizing Gait as a Deep Set.[J]. IEEE transactions on pattern analysis and machine intelligence,2021,PP.

[14] Sepas-Moghaddam A, Ghorbani S, Troje N F, et al. Gait Recognition using Multi-Scale Partial Representation Transformation with Capsules[J]. 2020.

[15] Fan C,Peng Y,Cao C,et al. GaitPart:Temporal Part-Based Model for Gait Recognition [C]//CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020.

[16] 赵志杰, 孙小英, 金雪松, 孙华东, 卢鑫. 多重图像轮廓特征结合的步态识别算法[J]. 哈尔滨工业大学学报, 2016, 48(04): 182-188.

[17] 袁浩.基于多特征融合的人体步态识别算法[J].信息与电脑(理论版), 2018(17): 85-86.

[18] 李林杰,顾广华,王成儒等.能量图和典型相关分析相结合的特征融合步态识别[J].小型微型计算机系统,2014,35(11):2558-2561.

[19] 刘志勇,杨关,冯国灿.基于Gabor小波和局部二值模式的步态识别[J].中山大学学报(自然科学版),2014,53(04):1-7.

[20] 李凯,王国超.融合腿部局部特征的步态识别方法[J].计算机工程与设计,2016,37(05):1340-1345.

[21] A Yu SHI, Chen H, Wang Q, et al. Invariant feature extraction for gait recognition using only one uniform model[J]. Neuro computing,2017,239:81-93.

[22] 刘文婷,卢新明.基于LBP和HOG特征分层融合的步态识别[J].计算机工程与应用,2018,54(24):168-175.

[23] Yu S Q, Liao R J, An W Z, et al. Gait GANv2: Invariant gait feature extraction using gener- ative adversarial networks[J]. Pattern Recognition, 2019 (87):179-189.

[24] 叶汉民,黄培亮.基于下肢动静态特征融合的步态识别[J].科学技术与工程,2016,16(08):218-222.

[25] 王希娟.基于下肢轮廓角度距离和步幅变化的步态识别算法研究[J].计算机应用与软件,2018,35(02):235-241.

[26] 王浩,夏利民.基于堆叠深度卷积沙漏网络的步态识别[J].计算机工程与应用,2019,55(14):127-133.

[27] 周倩,孙运强,姚爱琴,鲁旭涛.基于人体关节点定位的步态识别技术研究[J].国外电子测量技术,2019,38(01):52-56.

[28] 邹倩颖,王小芳.粒子群优化BP神经网络在步态识别中的研究[J].实验技术与管理, 2019,36(08):130-133+138.

[29] Cao Zhe, Hidalgo Martinez Gines, Simon Tomas, Wei Shih-En, Sheikh Yaser A. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields.[J].IEEE transactions on pattern analysis and machine intelligence, 2019.

[30] Haoshu Fang, Shuqin Xie, Cewu Lu. RMPE: Regional Multi-person Pose Estimation.[J]. CoRR, 2016, abs/1612.00137.

[31] Ke Sun, Bin Xiao, Dong Liu, Jingdong Wang. Deep High-Resolution Representation Learning for Human Pose Estimation.[J].CoRR, 2019, abs/1902.09212.

[32] Liao R, Cao C, Garcia E B, et al. Pose-Based Temporal-Spatial Network (PTSN)for Gait Recognition with Carrying and Clothing Variations[J]. Proceedings of the 12th Chinese Conference on Biometric Recognition. Shenzhen, China. 2017. 474–483.

[33] 廖日军. 基于人体姿态特征的步态识别研究[D].深圳大学,2018.

[34] Li N, Zhao X, Ma C. A model-based Gait Recognition Method based on Gait Graph Conv- olutional Networks and Joints Relationship Pyramid Mapping[J]. 2020.

[35] Shaik, S.OpenPose based gait recognition using triplet loss architecture[J].PhD thesis,Dub- lin, National College of Ireland, (2020).

[36] Thien Huynh-The,Cam-Hao Hua,Nguyen Anh Tu,Dong-Seong Kim. Learning 3D spatiot- emporal gait feature by convolutional network for person identification[J].Neurocoputing, 2020,397(prepublish).

[37] 戚艳军,孔月萍,王佳婧,朱旭东.一种LSTM与CNN相结合的步态识别方法[J].西安电子科技大学学报,2021,1-9.

[38] Hasan Md Mahedi, Mustafa Hossen Asiful. Learning view-invariant features using stacked autoencoder for skeleton-based gait recognition[J].IET Computer Vision,2021,15(7).

[39] Yan S, Xiong Y, Lin D, Spatial Temporal Graph Convolutional Networks for Skeleton Based Action Recognition[C]//AAAI.2018.

[40] Tang Y, Tian Y, Lu J, et al. Deep progressive reinforcement learning for skeleton-based action recognition[C].IEEE Conference on Computer Vision and Pattern Recognition, 2018:5323-5332.

[41] Thakkar K, Narayanan P J. Part-based graph convolutional network for action recognition [J],2018, arXiv preprint arXiv:1809. 04983.

[42] Si C, Chen W, Wang W, et al. An attention enhanced graph convolutional lstm network f- or skeleton-based action recognition[C]. IEEE Conference on Computer Vision and Patte- rn Recognition,2019: 1227-1236.

[43] Song Y F, Zhang Z, Shan C, et al. Stronger, Faster and More Explainable: A Graph Convol- tional Baseline for Skeleton-based Action Recognition[J]. ACM, 2020.

[44] Mao M, Song Y. Gait Recognition Based on 3D Skeleton Data and Graph Convolutional Network[C]//2020 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 2020.

[45] 周雨.结合人体姿态和图卷积的步态识别算法研究[D].华中科技大学,2020.

[46] 张继凯,顾兰君.基于骨架信息的人体动作识别与实时交互技术[J].内蒙古科技大学学报,2020,39(03):266-272.

[47] Chen,Yilun,Zhicheng Wang,Yuxiang Peng,Zhiqiang Zhang,Gang Yu and Jian Sun.Cascad- ed Pyramid Network for Multi-person Pose Estimation[C].IEEE/CVF Conference on Co- mputer Vision and Pattern Recognition,2018: 7103-7112.

[48] S. E. Wei,V. Ramakrishna,T. Kanade,et al. Convolutional pose machines[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.LasVegas,2016,4724 -4732.

[49] 宋青松,张超,陈禹,王兴莉,杨小军.组合全卷积神经网络和条件随机场的道路分割[J].清华大学学报(自然科学版,2018,58版),2018,58(08):725731.

[50] 赵小强,宋昭漾.Adam优化的CNN超分辨率重建[J].计算机科学与探索,2019,13 (05):858-865.

[51] 吕淑平,黄毅,王莹莹.基于双流卷积神经网络的人体动作识别研究[J].实验技术与管理,2021,38(08):144-148.

[52] 孙若钒,高建瓴,陈娅先.VansNet轻量化卷积神经网络[J].贵州大学学报(自然科学版),2020,37(02):51-56.

[53] Ziyuan Zhang, Luan Tran, Xi Yin, Yousef Atoum, Jian Wan, Nanxin Wang, and Xiaoming Liu. Gait recognition via disentangled representation learning[C]// CPVR,2019.1,2,3,6,7.

[54] 申小凤,王春佳.基于ASPP的高分辨率卷积神经网络2D人体姿态估计研究[J].现代计算机,2020(13):61-65.

[55] WEI S H，RAMAKRISHNA V，KANADE T，et al.Convolutional pose machines［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）. NewYork：IEEE Press，2016：4724-4732. DOI：10.1109/CVPR.2016.511.

[56] CHEN Y，SHEN C H，WEI X S，et al. Adversarial PoseNet：A structure-aware convolutional network for human pose estimation［C］//2017 IEEE International Conference on Computer Vision （ICCV）. New York：IEEE Press，2017：1221-1230. DOI：10.1109/ICCV.2017.137.

[57] YANG W，LI S，OUYANG W L，et al. Learning feature Pyramids for human pose estimation［C］//2017 IEEE International Conference on Computer Vision（ICCV）. New York：IEEE Press，2017：1290-1299. DOI：10.1109/ICCV.2017.144.

[58] TANG W，YU P，WU Y. Deeply learned compositional models for human pose estimation ［C］// European Conference on Computer Vision. Cham：Springer，2018：197-214. DOI：10.1007/978-3-030-01219-9_12.

[59] 黄友,张娜,包晓安.基于改进级联金字塔网络的人体骨架提取算法[J].智能计算机与应用,2021,11(07):54-59.

[60] 李建更,谢海征.基于姿态估计的人体异常行为识别算法[J].北京工业大学学报, 2022,48(07):710-720.

[61] Wu Xinhui, An Weizhi, Yu Shiqi, Guo Weiyu, Garcia Edel B. Spatial-Temporal Graph Attention Network for Video-Based Gait Recognition[C]//Asian Conference on Pattern Recognition. Springer, Cham, 2019: 274-286.

[62] Hu J,Shen L,Sun G.Squeeze-and-excitation networks[C]//roceedings of the IEEE confere- nce on computer vision and pattern recognition.2018:7132-71