近年来为了解决我国石油资源分布不均的问题，国家大力发展输油管道运输业，与此同时输油管道的安全问题也日益严峻，其中人为破坏行为是影响输油管道安全运行的主要危害因素，其不仅会造成巨大的经济损失，而且极有可能引起石油泄漏，导致周边生态环境的严重破坏，甚至引发爆炸事故。因此，对输油管道运输场景中人员进行异常行为监测刻不容缓。论文使用深度学习中的卷积神经网络技术对人员异常行为的识别算法进行研究及改进，实现对输油管道运输场景的人员异常行为识别，辅助管理人员保障输油管道的运输安全。主要内容及创新包括：

1.由于缺乏对输油管道场景适用的公开数据集，为了解决模型训练数据的获取问题，自行拍摄了模拟实验场景中的行为视频，并从UCF101、Kinetics700、HDBM51等多个数据集中筛选出适用的行为视频，融合构建了面向输油管道场景的人员异常行为数据集。

2.论文搭建了5种基于深度学习的输油管道场景中的人员异常行为识别算法模型，即分别使用C3D、VGG、ResNet3D、DenseNet3D、Inception3D作为骨干网络提取视频数据特征，并设计对比实验研判各模型的性能。实验结果表明，Inception3D网络的学习能力较强，能更充分地从视频流中提取出人员异常行为的RGB图像特征。

3.本文提出了一种新型的双流卷积神经网络模型。借助空间信息网络算法只需简单的预处理就能够充分获取图像空间特征的特点，使用Inception3D网络提取视频流中异常行为的静态特征。同时借助时序信息网络能够提取目标连续帧间的运动信息的特点，使用ResNet3D网络对识别目标进行动态特征的提取。并采用迁移学习的训练方式分别对两路网络进行改进，最后对两路网络的识别结果进行平均融合，实现对具体行为的分类识别。

通过多组对比实验，证明本文所提出的网络模型能够较好地区分出多种异常行为，有效地提高了异常行为的识别能力，在构建的面向输油管道的人员异常行为数据集中识别准确率达到95.4%，通过两块NVIDIA RTX3090 GPU加速后，视频流的检测速度达到30FPS，满足视频监控实时识别要求，实现了输油管道场景的人员异常行为识别，可以有效辅助解决输油管道安全保障繁难的问题。

In recent years, in order to solve the problem of uneven distribution of oil resources in China, the country has been vigorously developing the oil pipeline transportation industry, while the safety of oil pipelines has become increasingly serious, in which vandalism is the main hazard affecting the safe operation of oil pipelines, which not only causes huge economic losses, but also has a high risk of causing oil leaks, leading to serious damage to the surrounding ecological environment, and even triggering explosive accidents. . Therefore, it is imperative to monitor the abnormal behaviour of people in the pipeline transportation scenario. The paper uses convolutional neural network technology in deep learning to research and improve the recognition algorithm of abnormal behaviour of personnel, so as to achieve the recognition of abnormal behaviour of personnel in oil pipeline transportation scenarios and assist management personnel in ensuring the safety of oil pipeline transportation. The main contents and innovations include:

Due to the lack of public datasets applicable to oil pipeline scenarios, in order to solve the problem of acquiring model training data, we filmed behavioural videos in simulated experimental scenarios by ourselves, and selected some applicable behavioural videos from several large behavioural datasets such as UCF101, Kinetics700, HDBM51, etc., and fused them to build a personnel abnormal behaviour dataset for oil pipeline scenarios .

The paper builds five deep learning-based algorithmic models for identifying anomalous behaviours of people in oil pipeline scenes, i.e. using C3D, VGG, ResNet3D, DenseNet3D and Inception3D as backbone networks to extract video data features respectively, and designs comparative experiments to judge the performance of each model. The experimental results show that the Inception3D network has better learning ability and can more adequately extract RGB image features from video streams with abnormal behaviours of people.

In this paper, a novel dual-stream convolutional neural network model is proposed. With the feature that the spatial information network algorithm can fully acquire the image spatial features with only simple pre-processing, the static features of abnormal behaviour in the video stream are extracted using the Inception3D network. At the same time, the ResNet3D network is used to extract dynamic features of the recognition target with the help of the temporal information network, which can extract motion information between consecutive frames of the target. And the training method of migration learning is used to improve the two networks respectively, and finally the recognition results of the two networks are averaged and fused to achieve the classification and recognition of specific behaviours.

Through multiple sets of comparison experiments, it is proved that the network model proposed in this paper can better distinguish a variety of abnormal behaviours and effectively improve the recognition ability of abnormal behaviours. The recognition accuracy in the constructed dataset of abnormal behaviours of personnel facing oil pipelines reaches 95.38%, and the detection speed of video streams reaches 30FPS after accelerated by two NVIDIA RTX3090 GPUs, which meets the real-time recognition requirements of video monitoring, realizes the recognition of abnormal behavior of personnel in oil pipeline scenarios, and can effectively assist in solving the difficult problem of oil pipeline safety assurance.

[1]梅苑,帅健,李云涛,刘敏.不同泄漏条件下输油管道泄漏事故后果研究[J].石油与天然气化工,2021,50(03):127-133.

[2]王起全.输油管道泄漏火灾爆炸事故演化及应急疏散分析[J].中国安全科学学报,2016,26(05):24-29.

[3]Yun Hongguang,Rayhana Rakiba,Pant Shashank. Nonlinear ultrasonic testing and data analytics for damage characterization: A review[J]. Measurement,2021,186:110155.

[4]Nader VahdatiGuiqing Xi, Caojun Huang, Shiqi Liu. A Multi-sensor Data Fusion Method for Nondestructive Testing of Oil Pipelines[J]. Instrumentation Mesure Métrologie,2019,18(3):243-248.

[5],Xueting Wang,Oleg Shiryayev. External Corrosion Detection of Oil Pipelines Using Fiber Optics[J]. Sensors,2020,20(3):684.

[6]温江涛,王涛,孙洁娣,付磊,李刚,杨文明.基于深度迁移学习的复杂环境下油气管道周界入侵事件识别[J].仪器仪表学报,2019,40(08):12-19.

[7]Jiran Sun, Qiang Wu, Xin Zheng. Oil Pipelines Detection Based on Unmanned Aerial Vehicle Pipeline Patrol Image[C]// Association for Computing Machinery, 2021,11:126–129.

[8]Kabaldin Yu. G.,Shatagin D. A.,Kiselev A. V. A.. Drone-Based Autonomous Robot Diagnostic System for Gas and Oil Pipelines in the Arctic and Far North[J]. Russian Engineering Research,2018,38(9):677-679.

[9]Morteza Zadkarami,Ali Akbar Safavi,Mohammad Taheri. Data driven leakage diagnosis for oil pipelines: An integrated approach of factor analysis and deep neural network classifier[J]. Transactions of the Institute of Measurement and Control,2020,42(14):2708-2718.

[10]Fernando Moreira Suyama,Myriam Regattieri Delgado,Ricardo Dutra da Silva. Deep neural networks based approach for welded joint detection of oil pipelines in radiographic images with Double Wall Double Image exposure[J]. NDT and E International,2019,105:46-55.

[11]Fei Lei,Kai Long,Xueli Wang. A Target Detection Method Based on Statistical Mean Value Difference Model[C]//Proceedings of 2019 3rd International Conference on Artificial Intelligence, Automation and Control Technologies.2019:6.

[12]曹闯明.油气长输管道巡检中的智能视频监控技术[J].油气储运,2018,37(10):1192-1195+1200.

[13]COLQUE R VHM, CAETANO C, DE ANDRADE M T L, et al. Histograms of optical flow orientation and magnitude and entropy to detect anomalous events in videos[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(3): 673-682.

[14]CHEN Y,YU Y, LI T. A vision based traffic accident detection method using extreme learning machine[C]//2016 International Conference on Advanced Robotics and Mechatronics. 2016: 567-572.

[15]CHENG K W, CHEN Y T, FANG W H. Gaussian process regression-based video anomaly detection and localization with hierarchical feature representation[J]. IEEE Transactions on Image Processing, 2015, 24(12): 5288-5301.

[16]ZHANG Y, LU H, ZHANG L, et al. Combining motion and appearance cues for anomaly detection[J]. Pattern Recognition, 2016, 51(6): 443-452.

[17]MOUSAVI H, MOHAMMANI S, PERINA A, er al. Analyzing tracklets for detection of abnormal crowd behavior[C]//2015 IEEE Winter Conference on Applications of Computer Vision. 2015: 148-155.

[18]李娟,张冰怡,冯志勇,等.基于隐马尔可夫模型的视频异常场景检测[J]. 计算机工程与科学, 2017, 39(7):1300- 1308.

[19]Kim J, Grauman K. A space-time MRF for detecting abnormal activities with incremental updates[C]//Proceedings of the Computer Vision and Pattern Recognition. 2013: 2921-2928.

[20]Karpathy A, Toderici G, Shetty S, et al. Large-Scale Video Classification with Convolutional Neural Networks[C]//Proceedings of the Computer Vision and Pattern Recognition. 2014: 1725-1732.

[21]Yue-Hei Ng J, Hausknecht M, Vijayanarasimhan S, et al. Beyond short snippets: Deep networks for video classification[C]//Proceedings of the Computer Vision and Pattern Recognition. 2015: 4694-4702.

[22]Venugopalan S, Xu H, Donahue J, et al. Translating videos to natural language using deep recurrent neural networks[J]. Computer Science, 2015, 2(21): 1494-1504.

[23]Krizhevsky A, Sutskever I, Hinton G E. Image Net classification with deep convolutional neural networks[C]//the Neural Information Processing Systems (NIPS'12). 2012: 1097-1106.

[24]Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. Computer Science, 2014, 3(26): 1-14.

[25]Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]//Proceedings of the Computer Vision and Pattern Recognition. 2015: 1-9.

[26]Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the IEEE international conference on computer vision. 2015: 4489-4497.

[27]Li Y, Miao Q, Tian K, et al. Large-scale gesture recognition with a fusion of rgb-d data based on the c3d model[C]//2016 23rd International Conference on Pattern Recognition (ICPR). 2016: 25-30.

[28]DUAN J, WAN J, ZHOU S, et al. A Unified Framework for Multi-Modal Isolated Gesture Recognition[J]. Acm Transactions on Multimedia Computing Communications & Applications, 2018, 14(1s): 1-16.

[29]Qiu Z, Yao T, Mei T. Learning spatio-temporal representation with pseudo-3d residual networks[C]//Proceedings of the IEEE international conference on computer vision. 2017: 5533-5541.

[30]Wang L, Ge L, Li R, et al. Three-stream CNNs for action recognition[J]. Pattern Recognition Letters, 2017, 92(C):33-40.

[31]莫宏伟，汪海波.基于Faster R-CNN的人体行为检测研究[J].智能系统学 报,2018,13(06):107-113.

[32]Bilen H, Fernando B, Gavves E, et al. Action Recognition with Dynamic Image Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(12):2799-2813.

[33]Balderas D, Ponce P, Molina A. Convolutional long short term memory deep neural networks for image sequence prediction[J]. Expert Systems with Application, 2019, 122(MAY):152-162.

[34]Shao Z, Li Y, Guo Y, et al. A Hierarchical Model for Human Action Recognition From Body-Parts[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(10):2986-3000.

[35]Gu J, Wang Z, Kuen J, et al. Recent Advances in Convolutional Neural Networks[J]. Pattern Recognition, 2018,77:354-377.

[36]Goodfellow I , Bengio Y , Courville A . Deep Learning[M]. Cambridge, MA: The MIT Press, 2016.

[37]Hyvrinen A , Kster U . Complex cell pooling and the statistics of natural images[J]. Network Computation in Neural Systems, 2005, 18(2):81-100.

[38]Bruna J , Szlam A , Lecun Y . Signal recovery from Pooling Representations[J]. Statistics, 2014:307-315.

[39]Soomro K, Zamir A R, Shah M. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild[J]. Computer Science, arXiv:1212.0242,2012.

[40]Dima D, Hazel D, Maria F G, et al. The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(11):4125-4141

[41]Carreira J, Zisserman A. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset[J]. IEEE, 2017:4724-4733.

[42]Tang Y, D Ding, Rao Y, et al. COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis[C]// IEEE, 2019:1207-1216.

[43]Zhao H, Torralba A, Torresani L, et al. HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization[C]//CVF International Conference on Computer Vision (ICCV). IEEE, 2019:8667-8667

[44]Ellis C, Masood S Z, Tappen M F, et al. Exploring the Trade-off Between Accuracy and Observational Latency in Action Recognition[J]. International Journal of Computer Vision, 2013, 101(3):420-436..

[45]Shahroudy A, Liu J, Ng T T, et al. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis[J]. IEEE Computer Society, 2016:1010-1019.

[46]Mueller M, Smith N, Ghanem B. A Benchmark and Simulator for UAV Tracking[C]// European Conference on Computer Vision . Springer International Publishing, 2016:445-461.

[47]Simonyan, Karen, Zisserman, Andrew. Very Deep Convolutional Networks for Large-Scale Image Recoenitionf[J]. Computer ence. 2014,09:1556.

[48]Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge[J]. International Journal of Computer Vision, 2015,115(3):211-252.

[49]Inhae Ha, Hongjo Kim, Somin Park, et al.Image retrieval using BIM and features from pretrained VGG network for indoor localization[J].Building and Environment,2018,140(140):23-31.

[50]S A Pearline, V S Kumar, S Harini.A study on plant recognition using conventional image processing and deep learning approaches[J]. Journal of Intelligent & Fuzzy Systems,2019,36(3):1997-2004.

[51]He. K, Zhang. X, S. Ren, et al. Deep Residual Learning for Image Recognition[J]. 2016 IEEE Conference on Computer Vision and Pattern Recognition , 2016.

[52]Li Ma,Renjun Shuai,Xuming Ran,Wenjia Liu,Chao Ye.Combining DC-GAN with ResNet for blood cell image classification[J].Medical & Biological Engineering & Computing,2020,58(8):1251-1264.

[53]Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2017:4700-4708.

[54]Dolz J, Ayed I B, Jing Y, et al. HyperDense-Net: A hyper-densely connected CNN for multi-modal image semantic segmentation[J] IEEE Transactions on Medical Imaging,2019,38(5):1116-1126.