对驾驶员视域内的车辆行为识别和轨迹预测能极大地降低重大交通事故发生的可能性，车辆的运动预测结果能够为驾驶员的行车决策和无人驾驶系统的决策规划模块提供重要的辅助信息，以保证车辆在各种复杂的交通场景下安全和高效地行驶。本课题针对主驾车辆前方车辆的行驶视频，以车辆目标检测、跟踪、行为识别和未来的行驶轨迹的预测问题展开研究，主要内容如下：

（1）对SSD目标检测算法进行优化，提出一种轻量级特征提取网络用来替换SSD的特征提取网络，简化其网络结构。筛选包含车辆图像的数据集，并重新设置各预测层候选框的尺寸、数量和长宽比例。针对车辆锚框尺寸在数据集中的分布情况，在不同尺度上的特征图上进行特征提取，并进行边框回归和分类预测，在提高检测精度的同时极大地加快了检测速度。在优化的SSD车辆目标检测网络的基础上，使用卡尔曼滤波算法对车辆的运动状态进行估计，再结合匈牙利匹配算法计算检测结果和轨迹之间的关联程度，实现车辆目标跟踪，并提取车辆的历史轨迹。

（2）提出一种基于残差网络和LSTM网络的混合模型的车辆行为识别算法，设计两个残差网络对车辆行驶图像序列进行特征提取，再利用LSTM网络对提取的车辆行为信息进行时间序列建模，最后使用softmax函数计算车辆行为分类得分，以解决传统车辆行为预测算法延迟高和精度低的缺点。

（3）提出基于双注意力网络的车辆轨迹预测方法，采用darknet-53提取当前时步的车辆交互特征，并与目标车辆的历史轨迹特征和行为特征进行融合。引入了注意力机制对融合的特征进行权重分配，使得网络能够自适应提取对车辆轨迹影响最大的特征，提高轨迹预测的精度。

（4）基于真实行车路况视频数据，对视频中的车辆实现目标检测、跟踪并提取轨迹数据，对车辆进行行为识别并预测车辆未来运动轨迹，以验证本文面向城市交通的驾驶员视域内车辆运动预测方法的有效性和可靠性。

For drivers and unmanned driving systems, predicting the future movement trends of surrounding traffic participants is an important means to ensure driving safety. In particular, the behavior recognition and trajectory prediction of the preceding vehicle can greatly reduce the possibility of serious traffic accidents.The prediction results of the movement of the surrounding vehicles can provide important information for the driver's driving decision and the decision-making planning module of the unmanned driving system to ensure the safe and efficient driving of the vehicle in various complex traffic scenarios. In this paper, we focuses on the driving video of the vehicle in front of the main driving vehicle, and conducts research on the problem of vehicle objection detection, tracking, behavior recognition and future driving trajectory prediction. The main contents are as follows:

(1) The SSD objection detection algorithm is optimized, and a lightweight feature extraction network is proposed to replace the SSD feature extraction network and simplify its network structure. Filter the data set containing the image of the vehicle objection, and reset the size, number and aspect ratio of each prediction layer candidate frame. According to the distribution of vehicles in the data set, feature extraction is performed on feature maps at different scales, and border regression and classification prediction are performed, which greatly accelerates the detection speed while improving the detection accuracy. On the basis of the optimized SSD vehicle objection detection network, the Kalman filter algorithm is used to estimate the motion state of the vehicle, and the Hungarian matching algorithm is used to calculate the correlation between the detection result and the trajectory to achieve vehicle objection tracking and extract the vehicle’s Historical trajectory.

(2) A vehicle behavior recognition algorithm based on a hybrid model of residual network and LSTM network is proposed. Two residual networks are designed to extract features of vehicle driving image sequences, and then LSTM network is used to model the extracted vehicle behavior information in time series. Finally, the softmax function is used to calculate the vehicle behavior classification score, which solves the shortcomings of high delay and low accuracy of traditional vehicle behavior prediction algorithms

(3) A vehicle trajectory prediction method based on dual attention network is proposed. Darknet-53 is used to extract the vehicle interaction features at the current time step, and to fuse with the historical trajectory features and behavior features of the target vehicle. The attention mechanism is introduced to assign weights to the fused features, so that the network can adaptively extract the features that have the greatest impact on the vehicle trajectory, and improve the accuracy of trajectory prediction.

(4) Based on the video data of real traffic conditions, the vehicle in the video is detected, tracked and extracted trajectory data, and the behavior of the vehicle is recognized and the future trajectory of the vehicle is predicted to verify the effectiveness and reliability of the vehicle motion prediction method in this paper.

[1] 中国经济网. 交管局：2019年上半年全国汽车保有量达2.5亿辆[EB/OL], http://baijiahao.baidu.com/s?id=1639463714799101791&wfr=spider&for=pc, 2019-07-19.

[2] 国家统计局. 中国统计年鉴(2018)[EB/OL], http://www.stats.gov.cn/tjsj/ndsj/2018/indexch.htm, 2019-02-04.

[3] Singh S. Critical reasons for crashes investigated in the national motor vehicle crash causation survey[R]. 2015.

[4] Ahmad K A, Mohd Noor M H, Hussain Z, et al. Improvement moving vehicle detection using RGB removal shadow segmentation[C]// In Proceedings of IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2015). Penang 2015:22-26.

[5]张润生,黄小云,刘晶,马雷,韩睿,赵玉勤,杨新红.基于视觉复杂环境下车辆行驶轨迹预测方法[J].机械工程学报,2011,47(02):16-24.

[6]詹盛,徐远新,石涌泉,王畅,张亚岐.基于模糊着色Petri网的车辆运动轨迹预测[J].计算机工程与应用,2014,50(03):227-231.

[7]夏卓群,胡珍珍,罗君鹏.EAVTP:一种环境自适应车辆轨迹预测方法[J].小型微型计算机系统,2016,37(10):2375-2379.

[8]熊光明,鲁浩,郭孔辉,陈慧岩.基于滑动参数实时估计的履带车辆运行轨迹预测方法研究[J].兵工学报,2017,38(03):600-607.

[9]张金旺,章永进,徐友春.基于概率统计的车辆运动轨迹预测方法[J].军事交通学院学报,2017,19(08):41-46.

[10]黄建根,陈祯福,裴晓飞,董兴智,张杰.基于D-S推理算法的智能车辆轨迹预测研究[J].汽车工程学报,2018,8(01):24-30.

[11]朱坤.基于高斯混合-贝叶斯模型的轨迹预测[J].计算机与现代化,2019(02):72-81.

[12]高建,毛莺池,李志涛.基于高斯混合-时间序列模型的轨迹预测[J].计算机应用,2019,39(08):2261-2270.

[13] Atev S, Miller G, Papanikolopoulos N P. Clustering of Vehicle Trajectories[J]. IEEE Transactions on Intelligent Transportation Systems, 2010, 11(3):647-657.

[14] Tay C, Mekhnacha K, Laugier C. Probabilistic Vehicle Motion Modeling and Risk Estimation[M]. Springer London, 2012.

[15] Aoude G S, Luders B D, Joseph J M, et al. Probabilistically safe motion planning to avoid dynamic obstacles with uncertain motion patterns[J]. Autonomous Robots, 2013, 35(1):51-76.

[16] Li J, Wu J, Sun H, et al. Traffic Modeling Considering Motion Uncertainties[R]. SAE Technical Paper, 2017.

[17] Koschi M, Althoff M. Interaction-aware occupancy prediction of road vehicles[C]//2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2017: 1-8.

[18] Agamennoni G, Nieto J I, Nebot E M. Estimation of Multivehicle Dynamics by Considering Contextual Information[J]. IEEE Transactions on Robotics, 2012, 28(4):855-870.

[19] Kim B D, Kang C M, Kim J, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network[C]//2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2017: 399-404.

[20] Gindele T, Brechtel S, Dillmann R. A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments[C]//13th International IEEE Conference on Intelligent Transportation Systems. IEEE, 2010: 1625-1631.

[21] Lefèvre S, Laugier C, Ibañez-Guzmán J. Evaluating risk at road intersections by detecting conflicting intentions[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012: 4841-4846.

[22] Schlechtriemen J, Wedel A, Hillenbrand J, et al. A lane change detection approach using feature ranking with maximized predictive power[C]//2014 IEEE Intelligent Vehicles Symposium Proceedings. IEEE, 2014: 108-114.

[23] Kumar P, Perrollaz M, Lefevre S, et al. Learning-based approach for online lane change intention prediction[C]//2013 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2013: 797-802.

[24] Kim I H, Bong J H, Park J, et al. Prediction of driver’s intention of lane change by augmenting sensor information using machine learning techniques[J]. Sensors, 2017, 17(6): 1350.

[25] Connor J T, Martin R D, Atlas L E. Recurrent neural networks and robust time series prediction[J]. IEEE Transactions on Neural Networks, 2002, 5(2):240-254.

[26] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural computation, 1997, 9(8):1735-80.

[27] Lee N, Choi W, Vernaza P, et al. Desire: Distant future prediction in dynamic scenes with interacting agents[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 336-345.

[28] Alahi A, Goel K, Ramanathan V, et al. Social lstm: Human trajectory prediction in crowded spaces[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 961-971.

[29] Gupta A, Johnson J, Fei-Fei L, et al. Social gan: Socially acceptable trajectories with generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 2255-2264.

[30] Deo N, Trivedi M M. Convolutional social pooling for vehicle trajectory prediction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018: 1468-1476.

[31]刘创,梁军.基于注意力机制的车辆运动轨迹预测[J].浙江大学学报(工学版),2020,54(06):1156-1163.

[32] Xiao H, Wang C, Li Z, et al. UB-LSTM: A Trajectory Prediction Method Combined with Vehicle Behavior Recognition[J]. Journal of Advanced Transportation, 2020, 2020:1-12.

[33] Uijlings J R R, K. E. A. Van De Sande. Selective search for object recognition [J]. International Journal of Computer Vision, 2013, 104(2):154-171.

[34] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, Washington, USA, 2014: 580-587.

[35] Girshick R. Fast R-CNN[C]// Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 2015, 1440-1448.

[36] Ren S, He K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6):1137-1149.

[37] Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.

[38] Redmon J , Farhadi A . YOLO9000: Better, Faster, Stronger[J]. IEEE, 2017:6517-6525.

[39] Redmon J, Farhadi A. Yolov3: An incremental improvement[J]. arXiv preprint arXiv:1804.02767, 2018.

[40] Liu W, Anguelov D, Erhan D , et al. SSD: Single Shot MultiBox Detector[J]. Springer, Cham, 2016.

[41] Neubeck A, Van Gool L. Efficient non-maximum suppression[C]//18th International Conference on Pattern Recognition (ICPR'06). IEEE, 2006, 3: 850-855.

[42] Rezatofighi H, Tsoi N, Gwak J Y, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 658-666.

[43] Schöning J, Faion P, Heidemann G. Interactive feature growing for accurate object detection in megapixel images[C]//European Conference on Computer Vision. Springer, Cham, 2016: 546-556.

[44] Henriques J F, Caseiro R, Martins P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3):583-596.

[45] Danielsson P E. Euclidean distance mapping[J]. Computer Graphics and image processing, 1980, 14(3): 227-248.

[46] De Maesschalck R, Jouan-Rimbaud D, Massart D L. The mahalanobis distance[J]. Chemometrics and intelligent laboratory systems, 2000, 50(1): 1-18.

[47] Liao H, Xu Z. Approaches to manage hesitant fuzzy linguistic information based on the cosine distance and similarity measures for HFLTSs and their application in qualitative decision making[J]. Expert Systems with Applications, 2015, 42(12): 5328-5336.

[48] Wang N, Yeung D Y. Learning a deep compact image representation for visual tracking[C]// International Conference on Neural Information Processing Systems. Curran Associates Inc. 2013:809-817.

[49] Simonyan K, Zisserman A. very deep convolutional networks for large-scale image recognition[J]. International conference on learning representations, San Diego, CA, United States, 2015: 2-6.

[50] Tzeng E, Hoffman J, Saenko K, et al. Adversarial discriminative domain adaptation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 7167-7176.

[51] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//International conference on machine learning. PMLR, 2015: 448-456.

[52] DeAngelis G C, Ohzawa I, Freeman R D. Receptive-field dynamics in the central visual pathways[J]. Trends in neurosciences, 1995, 18(10): 451-458.

[53] Kaiming H, Xiangyu Z, Shaoqing R, Jian S. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016:770-778.

[54] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. arXiv preprint arXiv:1706.03762, 2017.

[55] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. science. 2006, 313(5786):504-7.

[56] Yu D, Wang H, Chen P, et al. Mixed pooling for convolutional neural networks[C]// International conference on rough sets and knowledge technology. Shanghai, China, 2014:364-375.

[57] Liang J, Jiang L, Niebles J C, et al. Peeking into the future: Predicting future person activities and locations in videos[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 5725-5734.

[58] Vemula A, Muelling K, Oh J. Social attention: Modeling attention in human crowds[C]//2018 IEEE international Conference on Robotics and Automation (ICRA). IEEE, 2018: 4601-4607.

[59] Britz D, Goldie A, Luong M T, et al. Massive exploration of neural machine translation architectures[J]. arXiv preprint arXiv:1703.03906, 2017.

[60] 刘宏鼎,秦世引. 基于图像特征的运动目标识别与伺服跟踪[J]. 仪器仪表学报,2008, 29(3):644-648.