基于无人机倾斜摄影测量建立的实景三维模型不仅可以为实景三维城市建设提供精确的地理信息数据还能为其提供视觉效果良好的三维模型数据。无人机倾斜摄影三维重建作为一种速度快，精度高且费用低的建模技术，在各项建模技术中突出重围，成为大场景实景三维重建方法之首选。然而，在获取的场景影像中通常会存在行人，车辆等处于动态状态的物体（移动目标），由此造成在影像重叠范围内缺少必要的同名点，从而导致重建的模型几何变形以及纹理失真，极大地降低了重建实景三维模型的真实性。

基于以上背景，本文对此展开研究，内容如下：

（1）基于二维影像信息的移动物体遮挡处理。针对倾斜摄影三维建模中移动物体遮挡导致纹理错误的问题，采用均匀分布采样点的颜色信息来代表三维构网的每个三角面，分别在YCBCR，RGB，LAB颜色空间计算每个面之间的协方差矩阵、均值、方差、相关系数等统计量。通过对影像上遮挡情况的观察确定各个统计量的阈值，最后利用这些统计量对三角面的可视影像进行一致性检测，将包含移动物体的异常影像从可视影像列表中剔除以实现纹理遮挡剔除。实验结果表明，采用该方法剔除移动物体遮挡后的三维模型在视觉上较好地反映了场景的真实情况。

（2）基于深度学习及三维Mesh信息的移动目标检测。顾及无人机倾斜影像数据量大、视角变化大、移动目标小的特点，首先对广泛应用于目标检测的YOLO v4模型进行改进，增加注意力模块；其次对倾斜影像进行标注，结合公开数据集制作适合无人机倾斜影像移动目标检测的模型训练样本；最终基于三维Mesh模型相关信息实现移动目标的检测。实验结果表明：改进后的YOLO v4模型mAP提升近8%，移动车辆判定准确率达100%。

（3）基于移动目标检测结果的几何变形和纹理遮挡修复。以移动物体掩膜影像为基础，对移动目标在三维Mesh范围内的三角面做几何变形修复及纹理遮挡处理。实验结果表明，修复后的三维模型移动物体纹理遮挡剔除效果明显，模型整体纹理映射更接近于现实，与国外商业软件CC (ContextCapture)相比效果更佳。

The real-world 3D model based on UAV oblique photogrammetry can not only provide accurate geographic information data for real-life 3D city construction, but also provide 3D model data with good visual effects. As a fast, high-precision and low-cost modeling technology, UAV oblique photography 3D reconstruction has become the first choice for large-scene real-world 3D reconstruction methods. However, there are usually objects (moving targets) in a dynamic state such as pedestrians and vehicles in the acquired scene images, resulting in the lack of necessary points with the same name within the overlapping range of the images, resulting in geometric deformation and texture distortion of the reconstructed model, which greatly reduces the authenticity of the reconstructed real-world 3D model.

Based on the above background, this paper conducts research on this as follows:

(1) Moving object occlusion processing based on two-dimensional image information. Aiming at the problem of texture error caused by occlusion of moving objects in oblique photography 3D modeling, the color information of evenly distributed sampling points is used to represent each triangular surface of the three-dimensional network, and the covariance matrix, mean, variance, correlation coefficient and other statistics between each face are calculated in the YCBCR, RGB, and LAB color spaces respectively. The threshold of each statistic is determined by observing the occlusion on the image, and finally the consistency detection of the visual image of the triangle is performed with these statistics, and the anomalous image containing moving objects is excluded from the visual image list to achieve texture occlusion culling. Experimental results show that the 3D model after removing the occlusion of moving objects by this method visually reflects the real situation of the scene.

(2) Moving target detection based on deep learning and 3D mesh information. Considering the characteristics of large oblique image data, large change of viewing angle and small moving target of UAV, the YOLO v4 model, which is widely used in target detection, is first improved, and the attention module is added. Secondly, the oblique images are labeled, and the model training samples suitable for the detection of moving objects of the oblique image of drones are made in combination with the public dataset. Finally, the detection of moving targets is realized based on the relevant information of the 3D Mesh model. Experimental results show that the improved YOLO v4 model mAP is improved by nearly 8%, and the accuracy of moving vehicles is 100%.

(3) Geometric distortion and texture occlusion repair based on moving target detection results. Based on the mask image of the moving object, geometric deformation repair and texture occlusion are performed on the triangular surface of the moving target within the 3D Mesh range. Experimental results show that the texture occlusion and culling effect of moving objects in the restored 3D model is obvious, and the overall texture mapping of the model is closer to reality, which is better than that of foreign commercial software CC (ContextCapture).

[1] 杨争艳. 倾斜摄影测量三维重建中纹理映射的研究[D]. 成都理工大学, 2017.

[2] 芦彦霖. 倾斜摄影测量实景三维模型构建及精度分析[D]. 中国矿业大学, 2019.

[3] 王伟, 黄雯雯, 镇姣. 倾斜摄影技术及其在3维城市建模中的应用[J]. 测绘与空间地理信息, 2011, 34(3): 181-183.

[4] Thomas Hanusch. Texture Mapping and True Orthophoto Generation of 3D Objects[D]. ETHZURICH, 2010.

[5] Velho, L., Sossai Jr.,J. Projective texture atlas construction for 3D photography[J].The Visual Computer, 2007, 23:9-11.

[6] 张春森, 张卫龙, 郭丙轩, 刘健辰, 李明. 倾斜影像的三维纹理快速重建[J]. 测绘学报, 2015, 44(07): 782-790.

[7] 乃古色拉,张云生,张明磊,邹峥嵘.基于图割算法的倾斜影像纹理映射优化方法[J].测绘与空间地理信息,2019,42(03):145-147.

[8] 王晓坤.基于无人机倾斜影像的三维模型纹理映射方法研究[D].山东建筑大学,2019.

[9] Sinha S N, Steedly D, Szeliski R, et al. Interactive 3D architectural modeling from unordered photo collections[J]. ACM Transactions on Graphics (TOG), 2008, 27(5): 1-10.

[10] Frahm J M, Fite-Georgel P, Gallup D, et al. Building rome on a cloudless day[C]//European conference on computer vision. Springer, Berlin, Heidelberg, 2010: 368-381.

[11] Shan Q, Adams R, Curless B, et al. The visual turing test for scene reconstruction[C]//2013 International Conference on 3D Vision-3DV 2013. IEEE, 2013: 25-32.

[12] Tan, Y. , Kwoh, L. , Ong, S. : Large scale texture mapping of building facades[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 37 (2008) 2.

[13] Garcia-Dorado I, Demir I, Aliaga D G. Automatic urban modeling using volumetric reconstruction with surface graph cuts[J]. Computers & Graphics, 2013, 37(7): 896-910.

[14] Grammatikopoulos L, Kalisperakis I, Karras G, et al. Automatic multi-view texture mapping of 3D surface projections[C]//Proceedings of the 2nd ISPRS International Workshop 3D-ARCH. 2007: 1-6.

[15] Waechter M, Moehrle N, Goesele M. Let there be color! Large-scale texturing of 3D reconstructions[C]//European conference on computer vision. Springer, Cham, 2014: 836-850.

[16] 刘亚文, 关振. 街景建筑物立面纹理遮挡恢复方法研究[J]. 武汉大学学报信息科学版, 2010(12): 1457-1460.

[17] 王崇倡, 张秀岩, 纪亮. 基于倾斜摄影测量技术三维纹理遮挡处理[J]. 测绘与空间地理信息, 2018, 41(03): 61-64.

[18] 李妍妍. 基于倾斜摄影三维模型纹理遮挡研究[J]. 测绘与空间地理信息, 2019, 42(09): 178-180+ 185.

[19] 贾程栋. 面向城市智能汽车的场景多模真实感重建技术研究[D]. 电子科技大学, 2020.

[20] 张守权. 三维城市场景下高质量纹理映射的研究[D]. 东华理工大学, 2020.

[21] Redmon J, Divvala S, Girshick R. You only look once: unified, real-time object detection[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016: 779-788.

[22] Liu S, Huang D. Receptive field block net for accurate and fast object detection[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 385-400.

[23] Ge Baoyi, Zuo Xianzhang, Hu Yongjiang. Review of research on visual target tracking methods[J]. Journal of Image and Graphics of China, 2018, 23(8): 1091-1107.

[24] Girshick R, Donahue J, Darrell T, ea al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE: 2014: 580-587.

[25] He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(9): 1904-1916.

[26] Girshick R. Fast r-cnn[C]//Proceedings of the IEEE international conference on computer vision. 2015: 1440-1448.

[27] Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.

[28] Redmon J, Divvala S, Girshick R. You only look once: unified, real-time object detection[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016: 779-788.

[29] Redmon J, Farhadi A. YOLO9000: better, faster, stronger[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 6517-6525. DOI:10.1109/CVPR.2017.690.

[30] Redmon J, Farhadi A. Yolov3: An incremental improvement[J]. arXiv preprint arXiv:1804.02767, 2018.

[31] Bochkovskiy A, Wang C Y, Liao H Y M. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv preprint arXiv:2004.10934, 2020.

[32] Liu W, Anguelov D, Erhan D, et al. SSD: single shot MultiBox detector[C]// Proceedings of the 14th European Conference on Computer Vision. Amsterdam: Springer: 2016: 21-37. DOI:10.1007/978-3-319-46448-0_2.

[33] Rezaee M, Zhang Y, Mishra R, et al. Using a Vgg-16 Network for Individual Tree Species Detection with an Object-Based Approach[C]//The 10th IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS), 2018: 1-7.

[34] Rujikietgumjorn S, Watcharapinchai N. Vehicle detection with sub-class training using R-CNN for the UA-DETRAC benchmark[C]// IEEE International Conference on Advanced Video & Signal Based Surveillance. IEEE, 2017.

[35] 彭怀宇. 基于深度学习的车辆检测及车型识别研究[D], 2019.

[36] 曹诗雨, 刘跃虎, 李辛昭. 基于 Fast R-CNN 的车辆目标检测[J]. 中国图象图形学报, 2017, 22(05): 671-677.

[37] 赵锟. 基于深度卷积神经网络的智能车辆目标检测方法研究[D]. 国防科学技术大学, 2015.

[38] Kim H, Lee Y, Yim B, et al. On-road object detection using deep neural network[C]// IEEE International Conference on Consumer Electronics-asia. IEEE, 2016.

[39] 陈冰曲, 邓涛. 基于改进型 SSD 算法的目标车辆检测研究[J]. 重庆理工大学学报(自然科学版), 2019, 33(01): 64-69+135.

[40] Tang T, Deng Z, Zhou S, et al. Fast vehicle detection in UAV images[C]// 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP). IEEE, 2017.

[41] 王宇宁, 庞智恒, 袁德明. 基于 YOLO 算法的车辆实时检测[J]. 武汉理工大学学报, 2016, 38(010): 41-46.

[42] 裴嘉欣, 孙韶媛, 宇岚, 等. 基于改进 YOLOv3 网络的无人车夜间环境感知[J]. 应用光学, 2019, 40(003): 380-386.

[43] Hu X, Xu X, Xiao Y, et al. SINet: A Scale-Insensitive Convolutional Neural Network for Fast Vehicle Detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(3): 1010-1019.

[44] 周苏, 支雪磊, 林飞滨, 等. 基于车载视频图像的车辆检测与跟踪算法[J]. 同济大学学报(自然科学版), 2019(S1).

[45] 冯加明,储茂祥,杨永辉,巩荣芬.改进YOLOv3算法的车辆信息检测[J].重庆大学学报,2021,44(12):71-79.

[46] 姜康. 基于深度学习的车辆检测算法研究[D]. 吉林大学, 2020.

[47] 刘家朋. 光照不均匀条件下的三维重建关键技术研究[D]. 上海交通大学, 2018.

[48] 乔新新. 基于颜色与纹理特征聚类的彩色图像分割研究[D]. 桂林理工大学, 2019.

[49] 陆星家, 郭璘, 陈志荣,等. 基于外观和运动的车辆检测和追踪算法研究[J]. 计算机工程, 2014, 40(8):6.

[50] TAN Heng-liang, YANG Bing, MA Zheng-ming. Face recongnition based on the fusion of global and local HOG features of face images[J]. IET Computer Vision, 2014, 8(3): 224-234.

[51] Viola P , Jones M . Robust real-time face detection[C]// International Conference on Computer Vision. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2001.

[52] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6) : 1229－1251．

[53] Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks [J]. Advances in Neural Information Processing Systems, 2012, 25(2):2012.

[54] 郭泽方. 图像物体检测深度学习算法综述[J]. 机械工程与自动化, 2019(1) : 220－222, 224．

[55] 李正明, 章金龙. 基于深度学习的抓取目标姿态检测与定位[J]. 信息与控制, 2020, 49(2) : 147－153．

[56] 管军霖, 智鑫. 基于YOLOv4 卷积神经网络的口罩佩戴检测方法[J]. 现代信息科技, 2020, 4 (11): 9-12.

[57] Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes (voc) challenge[J]. International journal of computer vision, 2010, 88(2): 303-338.

[58] Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248-255.

[59] Lin T Y, Maire M, Belongie S, et al. Microsoft coco: Common objects in context[C]//European conference on computer vision. Springer, Cham, 2014: 740-755.

[60] Kuznetsova A, Rom H, Alldrin N, et al. The open images dataset v4[J]. International Journal of Computer Vision, 2020, 128(7): 1956-1981.

[61] Robicquet A, Sadeghian A, Alahi A, et al. Learning social etiquette: Human trajectory understanding in crowded scenes[C]//European conference on computer vision. Springer, Cham, 2016: 549-565.

[62] Zhu P, Wen L, Du D, et al. Detection and Tracking Meet Drones Challenge[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2021 (01): 1-1.

[63] FISCHLER M A, BOLLES R C. Random Sample Consensus: A Paradigm for Model Fitting with Applications To Image Analysis and Automated Cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.

[64] 张萌萌. 无人机影像信息驱动的物方Mesh模型重建与优化方法研究[D]. 西安科技大学, 2018.

[65] Zhen-yu, Guo-zhao, WANG, et al. Adaptive triangular mesh coarsening with centroidal Voronoi tessellations[J]. 浙江大学学报: a 卷英文版, 2009, 10(4): 535-545.

[66] 罗键. 基于 VCGLib 库的曲面重构算法研究[D]. 同济大学, 2014.