钛钉钉匣是现代医学外科手术吻合器的关键组成部分，其缺陷检测的精确度与效率对于确保患者生命健康以及减少医疗事故至关重要。本文深入分析钛钉钉匣表面缺陷检测中的技术难题，对比测试AnomalyGPT等先进异常检测算法的检测结果，发现此类大模型的方法在微小缺陷检测方面存在局限性，需要多次提示（Prompt）才能捕获微弱异常信号。本文提出了一种创新的缺陷检测算法，通过优化PatchCore模型自动生成有效的提示信息，输入给EdegSAM进行精确定位和分割微小的表面缺陷。

(1) 针对AnomalyGPT等基于大模型的异常算法需要多次尝试才可生成高质量的提示来分割缺陷的问题，本文提出了一种基于PatchCore的异常稀疏点阵生成算法。该算法能够动态生成针对不同缺陷的异常稀疏点阵作为EdegSAM的点“提示”，为缺陷的形状和边界分割提供强有力的支持。该算法受PatchCore启发，首先将输入图像分割成多个小矩形区域，然后通过专门的卷积神经网络提取特征，并通过聚合机制形成全局特征表示；接着，使用稀疏点阵生成算法处理这些特征，提取和形成异常稀疏点阵。

(2) 提出结合缺陷稀疏点阵和多种缺陷适配器微调EdgeSAM相结合的钛钉钉匣缺陷检测方法，通过缺陷稀疏点阵和缺陷适配器来提示EdgeSAM模型，该方法能精确分割出缺陷边界及其类别，既解决了AnomalyGPT等大模型推理实时性能差，无法在工业领域直接应用的限制，又改进了PatchCore对微小缺陷分割和分类的不足。在钛钉钉匣数据集上的实验结果显示，所提方法的MIoU达到了63.9%，相比U-Net、FCN和Deeplabv3三种网络模型分别提升了12.8%、6.9%、11.1%，充分证明了我们方法的优越性。

研究结果表明，本文提出的基于稀疏点阵与多适配器的钛钉钉匣缺陷检测方法能够精确定位和分割微小的表面缺陷，在工业表面缺陷检测领域中展现了显著的应用潜力。

Titanium staples are a critical component of modern medical surgical staplers, and the precision and efficiency of their defect detection are crucial for ensuring patient health and reducing medical accidents. This study provides an in-depth analysis of the technical challenges in the surface defect detection of titanium staples, comparing the detection results of advanced anomaly detection algorithms such as AnomalyGPT. It was found that large model methods like AnomalyGPT have limitations in detecting minor defects and require multiple prompts to capture weak anomaly signals. An innovative defect detection algorithm is proposed, which optimizes the PatchCore model to automatically generate effective prompts for EdgeSAM to accurately locate and segment minor surface defects.

(1) To address the issue that anomaly algorithms based on large models like AnomalyGPT require multiple attempts to generate high-quality prompts for defect segmentation, an anomaly sparse point array generation algorithm based on PatchCore is proposed. The algorithm can dynamically generate anomaly sparse point arrays tailored to different defects as 'prompts' for EdgeSAM, providing strong support for the segmentation of defect shapes and boundaries. Inspired by PatchCore, the input image is divided into multiple small rectangular areas, and then features are extracted through a dedicated convolutional neural network, and a global feature representation is formed through an aggregation mechanism; subsequently, the anomaly sparse point array is extracted and formed through a sparse point array generation algorithm processes these features.

(2) A titanium staple defect detection method that combines the defect sparse point array with multiple defect adapters fine-tuned with EdgeSAM is proposed. This method uses the defect sparse point array and defect adapters to prompt the EdgeSAM model, enabling precise segmentation of defect boundaries and their categories. The limitations of large models like AnomalyGPT in terms of real-time performance for inference are addressed, which prevents their direct application in the industrial field, and also improves the shortcomings of PatchCore in segmenting and classifying minor defects. Experimental results on a titanium staple dataset show that the proposed method achieved an MIoU of 63.9%, which is an improvement of 12.8%, 6.9%, and 11.1% over the U-Net, FCN, and Deeplabv3 network models, respectively, fully demonstrating the superiority of our approach.

The research results indicate that the defect detection method for titanium staples based on sparse point arrays and multiple adapters proposed in this study can accurately locate and segment minor surface defects, and has shown significant application potential in the field of industrial surface defect detection.

[1]Rajpurkar P, Chen E, Banerjee O, et al. AI in health and medicine[J]. Nature medicine, 2022, 28(1): 31-38.

[2]Yang M. Research on vehicle automatic driving target perception technology based on improved MSRPN algorithm[J]. Journal of Computational and Cognitive Engineering, 2022, 1(3): 147-151.

[3]Litjens G, Kooi T, Bejnordi B E, et al. A survey on deep learning in medical image analysis[J]. Medical image analysis, 2017, 42(4): 60-88.

[4]Cheng G, Xie X, Han J, et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13(21): 3735-3756.

[5]Rosa L, Cruz T, de Freitas M B, et al. Intrusion and anomaly detection for the next-generation of industrial automation and control systems[J]. Future Generation Computer Systems, 2021, 119(4): 50-67.

[6]Zeng N, Wu P, Wang Z, et al. A small-sized object detection oriented multi-scale feature fusion approach with application to defect detection[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71(1): 1-14.

[7]Cheng X, Yu J. RetinaNet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70(1): 1-11.

[8]Jain S, Seth G, Paruthi A, et al. Synthetic data augmentation for surface defect detection and classification using deep learning[J]. Journal of Intelligent Manufacturing, 2022, 38(1): 1-14.

[9]Ulku I, Akagündüz E. A survey on deep learning-based architectures for semantic segmentation on 2d images[J]. Applied Artificial Intelligence, 2022, 36(1): 2032924.

[10]Chen X, Wang X, Zhang K, et al. Recent advances and clinical applications of deep learning in medical image analysis[J]. Medical Image Analysis, 2022, 79: 102444.

[11]Guan H, Liu M. Domain adaptation for medical image analysis: a survey[J]. IEEE Transactions on Biomedical Engineering, 2021, 69(23): 1173-1185.

[12]Budd S, Robinson E C, Kainz B. A survey on active learning and human-in-the-loop deep learning for medical image analysis[J]. Medical Image Analysis, 2021, 71: 102062.

[13]张云洲, 吴峻. Prewitt霍尔磁梯度张量系统结构设计[J]. 仪器仪表学报, 2021, 42(09): 115-123.

[14]Sobel I E. Camera models and machine perception[M]. stanford university, 1970.

[15]黄梦涛, 连一鑫. 基于改进Canny算子的锂电池极片表面缺陷检测[J]. 仪器仪表学报, 2021, 42(10):199-209.

[16]Zuo B, Wang F. Surface cutting defect detection of magnet using Fourier image reconstruction[J]. Computer Engineering and Applications, 2016, 52(3): 256-260.

[17]袁立, 刘威. 基于Gabor特征和遮挡字典的带遮挡人耳识别[J]. 仪器仪表学报, 2015, 36(09): 2037-2043.

[18]汪贺延, 刘国栋, 庙存彬. 基于PSNR-NC函数优化和非抽样双树复小波变换的自适应多重水印算法[J]. 光学学报, 2022, 42(05) :52-65.

[19]Chetverikov D, Hanbury A. Finding defects in texture using regularity and local orientation[J]. Pattern Recognition, 2002, 35(10): 2165-2180.

[20]Hou Z, Parker J M. Texture defect detection using support vector machines with adaptive gabor wavelet features[C]//2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05)-Volume 1. IEEE, 2005, 1: 275-280.

[21]Peng X, Chen Y, Yu W, et al. An online defects inspection method for float glass fabrication based on machine vision[J]. The International Journal of Advanced Manufacturing Technology, 2008, 39(7): 1180-1189.

[22]Murino V, Bicego M, Rossi I A. Statistical classification of raw textile defects[C]//Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. IEEE, 2004, 4: 311-314.

[23]黄祎婧. 基于决策树的多视图动态KNN分类[D]. 华东交通大学, 2023.

[24]刘传泽, 王霄, 陈龙现等. 基于随机森林算法的纤维板表面缺陷识别[J]. 林业科学, 2018, 54(11): 121-126.

[25]Li Z, Liu F, Yang W, et al. A survey of convolutional neural networks: analysis, applications, and prospects[J]. IEEE transactions on neural networks and learning systems, 2021, 33(12): 6999-7019

[26]Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015: 234-241.

[27]Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848.

[28]何宇超, 段中兴, 高静. 基于多尺度空洞卷积结构的路面裂缝分割方法[J]. 公路交通科技, 2024, 41(01): 1-9.

[29]瞿中, 陈雯 .基于空洞卷积和多特征融合的混凝土路面裂缝检测[J]. 计算机科学, 2022, 49(03): 192-196.

[30]Wang H, Zhu Y, Adam H, et al. Max-deeplab: End-to-end panoptic segmentation with mask transformers[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 5463-5474.

[31]Llorella F R, Azorin J M, Patow G. Black hole algorithm with convolutional neural networks for the creation of brain-computer interface based in visual perception and visual imagery[J]. Neural Computing and Applications, 2023, 35(8): 5631-5641.

[32]He H, Yang D, Wang S, et al. Road extraction by using atrous spatial pyramid pooling integrated encoder-decoder network and structural similarity loss[J]. Remote Sensing, 2019, 11(9): 1015.

[33]Huang Y, Cao X, Zhen X, et al. Attentive temporal pyramid network for dynamic scene classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2019, 33(01): 8497-8504.

[34]杨若兰, 刘超, 周佳润, 等. 结合注意力机制的带钢表面缺陷检测模型[J]. 钢铁研究学报, 2024, 36(05): 669-679.

[35]Khan S, Naseer M, Hayat M, et al. Transformers in vision: A survey[J]. ACM computing surveys (CSUR), 2022, 54(10): 1-41.

[36]胡越杰, 蒋高明. SwinBN: 一种基于Swin Transformer的针织物疵点检测模型[J]. 丝绸, 2023, 60(01): 59-69.

[37]Wang B, Ying S, Cheng G, et al. Log-based anomaly detection with the improved k-nearest neighbor[J]. International Journal of Software Engineering and Knowledge Engineering, 2020, 30(02): 239-262.

[38]Angiulli F. CFOF: a concentration free measure for anomaly detection[J]. ACM Transactions on Knowledge Discovery from Data (TKDD), 2020, 14(1): 1-53.

[39]McIntosh D, Albu A B. Inter-Realization Channels: Unsupervised Anomaly Detection Beyond One-Class Classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 6285-6295.

[40]Xu K, Jiang X, Sun T. Anomaly detection based on stacked sparse coding with intraframe classification strategy[J]. IEEE Transactions on Multimedia, 2018, 20(5): 1062-1074.

[41]张可, 乐全明, 黄文礼, 等. 联合剪枝和知识蒸馏的PCSA-YOLOs防振锤缺陷检测算法[J]. 电工电能新技术, 2022, 41(11): 59-69.

[42]刘太亨, 何昭水. 基于自编码和知识蒸馏的表面缺陷检测方法[J]. 计算机应用, 2021, 41(11): 3200-3205.

[43]Tao X, Zhang D, Ma W, et al. Unsupervised anomaly detection for surface defects with dual-siamese network[J]. IEEE Transactions on Industrial Informatics, 2022, 18(11): 7707-7717.

[44]Yang M, Wu P, Feng H. MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities[J]. Engineering Applications of Artificial Intelligence, 2023, 119: 105835.

[45]Cao Y, Xu X, Sun C, et al. Towards generic anomaly detection and understanding: Large-scale visual-linguistic model (gpt-4v) takes the lead[J]. arXiv preprint arXiv:2311.02782, 2023.

[46]Roth, K., Pemula, L., Zepeda, J., Scholkopf, B., Brox, T., & Gehler, P. (2021). Towards Total Recall in Industrial Anomaly Detection. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14298-14308.

[47]Cao Y, Xu X, Sun C, et al. Segment Any Anomaly without Training via Hybrid Prompt Regularization[J]. arXiv preprint arXiv:2305.10724, 2023.

[48]Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023.

[49]Kirillov A, Mintun E, Ravi N, et al. Segment anything[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 4015-4026.

[50]Gu Z, Zhu B, Zhu G, et al. Anomalygpt: Detecting industrial anomalies using large vision-language models[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(3): 1932-1940.

[51]Zhu D, Chen J, Shen X, et al. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models[C]//The Twelfth International Conference on Learning Representations. 2023.

[52]Bazi Y, Bashmal L, Al Rahhal M M, et al. RS-LLaVA: A Large Vision-Language Model for Joint Captioning and Question Answering in Remote Sensing Imagery[J]. Remote Sensing, 2024, 16(9): 1477.

[53]Su Y, Lan T, Li H, et al. Pandagpt: One model to instruction-follow them all[J]. arXiv preprint arXiv:2305.16355, 2023.

[54]Wang X, Li C, Wang Z, et al. Promptagent: Strategic planning with language models enables expert-level prompt optimization[J]. arXiv preprint arXiv:2310.16427, 2023.

[55]Zhou C, Li X, Loy C C, et al. Edgesam: Prompt-in-the-loop distillation for on-device deployment of sam[J]. arXiv preprint arXiv:2312.06660, 2023.

[56]Ji, W., Li, J., Bi, Q., Li, W., & Cheng, L. (2023). Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications. ArXiv, abs/2304.05750.

[57]Bergmann P, Fauser M, Sattlegger D, et al. MVTec AD-A comprehensive real-world dataset for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 9592-9600.

[58]Lee S, Lee S, Song B C. Cfa: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization[J]. IEEE Access, 2022, 10: 78446-78454.

[59]Xia X, Pan X, Li N, et al. GAN-based anomaly detection: A review[J]. Neurocomputing, 2022, 493: 497-535.

[60]Defard T, Setkov A, Loesch A, et al. Padim: a patch distribution modeling framework for anomaly detection and localization[C]//International Conference on Pattern Recognition. Cham: Springer International Publishing, 2021: 475-489.

[61]Dehaene D, Frigo O, Combrexelle S, et al. Iterative energy-based projection on a normal data manifold for anomaly localization[C]//International Conference on Learning Representations. 2019: 434-450.