基于特征通道和空间位置注意力的三维点云特征学习网络

摘要/Abstract

摘要： 点云模型的分类与部件分割是三维点云数据处理的基本任务，其核心在于获取可以有效表示三维模型的点云特征。提出一个引入注意力机制的三维点云特征学习网络。该网络采用多层次点云特征提取方法，首先使用特征通道注意力模块获取各通道间的关联，增强关键通道信息; 接着引入空间位置注意力机制，基于点的空间位置信息获取各点的注意力权重；然后结合以上2种注意力机制获取增强的点云特征;最后基于该特征继续进行多层次特征提取，获得面向下游任务的点云特征。分别在ModelNet40和ShapeNet数据集上进行形状分类与部件分割实验，结果表明，使用所提方法可以实现高精度、具有鲁棒性的三维点云形状分类与分割。

关键词: 点云模型, 注意力机制, 形状分类, 部件分割

Abstract: The classification and part segmentation of point cloud models are the basic tasks of 3D point cloud data processing, and the core is to obtain point cloud features that can effectively represent 3D models. This paper proposes a 3D point cloud feature learning network that introduces attention mechanisms. The network adopts a hierarchical point cloud feature extraction method. In the process of hierarchical feature extraction, the feature channel attention mechanism is adopted to obtain the correlation among channels, and the key channel information is enhanced. The spatial position attention mechanism is adopted to obtain the attention weight of each point based on the spatial information of the points. The enhanced point cloud feature is obtained by combining two or more attention mechanisms. Based on this feature, multi-level feature extraction is performed to obtain the final point cloud features for downstream tasks. Shape classification and part segmentation experiments are performed on ModelNet40 and ShapeNet datasets, respectively. The experimental results show that the proposed method can achieve high-precision and robust 3D point cloud shape classification and segmentation.

Key words: point cloud, attention mechanism, shape classification, part segmentation

吴亦奇, 韩放, 张德军, 何发智, 陈壹林. 基于特征通道和空间位置注意力的三维点云特征学习网络[J]. 计算机工程与科学, 2022, 44(07): 1239-1246.

WU Yi-qi, HAN Fang, ZHANG De-jun, HE Fa-zhi, CHEN Yi-lin. A 3D point cloud feature learning network based on feature channel and spatial position attentions[J]. Computer Engineering & Science, 2022, 44(07): 1239-1246.

参考文献［35］

［1］	Guo Y,Sohel F,Bennamoun M,et al.Rotational projection statistics for 3D local surface description and object recognition ［J］.International Journal of Computer Vision,2013,105(1):63-86.
［2］	Guo Y,Bennamoun M,Sohel F,et al.3D object recognition in cluttered scenes with local surface features:A survey ［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(11):2270-2287.
［3］	Minaee S,Boykov Y Y,Porikli F,et al.Image segmentation using deep learning:A survey ［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(7):1.
［4］	Chan T-H,Jia K,Gao S,et al.PCANet:A simple deep learning baseline for image classification? ［J］.IEEE Transactions on Image Processing,2015,24(12):5017-5032.
［5］	Guo Y,Wang H,Hu Q,et al.Deep learning for 3D point clouds:A survey ［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(12):4338-4364.
［6］	Ioannidou A,Chatzilari E,Nikolopoulos S,et al.Deep learning advances in computer vision with 3D data:A survey ［J］.ACM Computing Surveys (CSUR),2017,50(2):1-38.
［7］	Qi C R,Su H,Nieβner M,et al.Volumetric and multi-view CNNs for object classification on 3D data［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2016:5648-5656.
［8］	Su H,Maji S,Kalogerakis E,et al.Multi-view convolutional neural networks for 3D shape recognition［C］∥Proc of the IEEE International Conference on Computer Vision,2015:945-953.
［9］	Chen X,Ma H,Wan J,et al.Multi-view 3D object detection network for autonomous driving［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2017:1907-1915.
［10］	Yu T,Meng J,Yuan J.Multi-view harmonized bilinear network for 3D object recognition［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2018:186-194.
［11］	Yang Z,Wang L.Learning relationships for multi-view 3D object recognition［C］∥Proc of the IEEE/CVF International Conference on Computer Vision,2019:7505-7514.
［12］	Maturana D, Scherer S.VoxNet:A 3D convolutional neural network for real-time object recognition［C］∥Proc of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),2015:922-928.
［13］	Riegler G, Osman Ulusoy A, Geiger A. OctNet:Learning deep 3D representations at high resolutions［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2017:3577-3586.
［14］	Wang P-S, Liu Y, Guo Y-X, et al. O-CNN:Octree-based convolutional neural networks for 3D shape analysis ［J］.ACM Transactions on Graphics (TOG),2017,36(4):1-11.
［15］	Le T,Duan Y.PointGrid:A deep network for 3D shape understanding［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2018:9204-9214.
［16］	Qi C R,Su H,Mo K,et al.PointNet:Deep learning on point sets for 3D classification and segmentation［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2017:652-660.
［17］	Qi C R,Yi L,Su H,et al.PointNet++:Deep hierarchical feature learning on point sets in a metric space［C］∥Proc of the 31st International Conference on Neural Information Processing Systems,2017:5105-5114.
［18］	Duan Y,Zheng Y,Lu J,et al.Structural relational reasoning of point clouds［C］∥Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:949-958.
［19］	Yang J,Zhang Q,Ni B,et al.Modeling point clouds with self-attention and gumbel subset sampling［C］∥Proc of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:3318-3327.
［20］	Bai Jing, Si Qing-long,Qin Fei-wei.Lightweight real-time point cloud classification network LightPointNet［J］.Journal of Computer-Aided Design & Graphics,2019,31(4):612-621.(in Chinese)
［21］	Zhao H,Jiang L,Fu C-W,et al.PointWeb:Enhancing local neighborhood features for point cloud processing［C］∥Proc of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:5560-5568.
［22］	Xu Y,Fan T,Xu M,et al.SpiderCNN:Deep learning on point sets with parameterized convolutional filters［C］∥Proc of the European Conference on Computer Vision (ECCV),2018:87-102.
［23］	Wu W X,Qi Z A, Li F X.PointConv:Deep convolutional networks on 3D point clouds［C］∥Proc of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:9621-9630.
［24］	Li Y, Bu R,Sun M,et al.PointCNN:Convolution on x-transformed points ［C］∥Proc of Annual Conference on Neural Information Processing Systems,2018:828-838.
［25］	Peyghambarzadeh S M,Azizmalayeri F,Khotanlou H,et al.Point-PlaneNet:Plane kernel based convolutional neural network for point clouds analysis ［J］.Digital Signal Processing,2020,98:102633.
［26］	Thomas H, Qi C R,Deschaud J-E,et al.KPconv:Flexible and deformable convolution for point clouds［C］∥Proc of 2019 IEEE/CVF International Conference on Computer Vision,2019:6410-6419.
［27］	Shen Y, Feng C,Yang Y,et al.Mining point cloud local structures by kernel correlation and graph pooling［C］∥Proc of 2018 IEEE Conference on Computer Vision and Pattern Recognition,2018:4548-4557.
［28］	Wang C,Samari B,Siddiqi K.Local spectral graph convolution for point set feature learning［C］∥Proc of the European Conference on Computer Vision (ECCV),2018:52-66.
［29］	Wang Y,Sun Y,Liu Z,et al.Dynamic graph CNN for learning on point clouds ［J］.ACM Transactions on Graphics,2019,38(5):1-12.
［30］	Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need［C］∥Proc of the 31st International Conference on Neural Information Processing Systems,2017:5998-6008.
［31］	Lü Fan, Hu Fu-yuan,Zhang Yan-ning,et al.Feedback attention model for image captioning ［J］.Journal of Computer-Aided Design & Graphics,2019,31(7):1122-1129.(in Chinese)
［32］	Jia Ning,Zheng Chun-jun.Model of music theme recommendation based on attention LSTM ［J］.Computer Science,2019,46(S2):230-235.(in Chinese)
［33］	Huang Huan,Sun Li-juan,Cao Ying,et al.Multimodal sentiment analysis of short videos based on attention ［J］.Journal of Graphics,2021,42(1):8-14.(in Chinese)
［34］	Chen Qing-wen,Xie Hong-wen,Zha Hao,et al.Salient object detection based on deep clustering attention mechanism ［J］.Journal of Image and Graphics,2021,26(5):1017-1029.(in Chinese)
［35］	Li Xuan-ye,Hao Xing-wei,Jia Jin-gong,et al.Human action recognition method based on multi-attention mechanism and spatiotemporal graph convolution ［J］.Journal of Computer-Aided Design & Graphics,2021,33(7):1055-1063.(in Chinese)
［36］	Wu Z,Song S,Khosla A,et al.3D ShapeNets:A deep representation for volumetric shapes［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2015:1912-1920.
［37］	Li J,Chen B M,Lee G H.SO-Net:Self-organizing network for point cloud analysis［C］∥Proc of the IEEE Conference on Computer Vision and Pattern Recognition,2018:9397-9406.
［38］	Klokov R, Lempitsky V. Escape from cells:Deep kd- networks for the recognition of 3D point cloud models［C］∥Proc of 2017 IEEE International Conference on Computer Vision,2017:863-872.
［39］	He X,Cao H L,Zhu B.AdvectiveNet:An Eulerian-Lagrangian fluidic reservoir for point cloud processing ［J］.arXiv:200200118,2020.
［40］	Guo M H,Cai J X,Liu Z N,et al.PCT:Point cloud transformer ［J］.Computational Visual Media,2021,7(2):187-199.
［41］	Qiu S, Anwar S,Barnes N.Dense-resolution network for point cloud classification and segmentation［C］∥Proc of the IEEE/CVF Winter Conference on Applications of Computer Vision,2021:3813-3822.
［42］	Yi L,Kim V G,Ceylan D,et al.A scalable active framework for region annotation in 3D shape collections ［J］.ACM Transactions on Graphics,2016,35(6):1-12.
［43］	Huang Q,Wang W,Neumann U.Recurrent slice networks for 3D segmentation of point clouds［C］∥Proc of 2018 IEEE Conference on Computer Vision and Pattern Recognition,2018:2626-2635.
［44］	Zhang D,He L,Luo M,et al.Weight asynchronous update:Improving the diversity of filters in a deep convolutional network ［J］.Computational Visual Media,2020,6(4):455-66.
［45］	Zhang D,He F,Tu Z,et al.Pointwise geometric and semantic learning network on 3D point clouds ［J］.Integrated Computer-Aided Engineering,2020,27(1):57-75.
［46］	Zhang D,He L,Tu Z,et al.Learning motion representation for real-time spatio-temporal action localization ［J］.Pattern Recognition,2020,103:107312.
［47］	Chen Y,He F,Wu Y,et al.A local start search algorithm to compute exact Hausdorff distance for arbitrary point sets ［J］.Pattern Recognition,2017,67:139-48.
［48］	Chen Y L,He F Z, Li H R,et al.A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration ［J］.Applied Soft Computing,2020,93:106335.
	附中文参考文献：
［20］	白静,司庆龙,秦飞巍.轻量级实时点云分类网络LightPointNet ［J］.计算机辅助设计与图形学学报,2019,31(4):612-621.
［31］	吕凡,胡伏原,张艳宁,等.面向图像自动语句标注的注意力反馈模型［J］.计算机辅助设计与图形学学报,2019,31(7):1122-1129.
［32］	贾宁,郑纯军.基于注意力LSTM的音乐主题推荐模型［J］.计算机科学,2019,46(S2):230-235.
［33］	黄欢,孙力娟,曹莹,等.基于注意力的短视频多模态情感分析［J］.图学学报,2021,42(1):8-14.
［34］	陈庆文,谢宏文,查浩,等.深度聚类注意力机制下的显著对象检测［J］.中国图象图形学报,2021,26(5):1017-1029.
［35］	李炫烨,郝兴伟,贾金公,等.结合多注意力机制与时空图卷积网络的人体动作识别方法［J］.计算机辅助设计与图形学学报,2021,33(7):1055-1063.

[1]	马思远, 焦佳辉, 任晟岐, 宋伟. 基于注意力机制的城市多元空气质量数据缺失值填充[J]. 计算机工程与科学, 2023, 45(08): 1354-1364.
[2]	尹春勇, 冯梦雪. 基于注意力机制的半监督日志异常检测方法[J]. 计算机工程与科学, 2023, 45(08): 1405-1415.
[3]	余子丞, 凌捷. 基于Transformer和多特征融合的DGA域名检测方法[J]. 计算机工程与科学, 2023, 45(08): 1416-1423.
[4]	唐剑, 车文刚, 高盛祥. 融入注意力机制的多尺度卷积图像去雾方法[J]. 计算机工程与科学, 2023, 45(08): 1453-1462.
[5]	吴栋梁, 刘知贵, . 基于轻量化YOLOX的电子元器件缺陷检测方法研究[J]. 计算机工程与科学, 2023, 45(08): 1463-1471.
[6]	刘浩翰, 孙铖, 贺怀清, 惠康华. 基于改进YOLOv3的金属表面缺陷检测[J]. 计算机工程与科学, 2023, 45(07): 1226-1235.
[7]	白杉, 冯秀芳. 基于注意力增强的中心差分自适应图卷积的骨架行为识别[J]. 计算机工程与科学, 2023, 45(07): 1263-1273.
[8]	王剑, 姜林, 王琳钦, 余正涛, 张松, 高盛祥, . 基于BiLSTM的低资源老挝语文本正则化任务[J]. 计算机工程与科学, 2023, 45(07): 1292-1299.
[9]	田秀霞, 刘正, 刘秋旭, 李浩然. 一种改进Faster R-CNN的图像篡改检测模型[J]. 计算机工程与科学, 2023, 45(06): 1030-1039.
[10]	杨慧剑, 孟亮. 基于改进的YOLOv5的航拍图像中小目标检测算法[J]. 计算机工程与科学, 2023, 45(06): 1063-1070.
[11]	刘晓航, 姜晶菲, 许金伟. 基于脉动阵列的层融合注意力模型加速器结构[J]. 计算机工程与科学, 2023, 45(05): 802-809.
[12]	排日旦·阿布都热依木, 吐尔地·托合提, 艾斯卡尔·艾木都拉, . 基于深度学习的实体关系抽取方法研究[J]. 计算机工程与科学, 2023, 45(05): 895-902.
[13]	赵乐乐, 张丽萍, 赵凤荣. 基于注意力机制的Tree2Seq代码注释自动生成[J]. 计算机工程与科学, 2023, 45(04): 638-645.
[14]	厍向阳, 马亦骏. 改进的遥感图像语义分割算法[J]. 计算机工程与科学, 2023, 45(03): 504-511.
[15]	张若一, 金柳, 马慧芳, 王亦可, 李清风. 融合相似用户影响效应的知识图谱推荐模型[J]. 计算机工程与科学, 2023, 45(03): 520-527.