SparseMeshCNN with Self-Attention for Segmentation of Large Meshes
DOI:
https://doi.org/10.7557/18.6281Keywords:
Geometric Deep Learning, Mesh Segmentation, Self-AttentionAbstract
In many clinical applications, 3D mesh models of human anatomies are important tools for visualization, diagnosis, and treatment planning. Such 3D mesh models often have a high number of vertices to capture the complex shape, and processing these large meshes on readily available graphic cards can be a challenging task. To accommodate this, we present a sparse version of MeshCNN called SparseMeshCNN, which can process meshes with more than 60 000 edges. We further show that adding non-local attention in the network can mitigate the small receptive field and improve the results. The developed methodology was applied to separate the Left Atrial Appendage (LAA) from the Left Atrium (LA) on 3D mesh models constructed from medical images, but the method is general and can be put to use in any application within mesh classification or segmentation where memory can be a concern.
References
A. Barda, Y. Erel, and A. H. Bermano. Meshcnn fundamentals: Geometric learning through a reconstructable representation. CoRR, abs/2105.13277, 2021. URL https://arxiv.org/abs/2105.13277.
A. Cresti, M. A. Garc´ıa-Fern´andez, H. Sievert, P. Mazzone, P. Baratta, M. Solari, A. Geyer, F. De Sensi, and U. Limbruno. Prevalence of extra-appendage thrombosis in non-valvular atrial fibrillation and atrial flutter in patients undergoing cardioversion: a large transoesophageal echo study. EuroIntervention, 15: e225–e230, 2019. doi: 10.4244/eij-d-19-00128.
M. Glikson, R. Wolff, G. Hindricks, J. Mandrola, A. J. Camm, G. Y. Lip, L. Fauchier, T. R. Betts, T. Lewalter, J. Saw, A. Tzikas, L. Sternik, F. Nietlispach, S. Berti, H. Sievert, S. Bertog, and B. Meier. EHRA/EAPCI expert consensus statement on catheter-based left atrial appendage occlusion - An update. EuroIntervention, 15(13):1133–1180, 2020. doi: 10.4244/EIJY19M08 01.
S. Gong, L. Chen, M. Bronstein, and S. Zafeiriou. Spiralnet++: A fast and highly efficient mesh convolution operator. Proceedings - 2019 International Conference on Computer Vision Workshop, Iccvw 2019, 2019. doi: 10.1109/ICCVW.2019.00509.
R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-Or. Meshcnn: A network with an edge. Acm Transactions on Graphics, 38(4):90, 2019. doi: 10.1145/3306346.3322959.
A. Lahav and A. Tal. Meshwalker: Deep mesh understanding by random walks. CoRR, abs/2006.05353, 2020. URL https://arxiv.org/abs/2006.05353.
H. Leventi´c, D. Babin, L. Velicki, D. Devos, I. Gali´c, V. Zlokolica, K. Romi´c, and A. Piˇzurica. Left atrial appendage segmentation from 3D CCTA images for occluder placement procedure. Computers in Biology and Medicine, 104:163–174, 2019. doi: 10.1016/j.compbiomed.2018.11.006.
W. E. Lorensen and H. E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In ACM siggraph computer graphics, volume 21, pages 163–169. ACM, 1987. doi: 10.1145/37402.37422.
H. Maron, M. Galun, N. Aigerman, M. Trope, N. Dym, E. Yumer, V. G. Kim, and Y. Lipman. Convolutional neural networks on surfaces via seamless toric covers. Acm Transactions on Graphics, 36(4):71, 2017. doi: 10.1145/3072959.3073616.
F. Milano, A. Loquercio, A. Rosinol, D. Scaramuzza, and L. Carlone. Primal-dual mesh convolutional neural networks. In Conference on Neural Information Processing Systems (NeurIPS), 2020. URL https://github.com/MIT-SPARK/PD-MeshNet.
F. Monti, D. Boscaini, J. Masci, E. Rodol`a, J. Svoboda, and M. M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. Proceedings - 30th Ieee Conference on Computer Vision and Pattern Recognition, Cvpr 2017, pages 5425–5434, 2017. doi: 10.1109/CVPR.2017.576.
F. Monti, O. Shchur, A. Bojchevski, O. Litany, S. G¨unnemann, and M. M. Bronstein. Dualprimal graph convolutional networks. volume abs/1806.00770, 2018. URL http://arxiv.org/abs/1806.00770.
R. R. Paulsen, J. A. Baerentzen, and R. Larsen. Markov random field surface reconstruction. IEEE transactions on visualization and computer graphics, 16(4):636–646, 2009. doi: 10.1109/TVCG.2009.208.
C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. volume abs/1612.00593, 2016. URL http://arxiv.org/abs/1612.00593.
C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. CoRR, abs/1706.02413, 2017. URL http://arxiv.org/abs/1706.02413.
O. Ronneberger, P. Fischer, and T. Brox. Unet: Convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597, 2015. URL http://arxiv.org/abs/1505.04597.
L. Schneider, A. Niemann, O. Beuing, B. Preim, and S. Saalfeld. Medmeshcnn - enabling meshcnn for medical surface models. CoRR, abs/2009.04893, 2020. URL https://arxiv.org/abs/2009.04893.
H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. Proceedings of the IEEE International Conference on Computer Vision, 2015, 2015. doi: 10.1109/ICCV.2015.114.
M. Tatarchenko, J. Park, V. Koltun, and Q. Y. Zhou. Tangent convolutions for dense prediction in 3d. Proceedings of the Ieee Computer Society Conference on Computer Vision and Pattern Recognition, 2018. doi: 10.1109/CVPR.2018.00409.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. URL http://arxiv.org/abs/1706.03762.
P. Veliˇckovi´c, A. Casanova, P. Li`o, G. Cucurull, A. Romero, and Y. Bengio. Graph attention networks. 6th International Conference on Learning Representations, Iclr 2018 - Conference Track Proceedings, 2018. URL https://openreview.net/forum?id=rJXMpikCZ.
P. S. Wang, Y. Liu, Y. X. Guo, C. Y. Sun, and X. Tong. O-cnn: Octree-based convolutional neural networks for 3d shape analysis. Acm Transactions on Graphics, 36(4), 2017. doi: 10.1145/3072959.3073608.
X. Wang, R. Girshick, A. Gupta, and K. He. Non-local neural networks. Proceedings of the Ieee Computer Society Conference on Computer Vision and Pattern Recognition, 2018. doi: 10.1109/CVPR.2018.00813.
X. Yang, D. Xia, T. Kin, and T. Igarashi. INTRA: 3D intracranial aneurysm dataset for deep learning. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2653–2663. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. doi: 10.1109/CVPR42600.2020.00273.
C¸ i¸cek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger. 3d u-net: Learning dense volumetric segmentation from sparse annotation. Lecture Notes in Computer Science (including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9901:424–432, 2016. doi: 10.1007/978-3-319-46723-8 49.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 Bjørn Hansen, Mathias Lowes, Thomas Ørkild, Anders Dahl, Vedrana Dahl, Ole de Backer, Oscar Camara, Rasmus Paulsen, Christian Ingwersen, Kristine Sørensen
This work is licensed under a Creative Commons Attribution 4.0 International License.
