Automatic Postoperative Brain Tumor Segmentation with Limited Data using Transfer Learning and Triplet Attention
DOI:
https://doi.org/10.7557/18.6826Keywords:
Postoperative Segmentation, Transfer Learning, Triplet AttentionAbstract
Accurate brain tumor segmentation is clinically important for diagnosis and treatment planning. Convolutional neural networks (CNNs) have achieved promising performance in various visual recognition tasks. Training such networks usually requires large amount of labeled data, which is often challenging for medical applications. In this work, we address the segmentation problem by applying transfer learning to downstream segmentation tasks. Specifically, we explore how knowledge acquired from a large preoperative dataset can be transferred to postoperative tumor segmentation on a smaller dataset. To this end, we have developed a 3D CNN for brain tumor segmentation, and fine-tuned the pretrained models on the target domain data. To better exploit the inter-channel and spatial information, triplet attention has been incorporated and extended into existing segmentation network. Extensive experiments on our dataset demonstrate the effectiveness of transfer learning and attention modules for improved postoperative tumor segmentation performance when only limited amount of annotated data is available.
References
Z. Akkus, A. Galimzianova, A. Hoogi, D. L. Rubin, and B. J. Erickson. Deep learning for brain mri segmentation: state of the art and future directions. Journal of digital imaging, 30(4):449-459, 2017. doi: 10.1007/s10278-017-9983-4.
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby, J. B. Freymann, K. Farahani, and C. Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientic data, 4(1):1-13, 2017. doi: 10.1038/sdata.2017.117.
S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Remper, A. Crimi, R. T. Shinohara, C. Berger, S. M. Ha, M. Rozycki, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629, 2018.
S. Deepak and P. Ameer. Brain tumor classication using deep cnn features via transfer learning. Computers in biology and medicine, 111:103345, 2019. ISSN 0010-4825. doi: 10.1016/j.compbiomed.2019.103345.
A. K. Dhara, K. R. Ayyalasomayajula, E. Arvids, M. Fahlstrom, J. Wikstrom, E.-M. Larsson, and R. Strand. Segmentation of post-operative glioblastoma in mri by u-net with patient-specic interactive renement. In International MICCAI Brainlesion Workshop, pages 115-122. Springer, 2018. doi: 10.1007/978-3-030-11723-8 11.
M. Ghaari, G. Samarasinghe, M. Jameson, F. Aly, L. Holloway, P. Chlap, E.-S. Koh, A. Sowmya, and R. Oliver. Automated post-operative brain tumour segmentation: A deep learning model based on transfer learning from preoperative images. Magnetic resonance imaging, 86:28-36, 2022. doi: 10.1016/j.mri.2021.10.012.
M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, C. Pal, P.-M. Jodoin, and H. Larochelle. Brain tumor segmentation with deep neural networks. Medical image analysis, 35:18{31, 2017. doi: 10.1016/j.media.2016.05.004.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016. doi: 10.1109/CVPR.2016.90.
J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132-7141, 2018. doi: 10.1109/CVPR.2018.00745.
F. Isensee, M. Schell, I. Pueger, G. Brugnara, D. Bonekamp, U. Neuberger, A. Wick, H.-P. Schlemmer, S. Heiland, W. Wick, et al. Automated brain extraction of multisequence mri using articial neural networks. Human brain mapping, 40(17):4952-4964, 2019. doi: 10.1002/hbm.24750.
F. Isensee, P. F. Jager, P. M. Full, P. Vollmuth, and K. H. Maier-Hein. nnu-net for brain tumor segmentation. In International MICCAI Brainlesion Workshop, pages 118-132. Springer, 2020.
R. Mehta, A. Filos, U. Baid, C. Sako, R. McKinley, M. Rebsamen, K. Datwyler, R. Meier, P. Radojewski, G. K. Murugesan, et al. Qu-brats: Miccai brats 2020 challenge on quantifying uncertainty in brain tumor segmentation-analysis of ranking metrics and benchmarking results. arXiv preprint arXiv:2112.10074,2021.
B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging, 34(10):1993{2024, 2014. doi: 10.1109/TMI.2014.2377694.
D. Misra, T. Nalamada, A. U. Arasanipalai, and Q. Hou. Rotate to attend: Convolutional triplet attention module. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 3138-3147, 2021. doi: 10.1109/WACV48630.2021.00318.
A. Myronenko. 3d mri brain tumor segmentation using autoencoder regularization. In International MICCAI Brainlesion Workshop, pages 311-320. Springer, 2018. doi: 10.1007/978-3-030-11726-9 28.
O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015. doi: 10.1007/978-3-319-24574-4 28.
D. Ulyanov, A. Vedaldi, and V. Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.
J. M. Valverde, V. Imani, A. Abdollahzadeh, R. De Feo, M. Prakash, R. Ciszek, and J. Tohka. Transfer learning in magnetic resonance brain imaging: a systematic review. Journal of imaging, 7(4):66, 2021. doi: 10.3390/jimaging7040066.
J. Wacker, M. Ladeira, and J. E. V. Nascimento. Transfer learning for brain tumor segmentation. In International MICCAI Brainlesion Workshop, pages 241-251. Springer, 2020. doi: 10.1007/978-3-030-72084-1 22.
G. Wang, W. Li, S. Ourselin, and T. Vercauteren. Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In International MICCAI brainlesion workshop, pages 178-190. Springer, 2017.
R. Zoetmulder, E. Gavves, M. Caan, and H. Marquering. Domain-and taskspecic transfer learning for medical segmentation tasks. Computer Methods and Programs in Biomedicine, 214:106539, 2022. doi: 10.1016/j.cmpb.2021.106539.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Jingpeng Li, Atle Bjørnerud
This work is licensed under a Creative Commons Attribution 4.0 International License.
