Tumor Detection in Brain MRIs by Computing Dissimilarities in the Latent Space of a Variational AutoEncoder


  • Alexandra Albu Romanian Institute of Science an Technology
  • Alina Enescu Romanian Institute of Science an Technology and Universitatea Babeș-Bolyai
  • Luigi Malagò Romanian Institute of Science an Technology




variational autoencoders, anomaly detection, brain MRI


The ability to automatically detect anomalies in brain MRI scans is of great importance in computer-aided diagnosis. Unsupervised anomaly detection methods work primarily by learning the distribution of healthy images and identifying abnormal tissues as outliers. We propose a slice-wise detection method which first trains a pair of autoencoders on two different datasets, one with healthy individuals and the other one with images of normal and tumoral tissues. Next, it classifies slices based on the distance in the latent space between the enconding of the image and the encoding of the reconstructed image, obtained through the autoencoder trained on healthy images only. We validate our approach with a series of preliminary experiments on the HCP and BRATS-15 datasets.


J. Bandaria. Brain MRI image segmentation using stacked denoising autoencoders.

C. Baur, B. Wiestler, S. Albarqouni, andN. Navab. Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. international MICCAI BrainlesionWorkshop pages 161–169. Springer, 2018.

X. Chen and E. Konukoglu. Unsupervised detection of lesions in brain MRI using constrained adversarial auto-encoders.arXiv:1806.04972,2018.

X. Chen, N. Pawlowski, M. Rajchl, B. Glocker, and E. Konukoglu. Deep generative models in the real-world: an open challenge from medical imaging.arXiv:1806.05452, 2018.

H. Choi, H. Kang, D. S. Lee, A. D. N. Initiative, et al. Predicting aging of brain metabolic topography using variational autoencoder. Frontiers in aging neuroscience, 10:212, 2018.

S. Jegou. How to estimate the age of your brain with MRI data.

D. P. Kingma and M. Welling.Auto-encoding variational Bayes. arXiv preprintarXiv:1312.6114, 2013.

M. Kistler, S. Bonetti,M. Pfahler,R. Niklaus, and P. B ̈uchler. The virtual skeleton database: an open access repository for biomedical research and collaboration. Journal of medical Internet research, 15(11):e245, 2013.

K. W. Lau. Representation learning on brain MR images for tumor segmentation, 2018.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A.Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken, and C. I. S ́anchez.A survey on deep learning in medical image analysis.Medical Image Analysis, 42:60 – 88,2017.

B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats).IEEE transactions on medical imaging, 34(10):1993–2024, 2014.

A. Myronenko. 3d MRI brain tumor segmentation using autoencoder regularization. In International MICCAI Brainlesion Workshop, pages 311–320. Springer, 2018.

W. H. Pinaya, A. Mechelli, and J. R. Sato.Using deep autoencoders to identify abnormal brain structural patterns in neuropsychiatric disorders: A large-scale multi-sample study. Human brain mapping, 40(3):944–954, 2019.

D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXivpreprint arXiv:1401.4082, 2014.

D. C. Van Essen, K. Ugurbil, E. Auerbach, D. Barch, T. Behrens, R. Bucholz, A. Chang, L. Chen, M. Corbetta, S. W. Curtiss, et al.The human connectome project: a data acquisition perspective. Neuroimage, 62(4):2222–2231,2012.

Q. Zhao, E. Adeli, N. Honnorat, T. Leng, and K. M. Pohl. Variational autoencoder for regression: Application to brain aging analysis.arXiv:1904.05948, 2019.

D. Zimmerer, S. A. Kohl, J. Petersen,F. Isensee, and K. H. Maier-Hein. Context-encoding variational autoencoder for unsupervised anomaly detection.arXiv:1812.05941,2018.

D. Zimmerer, J. Petersen, S. A. Kohl, and K. H.Maier-Hein. A case for the score: Identifyingimage anomalies using variational autoencoder gradients.