MCMT-GAN: Multi-Task Coherent Modality Transferable GAN for 3D Brain Image Synthesis

Document Type

Article

Source of Publication

IEEE Transactions on Image Processing

Publication Date

1-1-2020

Abstract

© 1992-2012 IEEE. The ability to synthesize multi-modality data is highly desirable for many computer-aided medical applications, e.g. clinical diagnosis and neuroscience research, since rich imaging cohorts offer diverse and complementary information unraveling human tissues. However, collecting acquisitions can be limited by adversary factors such as patient discomfort, expensive cost and scanner unavailability. In this paper, we propose a multi-task coherent modality transferable GAN (MCMT-GAN) to address this issue for brain MRI synthesis in an unsupervised manner. Through combining the bidirectional adversarial loss, cycle-consistency loss, domain adapted loss and manifold regularization in a volumetric space, MCMT-GAN is robust for multi-modality brain image synthesis with visually high fidelity. In addition, we complement discriminators collaboratively working with segmentors which ensure the usefulness of our results to segmentation task. Experiments evaluated on various cross-modality synthesis show that our method produces visually impressive results with substitutability for clinical post-processing and also exceeds the state-of-the-art methods.

ISSN

1057-7149

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Volume

29

First Page

8187

Last Page

8198

Disciplines

Computer Sciences

Keywords

anatomical structure, brain MRI, GANs, multi-modality, Synthesis

Indexed in Scopus

no

Open Access

no

Link to Full Text

Share

COinS