Multi-style image transfer system using conditional cycleGAN

ABSTRACT

This paper aims to extend the capability of Cycle-Consistent Adversarial Network (CycleGAN) by equipping it with a conditional constraint and extend it into a multi-style image transfer system that can transfer images among more than two image domains. The conditional constraint is given in the form of the target style feature map instead of a one-hot vector, and has shown to provide better transfer results. The proposed system offers greater flexibility for users to choose the style for image transfer. Experimental results show that such an architecture is not only feasible but also yields good results. The proposed architecture can be extended to other transformation applications, such as facial expressions transfer, face aging, and synthesis of various features.

KEYWORDS:

• Convolutional neural network (CNN)
• deep learning
• generative adversarial net (GAN)
• conditional GAN (CGAN)
• CycleGAN
• conditional CycleGAN
• PatchGAN
• image style transfer

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was supported in part by the Ministry of Science and Technology of Taiwan, R.O.C. under the grant MOST-108-2221-E-032-033.

Notes on contributors

Ching-Ting Tu

Ching-Ting Tu received her B.S. degree and Ph.D. in Computer Science and Information Engineering from National Cheng Kung University, Tainan, Taiwan, in 2004 and 2010. She is currently an Associate Professor at the Department of Applied Mathematics, National Chung Hsing University, Taichung, Taiwan. Her research interests include computer vision and pattern recognition, human-computer interaction, and multimedia information analysis and retrieval.

Hwei Jen Lin

Hwei-Jen Lin received the B.S. degree in Applied Mathematics from National Chiao Tung University, Hsinchu, Taiwan in 1981 and the M.S. and the Ph.D. degrees in Mathematics from Northeastern University, Boston, U.S.A. in 1983 and 1989. She is currently a professor at the Department of Computer Science and Information Engineering of Tamkang University. Her current research interests include pattern recognition, image processing, computational intelligence, and deep learning.

Yihjia Tsia

Yihjia Tsai received the B.S. degree in Mechanical Engineering from National Taiwan University, Taipei, Taiwan in 1984 and the M.S. and the Ph.D. degrees in Computer Science from Michigan State University, Michigan, U.S.A. in 1993 and 1997. He is currently an Associate Professor at the Department of Information Engineering of Tamkang University, Taipei, Taiwan. His current research interests include Complex Network Analysis, Evolutionary Algorithms, and Nature-Inspired Computation.