DEEP LEARNING APPROACHES FOR ATTRIBUTE MANIPULATION AND TEXT-TO-IMAGE SYNTHESIS

Abstract

This thesis focuses on the problem of attribute manipulation and text-to-image synthesis. We first address the problem of image retrieval with attribute manipulation, e.g. replace the short sleeve attribute of the query image with the long sleeve. To solve this task, we propose FashionSearchNet which utilizes a weakly supervised localization method based on attribute activation mapping. Secondly, in order to solve the shortcoming of the image retrieval approach, we propose an image-to-image translation method based on Generative Adversarial Networks. Lastly, we propose enhanced-Attentional Generative Adversarial Network (e-AttnGAN) for text-to-image synthesis. Unlike conditioning on attributes, the use of text offers an alternative with more flexibility and semantics for specifying the desired item. Our experiments show the remarkable performance of the proposed methods which have a high potential for the image search/design.