Please use this identifier to cite or link to this item:
Title: Interpretable Multimodal Retrieval for Fashion Products.
Authors: Lizi Liao 
Xiangnan He 
Bo Zhao
Chong-Wah Ngo 
Tat-Seng Chua 
Keywords: Attribute manipulation
EI tree
Multimodal fashion retrieval
Issue Date: 26-Oct-2018
Publisher: Association for Computing Machinery, Inc
Citation: Lizi Liao, Xiangnan He, Bo Zhao, Chong-Wah Ngo, Tat-Seng Chua (2018-10-26). Interpretable Multimodal Retrieval for Fashion Products.. ACM Multimedia Conference 2018 : 1571-1579. ScholarBank@NUS Repository.
Abstract: Deep learning methods have been successfully applied to fashion retrieval. However, the latent meaning of learned feature vectors hinders the explanation of retrieval results and integration of user feedback. Fortunately, there are many online shopping websites organizing fashion items into hierarchical structures based on product taxonomy and domain knowledge. Such structures help to reveal how human perceive the relatedness among fashion products. Nevertheless, incorporating structural knowledge for deep learning remains a challenging problem. This paper presents techniques for organizing and utilizing the fashion hierarchies in deep learning to facilitate the reasoning of search results and user intent. The novelty of our work originates from the development of an EI (Exclusive & Independent) tree that can cooperate with deep models for end-to-end multimodal learning. EI tree organizes the fashion concepts into multiple semantic levels and augments the tree structure with exclusive as well as independent constraints. It describes the different relationships among sibling concepts and guides the end-to-end learning of multi-level fashion semantics. From EI tree, we learn an explicit hierarchical similarity function to characterize the semantic similarities among fashion products. It facilitates the interpretable retrieval scheme that can integrate the concept-level feedback. Experiment results on two large fashion datasets show that the proposed approach can characterize the semantic similarities among fashion items accurately and capture user's search intent precisely, leading to more accurate search results as compared to the state-of-the-art methods. © 2018 Association for Computing Machinery.
Source Title: ACM Multimedia Conference 2018
ISBN: 9781450356657
DOI: 10.1145/3240508.3240646
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
Interpretable multimodal retrieval for fashion products.pdf5.32 MBAdobe PDF




checked on Jun 25, 2022

Page view(s)

checked on Jun 23, 2022


checked on Jun 23, 2022

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.