Cross-modal Recipe Retrieval with Rich Food Attributes

Abstract

Food is rich of visible (e.g., colour, shape) and procedural (e. cutting, cooking) attributes. Proper leveraging of these attribut particularly the interplay among ingredients, cutting and cooki methods, for health-related applications has not been previous explored. This paper investigates cross-modal retrieval of recip specifically to retrieve a text-based recipe given a food picture query. As similar ingredient composition can end up with wild different dishes depending on the cooking and cutting procedur the difficulty of retrieval originates from fine-grained recogniti of rich attributes from pictures. With a multi-task deep learni model, this paper provides insights on the feasibility of predicti ingredient, cutting and cooking attributes for food recognition a recipe retrieval. In addition, localization of ingredient regions also possible even when region-level training examples are n provided. Experiment results validate the merit of rich attribut when comparing to the recently proposed ingredient-only retriev techniques. © 2017 ACM.