QUOTE: "querying" users as oracles in tag engines a semi-supervised learning approach to personalized image tagging

Abstract

Previous work has correctly identified that many of the tags that users provide on images are not visually relevant, and subsequently assume such user generated tags are noise or irrelevant. Another assumption about user generated tags is that the order of these tags provides no useful information for the prediction of tags on future images. We challenge the aforementioned assumptions, and provide a machine learning approach to the problem of personalized image tagging with the following contributions: 1.) We reformulate the personalized image tagging problem as a search/retrieval ranking problem, 2.) We leverage the order of tags, which does not always reflect visual relevance, provided by the user in the past as a cue to their tag preferences, similar to click data, 3.) We propose a technique to augment sparse user tag data (semi-supervision), and 4.) We demonstrate the efficacy of our method on a subset of Flickr images, showing improvement over our previous methods.