LEARNING SCENE HIERARCHY: FROM CATEGORY-LEVEL SIMILARITY TO ATTRIBUTE-LEVEL SIMILARITY

Abstract

Scene understanding is approached from a higher perspective in this work. We make our initial attempt at revealing the intricate connections among various scene concepts via a tree-like hierarchy, i.e. scene hierarchy. In particular, a data-driven algorithm is developed to learn the underlying scene hierarchy. Upon comparative experiments, it is found this algorithm is effective when representing visual scenes via CNN-yield features rather than handcrafted descriptors. Subsequently, in search of the best CNN-yield features, different optimization schemes are examined, and the corresponding category-level similarity, explicit attribute-level similarity, and implicit attribute-level similarity are investigated. Experiments performed on both 8-Scenes dataset and 80-Scenes dataset demonstrate the superiority of attribute-level similarities. To further promote this idea, a novel optimization criterion called Simplex triplet loss is additionally designed. It not only enables interpretable metric learning but also allows the measurement of interpretable attribute-level similarity, and the derived scene hierarchy agrees closely with human perception.