Multiple Hypothesis Video Relation Detection

Abstract

Video relation in the form of triplet (subject, predicate, object) plays a vital role in video content understanding. Existing works on video relation detection are limited to associating short-term relations into long-term relations throughout the video, because of the inaccurate and missing problem of short-term proposals. To alleviate the weakness of existing video relation detection methods, this work proposes a novel approach called Multi-Hypothesis Relational Association (MHRA), that can generate multiple hypotheses for video relation instances for more robust long-term relation prediction. Experiments on the benchmark dataset show that MHRA is able to outperform the state-of-the-art methods. © 2019 IEEE.