NEURAL NETWORK REPRESENTATION SIMILARITY REVISITED

Abstract

As neural networks achieve impressive empirical success across a wide range of tasks, an increasing number of studies have sought to understand and interpret neural networks. One popular direction is analyzing intermediate representations learned by networks, which motivated the development of representation similarity metrics. These metrics are applied to interpret how different network architectures and hyperparameters affect learned representations. Existing similarity metrics are designed to be invariant to neuron permutations. However, this property may not be suitable for many popular network architectures, including convolutional networks. Therefore we introduce a simple and generally applicable fix to adjust existing similarity metrics. The improvement leads to a higher accuracy in the layer prediction task. Further, we apply similarity metrics in model ensembles and successfully improve ensemble accuracy by enforcing model diversity based on similarity metrics. One of the major concerns in the representational similarity field is the lack of ground truth. In the case where two metrics yield different outcomes, it is hard to tell which one we should believe. We design a sanity check to verify the reliability of similarity metrics under the ”same network, different input” setting.