TOWARD EFFECTIVE AND EFFICIENT GRAPH NEURAL NETWORKS WITH IMPLICIT LAYERS

Abstract

Motivated by limitations regarding effectiveness and memory efficiency, in the first work, we propose a GNN model with infinite depth, which we call Efficient Infinite-Depth Graph Neural Networks (EIGNN), to efficiently capture very long-range dependencies. In the second work, we introduce and justify two limitations of EIGNN and previous implicit GNNs: the constrained expressiveness due to their limited effective range for capturing long-range dependencies, and their lack of ability to capture multiscale information on graphs at multiple resolutions. Lastly, we focus on making implicit GNNs able to be trained on large graphs. Despite the advantages of memory efficiency and the better ability to capture long-range dependencies, existing implicit GNNs still have some limitations such as scalability and training efficiency issues on large graphs.