103 citations · 623 across the 53 of their papers we have counts for
9 papers · 1 filter
Topology-aware Generalization of Decentralized SGD
Tongtian Zhu, Fengxiang He, Lan Zhang +3
This paper studies the algorithmic stability and generalizability of decentralized stochastic gradient descent (D-SGD). We prove that the consensus model learned by D-SGD is $\math…
Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
Gongfan Fang, Yifan Bao, Jie Song +4
Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain. Prior KD approaches, despite their gratify…
Distribution Knowledge Embedding for Graph Pooling
Kaixuan Chen, Jie Song, Shunyu Liu +4
Graph-level representation learning is the pivotal step for downstream tasks that operate on the whole graph. The most common approach to this problem heretofore is graph pooling,…
SPAGAN: Shortest Path Graph Attention Network
Yiding Yang, Xinchao Wang, Mingli Song +2
Graph convolutional networks (GCN) have recently demonstrated their potential in analyzing non-grid structure data that can be represented as graphs. The core idea is to encode the…
Factorizable Graph Convolutional Networks
Yiding Yang, Zunlei Feng, Mingli Song +1
Graphs have been widely adopted to denote structural connections between entities. The relations are in many cases heterogeneous, but entangled together and denoted merely as a sin…
Data-Free Adversarial Distillation
Gongfan Fang, Jie Song, Chengchao Shen +3
Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer. However, almost all exist…