787 citations · 2.9k across the 66 of their papers we have counts for
22 papers · 1 filter
GraphMAE: Self-Supervised Masked Graph Autoencoders
Zhenyu Hou, Xiao Liu, Yukuo Cen +4
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fie…
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)
Yu Huang, Junyang Lin, Chang Zhou +2
Despite the remarkable success of deep multi-modal learning in practice, it has not been well-explained in theory. Recently, it has been observed that the best uni-modal network ou…
KNAS: Green Neural Architecture Search
Jingjing Xu, Liang Zhao, Junyang Lin +3
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations. Considering that these computat…
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
Jiangchao Yao, Shengyu Zhang, Yang Yao +15
Influenced by the great success of deep learning via cloud computing and the rapid development of edge chips, research in artificial intelligence (AI) has shifted to both of the co…
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Junyang Lin, An Yang, Jinze Bai +9
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 a…
Stable Prediction on Graphs with Agnostic Distribution Shift
Shengyu Zhang, Kun Kuang, Jiezhong Qiu +5
Graph is a flexible and effective tool to represent complex structures in practice and graph neural networks (GNNs) have been shown to be effective on various graph tasks with rand…