252 citations · 286 across the 17 of their papers we have counts for
12 papers · 1 filter
Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry
Ye Su, Huayi Tang, Zixuan Gong +1
While Mixture-of-Experts (MoE) architectures define the state-of-the-art, their theoretical success is often attributed to heuristic efficiency rather than geometric expressivity.…
Effective Frontiers: A Unification of Neural Scaling Laws
Jiaxuan Zou, Zixuan Gong, Ye Su +2
Neural scaling laws govern the prediction power-law improvement of test loss with respect to model capacity (), datasize (), and compute (). However, existing theoretical…
Transformers as Intrinsic Optimizers: Forward Inference through the Energy Principle
Ruifeng Ren, Sheng Ouyang, Huayi Tang +1
Attention-based Transformers have demonstrated strong adaptability across a wide range of tasks and have become the backbone of modern Large Language Models (LLMs). However, their…
PAC-Bayesian Generalization Bounds for Graph Convolutional Networks on Inductive Node Classification
Huayi Tang, Yong Liu
Graph neural networks (GNNs) have achieved remarkable success in processing graph-structured data across various applications. A critical aspect of real-world graphs is their dynam…
On Weak-to-Strong Generalization and f-Divergence
Wei Yao, Gengze Xu, Huayi Tang +4
Weak-to-strong generalization (W2SG) has emerged as a promising paradigm for stimulating the capabilities of strong pre-trained models by leveraging supervision from weaker supervi…
Understanding Model Ensemble in Transferable Adversarial Attack
Wei Yao, Zeliang Zhang, Huayi Tang +1
Model ensemble adversarial attack has become a powerful method for generating transferable adversarial examples that can target even unknown models, but its theoretical foundation…
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications
Huayi Tang, Yong Liu
In this paper, we establish generalization bounds for transductive learning algorithms in the context of information theory and PAC-Bayes, covering both the random sampling and the…
Perfect Alignment May be Poisonous to Graph Contrastive Learning
Jingyu Liu, Huayi Tang, Yong Liu
Graph Contrastive Learning (GCL) aims to learn node representations by aligning positive pairs and separating negative ones. However, few of researchers have focused on the inner l…
Can Large Language Models Empower Molecular Property Prediction?
Chen Qian, Huayi Tang, Zhirui Yang +2
Molecular property prediction has gained significant attention due to its transformative potential in multiple scientific disciplines. Conventionally, a molecule graph can be repre…
Towards Understanding the Generalization of Graph Neural Networks
Huayi Tang, Yong Liu
Graph neural networks (GNNs) are the most widely adopted model in graph-structured data oriented learning and representation. Despite their extraordinary success in real-world appl…
Multi-level Feature Learning for Contrastive Multi-view Clustering
Jie Xu, Huayi Tang, Yazhou Ren +3
Multi-view clustering can explore common semantics from multiple views and has attracted increasing attention. However, existing works punish multiple objectives in the same featur…
Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering
Jie Xu, Yazhou Ren, Huayi Tang +6
Multi-view clustering is an important research topic due to its capability to utilize complementary information from multiple views. However, there are few methods to consider the…