252 citations · 286 across the 17 of their papers we have counts for
17 papers
Statistical Priors for Implicit Preferences: Decoupling Skill Selection as a Local Harness in Personal Agents
Zeyu Gan, Huayi Tang, Yong Liu
As Large Language Model (LLM) capabilities advance, locally deployed personal agents relying on API-based remote models and external skills have emerged as a novel paradigm. With t…
Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry
Ye Su, Huayi Tang, Zixuan Gong +1
While Mixture-of-Experts (MoE) architectures define the state-of-the-art, their theoretical success is often attributed to heuristic efficiency rather than geometric expressivity.…
Effective Frontiers: A Unification of Neural Scaling Laws
Jiaxuan Zou, Zixuan Gong, Ye Su +2
Neural scaling laws govern the prediction power-law improvement of test loss with respect to model capacity (), datasize (), and compute (). However, existing theoretical…
Beyond the Black Box: A Survey on the Theory and Mechanism of Large Language Models
Zeyu Gan, Ruifeng Ren, Wei Yao +9
The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence, delivering monumental engineering successes that increasi…
Transformers as Intrinsic Optimizers: Forward Inference through the Energy Principle
Ruifeng Ren, Sheng Ouyang, Huayi Tang +1
Attention-based Transformers have demonstrated strong adaptability across a wide range of tasks and have become the backbone of modern Large Language Models (LLMs). However, their…
Active Semantic Perception
Huayi Tang, Pratik Chaudhari
We develop an approach for active semantic perception, which refers to using the semantics of the scene for tasks such as exploration. We build a compact, multi-layer scene graph t…
PAC-Bayesian Generalization Bounds for Graph Convolutional Networks on Inductive Node Classification
Huayi Tang, Yong Liu
Graph neural networks (GNNs) have achieved remarkable success in processing graph-structured data across various applications. A critical aspect of real-world graphs is their dynam…
On Weak-to-Strong Generalization and f-Divergence
Wei Yao, Gengze Xu, Huayi Tang +4
Weak-to-strong generalization (W2SG) has emerged as a promising paradigm for stimulating the capabilities of strong pre-trained models by leveraging supervision from weaker supervi…
Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
Zixuan Gong, Xiaolin Hu, Huayi Tang +1
Large language models (LLMs) have demonstrated remarkable in-context learning (ICL) abilities. However, existing theoretical analysis of ICL primarily exhibits two limitations: (a)…
Understanding Model Ensemble in Transferable Adversarial Attack
Wei Yao, Zeliang Zhang, Huayi Tang +1
Model ensemble adversarial attack has become a powerful method for generating transferable adversarial examples that can target even unknown models, but its theoretical foundation…
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications
Huayi Tang, Yong Liu
In this paper, we establish generalization bounds for transductive learning algorithms in the context of information theory and PAC-Bayes, covering both the random sampling and the…
Perfect Alignment May be Poisonous to Graph Contrastive Learning
Jingyu Liu, Huayi Tang, Yong Liu
Graph Contrastive Learning (GCL) aims to learn node representations by aligning positive pairs and separating negative ones. However, few of researchers have focused on the inner l…
Can Large Language Models Empower Molecular Property Prediction?
Chen Qian, Huayi Tang, Zhirui Yang +2
Molecular property prediction has gained significant attention due to its transformative potential in multiple scientific disciplines. Conventionally, a molecule graph can be repre…
Towards Understanding the Generalization of Graph Neural Networks
Huayi Tang, Yong Liu
Graph neural networks (GNNs) are the most widely adopted model in graph-structured data oriented learning and representation. Despite their extraordinary success in real-world appl…
Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering
Jie Xu, Yazhou Ren, Huayi Tang +4
Multi-view clustering, a long-standing and important research problem, focuses on mining complementary information from diverse views. However, existing works often fuse multiple v…
Multi-level Feature Learning for Contrastive Multi-view Clustering
Jie Xu, Huayi Tang, Yazhou Ren +3
Multi-view clustering can explore common semantics from multiple views and has attracted increasing attention. However, existing works punish multiple objectives in the same featur…
Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering
Jie Xu, Yazhou Ren, Huayi Tang +6
Multi-view clustering is an important research topic due to its capability to utilize complementary information from multiple views. However, there are few methods to consider the…