23 citations · 60 across the 16 of their papers we have counts for
7 papers · 1 filter
Harnessing the Potential of Optimizing Data Mixtures via Bayesian Domain Reweighting
Xiang Yuan, Kaiqing Lei, Zhenyu Jin +3
The performance of Large Language Models (LLMs) is fundamentally influenced by the distributional composition of multi-domain pre-training data. While manual heuristics were preval…
BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning
Zhibin Duan, Yuhong Wang, Jiahong Fu +3
While Low-rank adaptation (LoRA) enables highly efficient fine-tuning by constraining task-specific updates to fixed low-rank subspaces, this rigid design limits representational f…
A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws
Jun Shu, Junxiong Jia, Deyu Meng +1
Emergent intelligence have played a major role in the modern AI development. While existing studies primarily rely on empirical observations to characterize this phenomenon, a rigo…
KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra
Liangyu Su, Jun Shu, Rui Liu +2
Representing and predicting high-dimensional and spatiotemporally chaotic dynamical systems remains a fundamental challenge in dynamical systems and machine learning. Although data…
Feed Two Birds with One Scone: Exploiting Function-Space Regularization for Both OOD Robustness and ID Fine-Tuning Performance
Xiang Yuan, Jun Shu, Deyu meng +1
Robust fine-tuning aims to achieve competitive in-distribution (ID) performance while maintaining the out-of-distribution (OOD) robustness of a pre-trained model when transferring…
Improving Memory Efficiency for Training KANs via Meta Learning
Zhangchi Zhao, Jun Shu, Deyu Meng +1
Inspired by the Kolmogorov-Arnold representation theorem, KANs offer a novel framework for function approximation by replacing traditional neural network weights with learnable uni…
Meta Transition Adaptation for Robust Deep Learning with Noisy Labels
Jun Shu, Qian Zhao, Zongben Xu +1
To discover intrinsic inter-class transition probabilities underlying data, learning with noise transition has become an important approach for robust deep learning on corrupted la…