5 papers
VALG: An Agentic System for ML Theory Research
Dechen Zhang, Xuan Tang, Xinxiang Yin +3
Machine learning theory studies learning procedures through mathematical setups in which the data model, training protocol, oracle access, loss, metric, and randomness define the p…
Divergence Decoding: Training-Free Capability Fusion
Yimi Wang, Hao Li, Shuo Yang +6
While large language models excel in reasoning, these generalists often lack knowledge for specialized scientific domains. Conversely, domain models~(specialists), while knowledgea…
Scaling Laws for Precision in High-Dimensional Linear Regression
Dechen Zhang, Xuan Tang, Yingyu Liang +1
Low-precision training is critical for optimizing the trade-off between model quality and training costs, necessitating the joint allocation of model size, dataset size, and numeri…
Kernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning
Dechen Zhang, Zhenmei Shi, Yi Zhang +2
Kernel ridge regression (KRR) is a foundational tool in machine learning, with recent work emphasizing its connections to neural networks. However, existing theory primarily addres…
Learning under Quantization for High-Dimensional Linear Regression
Dechen Zhang, Junwei Su, Difan Zou
The use of low-bit quantization has emerged as an indispensable technique for enabling the efficient training of large-scale models. Despite its widespread empirical success, a rig…