7 papers
Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
Tianyu Pang, Vignesh Kothapalli, Shenyang Deng +3
We study optimal learning-rate selection in two-layer and three-layer linear neural networks trained to learn linear target functions. In particular, we derive the exact closed-for…
Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures
Haohui Wang, Jingyuan Qi, Jianpeng Chen +9
The rapid progress of large language models (LLMs) is fueled by the growing reliance on datasets that blend real and synthetic data. While synthetic data offers scalability and cos…
HeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relations
Shuaicheng Zhang, Haohui Wang, Junhong Lin +5
Graph heterophily, where connected nodes have different labels, has attracted significant interest recently. Most existing works adopt a simplified approach - using low-pass filter…
EVINET: Towards Open-World Graph Learning via Evidential Reasoning Network
Weijie Guan, Haohui Wang, Jian Kang +2
Graph learning has been crucial to many real-world tasks, but they are often studied with a closed-world assumption, with all possible labels of data known a priori. To enable effe…
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection
Xinyue Zeng, Haohui Wang, Junhong Lin +3
The proliferation of open-sourced Large Language Models (LLMs) and diverse downstream tasks necessitates efficient model selection, given the impracticality of fine-tuning all cand…
MetamatBench: Integrating Heterogeneous Data, Computational Tools, and Visual Interface for Metamaterial Discovery
Jianpeng Chen, Wangzhi Zhan, Haohui Wang +10
Metamaterials, engineered materials with architected structures across multiple length scales, offer unprecedented and tunable mechanical properties that surpass those of conventio…