5 papers
ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning
Xiaohan Qin, Xiaoxing Wang, Ning Liao +5
Data quality plays a critical role in enhancing supervised fine-tuning (SFT) for large language models (LLMs), and token-level data selection has emerged as a promising direction f…
Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
Shaobo Wang, Cong Wang, Wenjie Fu +11
As the demand for comprehensive evaluations of diverse model capabilities steadily increases, benchmark suites have correspondingly grown significantly in scale. Despite notable ad…
KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches
Mingquan Feng, Yifan Fu, Tongcheng Zhang +3
Despite the widely recognized success of residual connections in modern neural networks, their design principles remain largely heuristic. This paper introduces KITINet (Kinetics T…
KO: Kinetics-inspired Neural Optimizer with PDE Simulation Approaches
Mingquan Feng, Yixin Huang, Yifan Fu +2
The design of optimization algorithms for neural networks remains a critical challenge, with most existing methods relying on heuristic adaptations of gradient-based approaches. Th…
Optimal Control Operator Perspective and a Neural Adaptive Spectral Method
Mingquan Feng, Zhijie Chen, Yixin Huang +2
Optimal control problems (OCPs) involve finding a control function for a dynamical system such that a cost functional is optimized. It is central to physical systems in both academ…