5 papers
A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
Qingyu Song, Rui Liu, Wei Lin +11
Deploying Large Language Models (LLMs) on edge devices enhances privacy but faces performance hurdles due to limited resources. We introduce a systematic methodology to evaluate on…
The Multi-Query Paradox in Zeroth-Order Optimization
Wei Lin, Qingyu Song, Hong Xu
Zeroth-order (ZO) optimization provides a powerful framework for problems where explicit gradients are unavailable and have to be approximated using only queries to function value.…
Learning Provably Improves the Convergence of Gradient Descent
Qingyu Song, Wei Lin, Hong Xu
Learn to Optimize (L2O) trains deep neural network-based solvers for optimization, achieving success in accelerating convex problems and improving non-convex solutions. However, L2…
Towards Robust Learning to Optimize with Theoretical Guarantees
Qingyu Song, Wei Lin, Juncheng Wang +1
Learning to optimize (L2O) is an emerging technique to solve mathematical optimization problems with learning-based methods. Although with great success in many real-world scenario…
A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methods
Wei Lin, Qingyu Song, Hong Xu
Tuning step sizes is crucial for the stability and efficiency of optimization algorithms. While adaptive coordinate-wise step sizes have been shown to outperform scalar step size i…