5 papers
Beyond Shadow Weights: Quantization-Aware Training as Quantized-Endpoint Descent
Sheng-An Xu, Hanyang Li, Jianhao Ma +1
Quantization-aware training (QAT) updates a full-precision shadow weight but deploys the quantized endpoint . Existing explanations for QAT largely view…
Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
Hanyang Li, Jianhao Ma, Ying Cui
Post-training quantization (PTQ) converts a trained full-precision model into low-bit weights without task-level retraining, while quantization-aware training (QAT) incorporates qu…
Subgradient Regularization: A Descent-Oriented Subgradient Method for Nonsmooth Optimization
Hanyang Li, Ying Cui
In nonsmooth optimization, a negative subgradient is not necessarily a descent direction, making the design of convergent descent methods based on zeroth-order and first-order info…
Variational Theory and Algorithms for a Class of Asymptotically Approachable Nonconvex Problems
Hanyang Li, Ying Cui
We investigate a class of composite nonconvex functions, where the outer function is the sum of univariate extended-real-valued convex functions and the inner function is the limit…
A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse
Hanyang Li, Ying Cui
In this paper, we have studied a decomposition method for solving a class of nonconvex two-stage stochastic programs, where both the objective and constraints of the second-stage p…