411 citations · 887 across the 22 of their papers we have counts for
7 papers · 1 filter
A Speed Odyssey for Deployable Quantization of LLMs
Qingyuan Li, Ran Meng, Yiduo Li +6
The large language model era urges faster and less costly inference. Prior model compression works on LLMs tend to undertake a software-centric approach primarily focused on the si…
Robust MAE-Driven NAS: From Mask Reconstruction to Architecture Innovation
Yiming Hu, Xiangxiang Chu, Yong Wang
Neural Architecture Search (NAS) relies heavily on labeled data, which is labor-intensive and time-consuming to obtain. In this paper, we propose a novel NAS method based on an uns…
Norm Tweaking: High-performance Low-bit Quantization of Large Language Models
Liang Li, Qingyuan Li, Bo Zhang +1
As the size of large language models (LLMs) continues to grow, model compression without sacrificing accuracy has become a crucial challenge for deployment. While some quantization…
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation
Xiaoxing Wang, Xiangxiang Chu, Yuda Fan +4
Albeit being a prevalent architecture searching approach, differentiable architecture search (DARTS) is largely hindered by its substantial memory cost since the entire supernet re…
DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
Xiangxiang Chu, Xiaoxing Wang, Bo Zhang +3
Despite the fast development of differentiable architecture search (DARTS), it suffers from long-standing performance instability, which extremely limits its application. Existing…
MoGA: Searching Beyond MobileNetV3
Xiangxiang Chu, Bo Zhang, Ruijun Xu
The evolution of MobileNets has laid a solid foundation for neural network applications on mobile end. With the latest MobileNetV3, neural architecture search again claimed its sup…
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search
Xiangxiang Chu, Bo Zhang, Ruijun Xu
One of the most critical problems in weight-sharing neural architecture search is the evaluation of candidate models within a predefined search space. In practice, a one-shot super…