6 papers
YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
PSBC LLM Team, Huawei LLM Team, Ruihan Long +56
Large language models (LLMs) drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates inf…
Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining
Jianwei Li, Yijun Dong, Qi Lei
To remove redundant components of large language models (LLMs) without incurring significant computational costs, this work focuses on single-shot pruning without a retraining phas…
FP8-BERT: Post-Training Quantization for Transformer
Jianwei Li, Tianchi Zhang, Ian En-Hsu Yen +1
Transformer-based models, such as BERT, have been widely applied in a wide range of natural language processing tasks. However, one inevitable side effect is that they require mass…
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning
Jianwei Li, Sheng Liu, Qi Lei
Language models trained via federated learning (FL) demonstrate impressive capabilities in handling complex tasks while protecting user privacy. Recent studies indicate that levera…
Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
Jianwei Li, Qi Lei, Wei Cheng +1
The pruning objective has recently extended beyond accuracy and sparsity to robustness in language models. Despite this, existing methods struggle to enhance robustness against adv…
Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection
Jianwei Li, Weizhi Gao, Qi Lei +1
It is widely acknowledged that large and sparse models have higher accuracy than small and dense models under the same model size constraints. This motivates us to train a large mo…