4 papers
YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
PSBC LLM Team, Huawei LLM Team, Ruihan Long +56
Large language models (LLMs) drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates inf…
SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
Guoxuan Chen, Han Shi, Jiawei Li +7
Large Language Models (LLMs) have exhibited exceptional performance across a spectrum of natural language processing tasks. However, their substantial sizes pose considerable chall…
Self-Adjust Softmax
Chuanyang Zheng, Yihang Gao, Guoxuan Chen +7
The softmax function is crucial in Transformer attention, which normalizes each row of the attention scores with summation to one, achieving superior performances over other altern…
Scaling Law for Language Models Training Considering Batch Size
Xian Shuai, Yiding Wang, Yimeng Wu +2
Large language models (LLMs) have made remarkable advances in recent years, with scaling laws playing a critical role in this rapid progress. In this paper, we empirically investig…