14 papers
Janus: Disaggregating Attention and Experts for Scalable MoE Inference
Zhexiang Zhang, Ye Wang, Yumiao Zhao +10
Serving large Mixture-of-Experts (MoE) models is challenging because of their large memory footprints, heterogeneous resource demands, and highly dynamic inference workloads. Most…
Understanding Dynamic Compute Allocation in Recurrent Transformers
Ibraheem Muhammad Moosa, Suhas Lohit, Ye Wang +2
Token-level adaptive computation seeks to reduce inference cost by allocating more computation to harder tokens and less to easier ones. However, prior work is primarily evaluated…
ERNIE 5.0 Technical Report
Haifeng Wang, Hua Wu, Tian Wu +432
In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio…
PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning
Langming Liu, Kangtao Lv, Haibin Chen +8
Large language models (LLMs), despite their powerful capabilities, suffer from factual hallucinations where they generate verifiable falsehoods. We identify a root of this issue: t…
TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
Dabiao Ma, Ziming Dai, Zhimin Xin +3
Current Parameter-Efficient Fine-Tuning (PEFT) methods typically operate under an implicit assumption: Once a target module is selected, every token passing through it contributes…
Comparative Study of Large Language Models on Chinese Film Script Continuation: An Empirical Analysis Based on GPT-5.2 and Qwen-Max
Yuxuan Cao, Zida Yang, Ye Wang
As large language models (LLMs) are increasingly applied to creative writing, their performance on culturally specific narrative tasks warrants systematic investigation. This study…