4 papers
ERNIE 5.0 Technical Report
Haifeng Wang, Hua Wu, Tian Wu +432
In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio…
AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedback
Zhitao Gao, Jie Ma, Xuhong Li +5
Large Language Models (LLMs) have achieved significant success in complex reasoning but remain bottlenecked by reliance on expert-annotated data and external verifiers. While exist…
From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning
Haoyu Li, Xuhong Li, Yiming Dong +1
Dataset diversity plays a pivotal role for the successful training of many machine learning models, particularly in the supervised fine-tuning (SFT) stage of large language model (…
Tokenization Falling Short: On Subword Robustness in Large Language Models
Yekun Chai, Yewei Fang, Qiwei Peng +1
Language models typically tokenize raw text into sequences of subword identifiers from a predefined vocabulary, a process inherently sensitive to typographical errors, length varia…