4 papers · 1 filter
On Representation Redundancy in Large-Scale Instruction Tuning Data Selection
Youwei Shu, Shaomian Zheng, Dingnan Jin +5
Data quality is a crucial factor in large language models training. While prior work has shown that models trained on smaller, high-quality datasets can outperform those trained on…
UniGeM: Unifying Data Mixing and Selection via Geometric Exploration and Mining
Changhao Wang, Yunfei Yu, Xinhao Yao +5
The scaling of Large Language Models (LLMs) is increasingly limited by data quality. Most methods handle data mixing and sample selection separately, which can break the structure…
The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
Xinhao Yao, Lu Yu, Xiaolin Hu +4
The ongoing debate on whether reinforcement learning with verifiable rewards (RLVR) expands or shrinks the reasoning capabilities of large language models (LLMs) remains unresolved…
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
Codefuse, Ling Team, : +30
Recent advancements in code large language models (LLMs) have demonstrated remarkable capabilities in code generation and understanding. It is still challenging to build a code LLM…