19 papers
OACap: Caption-Guided Active Observation for Detailed Video Captioning
Hao Shi, Hong Yan, Xiaoyang Zhang +3
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a cohere…
ZoomCap: Evidence-Routed Visual Zooming for Recoverable Visual Information Transfer
Hao Shi, Jiafeng Wang, Yuzhuo Fu +5
Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss small objects…
OmniTable: A Unified Wide-Table System for Petabyte-Scale LLM Data Curation and Exploration
Yuzhuo Fu, Xiangchun Wang, Chao Huang +16
Data curation is a critical bottleneck in industrial-grade LLM development, where petabyte-scale unstructured corpora are scattered across hundreds of physical tables, feature engi…
Tracking the Moving Frontier: Long-Short Term Advantage Estimator
Xinhao Yao, Lu Yu, Changhao Wang +5
Group-based RLVR methods estimate advantages by repeatedly sampling multiple trajectories for each prompt, making long-horizon agent training expensive and discarding useful experi…
Robust Code RL via Faulty-Code-Driven Test case Synthesis and Dense Reward Shaping
Yiwen Zhang, Xiaodong Yan, Zhenyu Huang +6
Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading t…
Improving Cross-Format Robustness in Language Models with Multi-Format Training
June M. Liu, Shaomian Zheng, He Cao +3
Large language models often remain sensitive to answer format: a question solved correctly in one form may fail in another semantically equivalent form. To study this gap, we defin…
What Really Improves Mathematical Reasoning: Structured Reasoning Signals Beyond Pure Code
Yuze Zhao, Junpeng Fang, Lu Yu +6
Code has become a standard component of modern foundation language model (LM) training, yet its role beyond programming remains unclear. We revisit the claim that code improves rea…
DiffScore: Text Evaluation Beyond Autoregressive Likelihood
Wen Lai, Yingli Shen, Dingnan Jin +4
Autoregressive language models are widely used for text evaluation, however, their left-to-right factorization introduces positional bias, i.e., early tokens are scored with only l…
On Representation Redundancy in Large-Scale Instruction Tuning Data Selection
Youwei Shu, Shaomian Zheng, Dingnan Jin +5
Data quality is a crucial factor in large language models training. While prior work has shown that models trained on smaller, high-quality datasets can outperform those trained on…
GRIP: Geometric Refinement and Adaptive Information Potential for Data Efficiency
Changhao Wang, Jiaolong Yang, Xinhao Yao +7
The performance of Large Language Models (LLMs) is increasingly governed by data efficiency rather than raw scaling volume. However, existing selection methods often decouple globa…
UniGeM: Unifying Data Mixing and Selection via Geometric Exploration and Mining
Changhao Wang, Yunfei Yu, Xinhao Yao +5
The scaling of Large Language Models (LLMs) is increasingly limited by data quality. Most methods handle data mixing and sample selection separately, which can break the structure…
Improving Autoformalization Using Direct Dependency Retrieval
Shaoqi Wang, Lu Yu, Siwei Lou +4
The convergence of deep learning and formal mathematics has spurred research in formal verification. Statement autoformalization, a crucial first step in this process, aims to tran…
Every Activation Boosted: Scaling General Reasoner to 1 Trillion Open Language Foundation
Ling Team, Ang Li, Ben Liu +138
We introduce Ling 2.0, a series reasoning-oriented language foundation built upon the principle that every activation boosts reasoning capability. Designed to scale from tens of bi…
The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
Xinhao Yao, Lu Yu, Xiaolin Hu +4
The ongoing debate on whether reinforcement learning with verifiable rewards (RLVR) expands or shrinks the reasoning capabilities of large language models (LLMs) remains unresolved…
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
Ling Team, Bin Hu, Cai Chen +43
We present Ring-lite, a Mixture-of-Experts (MoE)-based large language model optimized via reinforcement learning (RL) to achieve efficient and robust reasoning capabilities. Built…
SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learning
Xiong Jun Wu, Zhenduo Zhang, ZuJie Wen +11
Training large reasoning models (LRMs) with reinforcement learning in STEM domains is hindered by the scarcity of high-quality, diverse, and verifiable problem sets. Existing synth…
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
Codefuse, Ling Team, : +30
Recent advancements in code large language models (LLMs) have demonstrated remarkable capabilities in code generation and understanding. It is still challenging to build a code LLM…
MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models
Jiazheng Li, Lu Yu, Qing Cui +4
High-quality data plays a critical role in the pretraining and fine-tuning of large language models (LLMs), even determining their performance ceiling to some degree. Consequently,…
Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
Jinluan Yang, Dingnan Jin, Anke Tang +10
Achieving balanced alignment of large language models (LLMs) in terms of Helpfulness, Honesty, and Harmlessness (3H optimization) constitutes a cornerstone of responsible AI. Exist…