20 papers
Distillation as Probability Transport: Routed On-Policy Distillation
Tianle Xia, Lingxiang Hu, Yiding Sun +6
On-policy distillation (OPD) transfers teacher knowledge on student-generated trajectories, but efficient sampled objectives reduce the teacher distribution to scalar credit on ind…
Teaching a Moving Student: Rethinking the Curriculum of On-Policy Distillation
Lingxiang Hu, Tianle Xia, Ming Xu +6
In on-policy distillation (OPD), the student determines which states receive teacher supervision. As its policy evolves, earlier response prefixes become less likely even though te…
When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents
Linfang Shang, Ming Xu, Yiding Sun +4
Self-evolving agents accumulate capability by distilling reusable skills from their execution trajectories, but we find this process is not monotonic: past a critical pool size, ne…
Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning
Leichao Dong, Dongxu Zhang, Yiding Sun +4
Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification,…
SPARK: Susceptibility-Guided Profiling and Steering of Latent Reasoning States in Large Language Models
Dongxu Zhang, Yiding Sun, Zihao Guo +5
Reasoning failures in large language models (LLMs) are usually evaluated from final answers, but a wrong answer does not reveal why the model failed. The same incorrect output may…
FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models
Yichen Guo, Kai Tang, Fenglai Lin +6
Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image. Recent…
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
Kai Tang, Jinhao You, Bohua Zhang +6
Large Vision-Language Models (LVLMs) have achieved remarkable progress in visual understanding tasks such as image captioning and visual question answering. However, they remain su…
GraSP: Graph-Structured Skill Compositions for LLM Agents
Tianle Xia, Lingxiang Hu, Yiding Sun +5
Skill ecosystems for LLM agents have matured rapidly, yet recent benchmarks show that providing agents with more skills does not monotonically improve performance -- focused sets o…
CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning
Qixian Huang, Hongqiang Lin, Tong Fu +5
Reasoning over tabular data is a crucial capability for tasks like question answering and fact verification, as it requires models to comprehend both free-form questions and semi-s…
PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference
Qirui Wang, Qi Guo, Yiding Sun +4
Personalized text-to-image generation lets users fine-tune diffusion models into repositories of concept-specific checkpoints, but serving these repositories efficiently is difficu…
Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA
Wenwei Li, Ming Xu, Tianle Xia +7
Industrial advertising question answering (QA) is a high-stakes task in which hallucinated content, particularly fabricated URLs, can lead to financial loss, compliance violations,…
Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training
Tianle Xia, Ming Xu, Lingxiang Hu +7
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-…
Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perception
Yiding Sun, Jihua Zhu, Haozhe Cheng +4
Point cloud video understanding is critical for robotics as it accurately encodes motion and scene interaction. We recognize that 4D datasets are far scarcer than 3D ones, which ha…
AD-Bench: A Real-World, Trajectory-Aware Advertising Analytics Benchmark for LLM Agents
Lingxiang Hu, Yiding Sun, Tianle Xia +6
While Large Language Model (LLM) agents have made remarkable progress on complex reasoning, evaluating them in real-world environments remains an open problem. Existing benchmarks…
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
Dongxu Zhang, Yiding Sun, Cheng Tan +4
While Chain-of-Thought (CoT) reasoning significantly enhances the performance of Multimodal Large Language Models (MLLMs), its autoregressive nature incurs prohibitive latency cons…
Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning
Dongxu Zhang, Yujun Wu, Yiding Sun +5
While Chain-of-Thought (CoT) prompting empowers Large Language Models (LLMs), ensuring reasoning reliability remains an open challenge. Contrary to the prevailing cascading failure…
FinS-Pilot: A Benchmark for Online Financial RAG System
Feng Wang, Yiding Sun, Jiaxin Mao +2
Large language models (LLMs) have demonstrated remarkable capabilities across various professional domains, with their performance typically evaluated through standardized benchmar…
BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline
Guosheng Dong, Da Pan, Yiding Sun +17
The general capabilities of Large Language Models (LLM) highly rely on the composition and selection on extensive pretraining datasets, treated as commercial secrets by several ins…
YuLan: An Open-source Large Language Model
Yutao Zhu, Kun Zhou, Kelong Mao +35
Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many o…
An Integrated Data Processing Framework for Pretraining Foundation Models
Yiding Sun, Feng Wang, Yutao Zhu +2
The ability of the foundation models heavily relies on large-scale, diverse, and high-quality pretraining data. In order to improve data quality, researchers and practitioners ofte…