8 papers · 1 filter
PReM: Learning What to Preserve and When to Refresh for Context Compression
Bohan Yu, Lei Shen, Chenxi Zhou +5
Efficient long-context inference is not only about reducing memory cost, but also about keeping useful contextual evidence accessible as generation proceeds. However, existing comp…
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2
Recent multimodal large language models (MLLMs) have shown strong chain-of-thought (CoT) reasoning ability on vision-language tasks, but their direct deployment in real-world syste…
Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards
Yuanjie Lyu, Chengyu Wang, Lei Shen +2
Small LLMs often struggle to match the agentic capabilities of large, costly models. While reinforcement learning can help, progress has been limited by two structural bottlenecks:…
BARD: budget-aware reasoning distillation
Lujie Niu, Lei Shen, Yi Jiang +4
While long Chain-of-Thought (CoT) distillation effectively transfers reasoning capability to smaller language models, the reasoning process often remains redundant and computationa…
TDR: Task-Decoupled Retrieval with Fine-Grained LLM Feedback for In-Context Learning
Yifu Chen, Bingchen Huang, Zhiling Wang +4
In-context learning (ICL) has become a classic approach for enabling LLMs to handle various tasks based on a few input-output examples. The effectiveness of ICL heavily relies on t…
SEO: Stochastic Experience Optimization for Large Language Models
Jitao Xu, Hongyun Zhou, Lei Shen +3
Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvi…