11 papers
PReM: Learning What to Preserve and When to Refresh for Context Compression
Bohan Yu, Lei Shen, Chenxi Zhou +5
Efficient long-context inference is not only about reducing memory cost, but also about keeping useful contextual evidence accessible as generation proceeds. However, existing comp…
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2
Recent multimodal large language models (MLLMs) have shown strong chain-of-thought (CoT) reasoning ability on vision-language tasks, but their direct deployment in real-world syste…
Rewards as Labels: Revisiting RLVR from a Classification Perspective
Zepeng Zhai, Meilin Chen, Jiaxuan Zhao +3
Reinforcement Learning with Verifiable Rewards has recently advanced the capabilities of Large Language Models in complex reasoning tasks by providing explicit rule-based supervisi…
LEMON: How Well Do MLLMs Perform Temporal Multimodal Understanding on Instructional Videos?
Zhuang Yu, Lei Shen, Jing Zhao +1
Recent multimodal large language models (MLLMs) have shown remarkable progress across vision, audio, and language tasks, yet their performance on long-form, knowledge-intensive, an…
Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards
Yuanjie Lyu, Chengyu Wang, Lei Shen +2
Small LLMs often struggle to match the agentic capabilities of large, costly models. While reinforcement learning can help, progress has been limited by two structural bottlenecks:…
BARD: budget-aware reasoning distillation
Lujie Niu, Lei Shen, Yi Jiang +4
While long Chain-of-Thought (CoT) distillation effectively transfers reasoning capability to smaller language models, the reasoning process often remains redundant and computationa…
QAgent: A modular Search Agent with Interactive Query Understanding
Yi Jiang, Lei Shen, Lujie Niu +3
Large language models (LLMs) excel at natural language tasks but are limited by their static parametric knowledge, especially in knowledge-intensive task. Retrieval-augmented gener…
TDR: Task-Decoupled Retrieval with Fine-Grained LLM Feedback for In-Context Learning
Yifu Chen, Bingchen Huang, Zhiling Wang +4
In-context learning (ICL) has become a classic approach for enabling LLMs to handle various tasks based on a few input-output examples. The effectiveness of ICL heavily relies on t…
SEO: Stochastic Experience Optimization for Large Language Models
Jitao Xu, Hongyun Zhou, Lei Shen +3
Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvi…
GEB-1.3B: Open Lightweight Large Language Model
Jie Wu, Yufeng Zhu, Lei Shen +1
Recently developed large language models (LLMs) such as ChatGPT, Claude, and Llama have demonstrated impressive abilities, and even surpass human-level performance in several tasks…
A Group Fairness Lens for Large Language Models
Guanqun Bi, Yuqiang Xie, Lei Shen +1
The need to assess LLMs for bias and fairness is critical, with current evaluations often being narrow, missing a broad categorical view. In this paper, we propose evaluating the b…