8 papers · 1 filter
ArborMem: Navigating Interaction States with Memory Forests
Zongwei Lv, Yuemeng Xu, Yilun Yao +8
Large language models increasingly serve as persistent conversational assistants, requiring memory that preserves relevant experience and maintains continuity across interactions.…
RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessions
Zongwei Lv, Yaoming Li, Zhewen Tan +7
Agent benchmarks should reflect what users actually ask deployed agents to do, yet existing benchmarks often miss key realism properties of real developer-agent sessions. We introd…
A Primer in Post-Training Reasoning Data: What We Know About How It Works
Yaoming Li, Guangxiang Zhao, Qilong Shi +3
Post-training has become a primary driver of recent progress in large reasoning models, and reasoning data are often the key variable determining whether this stage succeeds. Work…
Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
Junfeng Ran, Guangxiang Zhao, Yuhan Wu +7
The Mixture-of-Experts (MoE) models have gained significant attention in deep learning due to their dynamic resource allocation and superior performance across diverse tasks. Howev…
TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation
Lin Sun, Guangxiang Zhao, Xiaoqi Jian +18
The challenge of reducing the size of Large Language Models (LLMs) while maintaining their performance has gained significant attention. However, existing methods, such as model di…
Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision
Dawei Zhu, Xiyu Wei, Guangxiang Zhao +7
Recent advances in Large Language Models (LLMs) have highlighted the challenge of handling long-context tasks, where models need to reason over extensive input contexts to aggregat…
LongAttn: Selecting Long-context Training Data via Token-level Attention
Longyun Wu, Dawei Zhu, Guangxiang Zhao +5
With the development of large language models (LLMs), there has been an increasing need for significant advancements in handling long contexts. To enhance long-context capabilities…
Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacements
Guangxiang Zhao, Saier Hu, Xiaoqi Jian +5
In this paper, we propose a ``Generalization Stress Test" to assess Large Language Models' (LLMs) generalization ability under slight and controlled perturbations, including option…