1 citations · 2 across the 30 of their papers we have counts for
32 papers
Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation
Xincheng Wei, Yifan Ding, Yoshua Li +6
On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard…
Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models
Xincheng Wei, Yifan Ding, Fucheng Xiong +6
Self-play supports the self-evolution of language models, but solver performance can plateau or decline across rounds without guidance. Existing unguided methods typically use diff…
RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges
Kaiyuan Liu, Ziyuan Zhuang, Rongxiang Weng +1
LLM-as-judge is essential for evaluating open-ended text and steering post-training, yet improving the judge itself typically relies on expensive annotations, reward models, or dis…
SAF-OPD: Stable Advantage Fusion for On-Policy Distillation
Yifan Ding, Xincheng Wei, Yoshua Y. Li +7
Reinforcement learning with verifiable rewards (RLVR) broadcasts a single response-level reward to every token, while on-policy distillation (OPD) scores each token against a stron…
ACE: Pluggable Adaptive Context Elasticizer across Agents
Ning Liao, Zihao Long, Xiaoxing Wang +6
The increasing complexity of agentic tasks has led to rapidly growing trajectory lengths, which poses significant challenges for large language model (LLM) based agents with fixed…
CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
Peiguang Li, Yongwei Zhou, Juncheng Diao +12
Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance. Ho…
HIPIF: Hierarchical Planning and Information Folding for Long-Horizon LLM Agent Learning
Juncheng Diao, Zhicong Lu, Peiguang Li +6
While Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents across a wide range of tasks, their performance often degrades in multi-turn long-hori…
Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training
Yongwei Zhou, Juncheng Diao, Junlin Shang +2
The efficacy of continued pre-training for Large Language Models (LLMs) hinges upon hyperparameter configurations, such as learning rate and batch size. However, current practices…
Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
Kaiyuan Liu, Ziyuan Zhuang, Yang Bai +3
On-policy distillation (OPD) trains a student model on its own rollouts using dense feedback from a stronger teacher. Prior literature suggests that, provided teacher feedback is a…
LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
Yuchun Fan, Bei Li, Peiguang Li +9
Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models (LLMs), yet its benefits have not fully translated to multilingual contexts.…
Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
Yang Bai, Kaiyuan Liu, Ziyuan Zhuang +5
Complex reinforcement learning environments frequently employ multi-task and mixed-reward formulations. In these settings, heterogeneous reward distributions and correlated reward…
LongCat-Flash-Thinking-2601 Technical Report
Meituan LongCat Team, Anchun Gui, Bei Li +162
We introduce LongCat-Flash-Thinking-2601, a 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model with superior agentic reasoning capability. LongCat-Flash-Thi…
Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
Zhihao Xu, Rumei Li, Jiahuan Li +4
Enabling Large Language Models (LLMs) to effectively utilize tools in multi-turn interactions is essential for building capable autonomous agents. However, acquiring diverse and re…
Efficient Context Scaling with LongCat ZigZag Attention
Chen Zhang, Yang Bai, Jiahuan Li +19
We introduce LongCat ZigZag Attention (LoZA), which is a sparse attention scheme designed to transform any existing full-attention models into sparse versions with rather limited c…
A Survey on LLM Mid-Training
Chengying Tu, Xuemiao Zhang, Rongxiang Weng +6
Recent advances in foundation models have highlighted the significant benefits of multi-stage training, with a particular emphasis on the emergence of mid-training as a vital stage…
OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
Han Li, Xinyu Peng, Yaoming Wang +7
We introduce OneCAT, a unified multimodal model that seamlessly integrates understanding, generation, and editing within a novel, pure decoder-only transformer architecture. Our fr…
AdaR: A Framework for Equipping LLMs with Adaptive Reasoning
Zhejian Lai, Xiang Geng, Zhijun Wang +7
Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures in robustness and generalization. This paper att…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…
Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
Xuemiao Zhang, Can Ren, Chengying Tu +5
Recent progress in large reasoning models for challenging mathematical reasoning has been driven by reinforcement learning (RL). Incorporating long chain-of-thought (CoT) data duri…
Introducing LongCat-Flash-Thinking: A Technical Report
Meituan LongCat Team, Anchun Gui, Bei Li +122
We present LongCat-Flash-Thinking, an efficient 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model. Its advanced capabilities are cultivated through a metic…
LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
Xuemiao Zhang, Can Ren, Chengying Tu +4
The advancement of large language models (LLMs) struggles with the scarcity of high-quality, diverse training data. To address this limitation, we propose LinkSyn, a novel knowledg…
Large-Scale Diverse Synthesis for Mid-Training
Xuemiao Zhang, Chengying Tu, Can Ren +4
The scarcity of high-quality, knowledge-intensive training data hinders the development of large language models (LLMs), as traditional corpora provide limited information. Previou…
Libra: Assessing and Improving Reward Model by Learning to Think
Meng Zhou, Bei Li, Jiahao Liu +5
Reinforcement learning (RL) has significantly improved the reasoning ability of large language models. However, current reward models underperform in challenging reasoning scenario…
Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
Jiazheng Zhang, Wenqing Jing, Zizhuo Zhang +12
Reward models (RMs) are central to aligning large language models (LLMs) with human values, yet noisy preference feedback can induce reward misgeneralization through spurious corre…