1 citations · 2 across the 13 of their papers we have counts for
13 papers
CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
Peiguang Li, Yongwei Zhou, Juncheng Diao +12
Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance. Ho…
HIPIF: Hierarchical Planning and Information Folding for Long-Horizon LLM Agent Learning
Juncheng Diao, Zhicong Lu, Peiguang Li +6
While Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents across a wide range of tasks, their performance often degrades in multi-turn long-hori…
Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training
Yongwei Zhou, Juncheng Diao, Junlin Shang +2
The efficacy of continued pre-training for Large Language Models (LLMs) hinges upon hyperparameter configurations, such as learning rate and batch size. However, current practices…
LongCat-Flash-Thinking-2601 Technical Report
Meituan LongCat Team, Anchun Gui, Bei Li +162
We introduce LongCat-Flash-Thinking-2601, a 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model with superior agentic reasoning capability. LongCat-Flash-Thi…
LongCat-Flash Technical Report
Meituan LongCat Team, Bayan, Bei Li +179
We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming f…
Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
Xuemiao Zhang, Liangyu Xu, Feiyu Duan +5
Large language models (LLMs) generally utilize a consistent data distribution throughout the pretraining process. However, as the model's capability improves, it is intuitive that…
FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy
Xuemiao Zhang, Feiyu Duan, Liangyu Xu +5
Large language models (LLMs) have significantly advanced human language understanding and generation, with pretraining data quality and organization being crucial to their performa…
Dual Instruction Tuning with Large Language Models for Mathematical Reasoning
Yongwei Zhou, Tiejun Zhao
Recent advancements highlight the success of instruction tuning with large language models (LLMs) utilizing Chain-of-Thought (CoT) data for mathematical reasoning tasks. Despite th…
HopPG: Self-Iterative Program Generation for Multi-Hop Question Answering over Heterogeneous Knowledge
Yingyao Wang, Yongwei Zhou, Chaoqun Duan +2
The semantic parsing-based method is an important research branch for knowledge-based question answering. It usually generates executable programs lean upon the question and then c…
UniRPG: Unified Discrete Reasoning over Table and Text as Program Generation
Yongwei Zhou, Junwei Bao, Chaoqun Duan +3
Question answering requiring discrete reasoning, e.g., arithmetic computing, comparison, and counting, over knowledge is a challenging task. In this paper, we propose UniRPG, a sem…
OPERA:Operation-Pivoted Discrete Reasoning over Text
Yongwei Zhou, Junwei Bao, Chaoqun Duan +7
Machine reading comprehension (MRC) that requires discrete reasoning involving symbolic operations, e.g., addition, sorting, and counting, is a challenging task. According to this…
RoR: Read-over-Read for Long Document Machine Reading Comprehension
Jing Zhao, Junwei Bao, Yifan Wang +4
Transformer-based pre-trained models, such as BERT, have achieved remarkable results on machine reading comprehension. However, due to the constraint of encoding length (e.g., 512…
EviDR: Evidence-Emphasized Discrete Reasoning for Reasoning Machine Reading Comprehension
Yongwei Zhou, Junwei Bao, Haipeng Sun +5
Reasoning machine reading comprehension (R-MRC) aims to answer complex questions that require discrete reasoning based on text. To support discrete reasoning, evidence, typically t…