111 citations · 352 across the 17 of their papers we have counts for
18 papers
The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesis
Jinyang Zhang, Weibin Liao, Keqin Bao +10
Large Language Models (LLMs) excel at programming tasks but frequently fail at deterministic, fine-grained reasoning in natural language, relying heavily on semantic approximations…
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
Zihan Qiu, Zekun Wang, Xiao Li +33
We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and additional 51B parameters of n…
Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?
Qingjie Zhang, Xingzhang Ren, Zixuan Chen +6
Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus mixtures or traced specific…
The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
Shaobo Wang, Guo Chen, Ziyue Wang +5
With the rapid progress of large language models (LLMs), reliably evaluating the capabilities of pre-trained LLMs has become increasingly important. The challenge is that base pre-…
DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
Shaobo Wang, Yujie Chen, Yafeng Sun +9
Modern reasoning agents are increasingly evaluated on their ability to generate multiple valid solution paths, plans, or tool-use traces for a given input. Standard reward-maximizi…
OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
Shaobo Wang, Xuan Ouyang, Tianyi Xu +9
As high-quality public text approaches exhaustion, a phenomenon known as the Data Wall, pre-training is shifting from more tokens to better tokens. However, existing methods either…
Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
Shaobo Wang, Cong Wang, Wenjie Fu +11
As the demand for comprehensive evaluations of diverse model capabilities steadily increases, benchmark suites have correspondingly grown significantly in scale. Despite notable ad…
Qwen3-Omni Technical Report
Jin Xu, Zhifang Guo, Hangrui Hu +35
We present Qwen3-Omni, a single multimodal model that, for the first time, maintains state-of-the-art performance across text, image, audio, and video without any degradation relat…
RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
Baolong Bi, Shenghua Liu, Xingzhang Ren +7
The foundational capabilities of large language models (LLMs) are deeply influenced by the quality of their pre-training corpora. However, enhancing data quality at scale remains a…
Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
Yufa Zhou, Shaobo Wang, Xingyu Dong +7
Directly training Large Language Models (LLMs) for Multi-Agent Systems (MAS) remains challenging due to intricate reward modeling, dynamic agent interactions, and demanding general…
WorldPM: Scaling Human Preference Modeling
Binghai Wang, Runji Lin, Keming Lu +17
Motivated by scaling laws in language modeling that demonstrate how test loss scales as a power law with model and dataset sizes, we find that similar laws exist in preference mode…
Qwen3 Technical Report
An Yang, Anfeng Li, Baosong Yang +57
In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, a…
Qwen2.5-1M Technical Report
An Yang, Bowen Yu, Chengyuan Li +25
We introduce Qwen2.5-1M, a series of models that extend the context length to 1 million tokens. Compared to the previous 128K version, the Qwen2.5-1M series have significantly enha…
Qwen2.5 Technical Report
Qwen, :, An Yang +41
In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been sign…
Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
An Yang, Beichen Zhang, Binyuan Hui +13
In this report, we present a series of math-specific large language models: Qwen2.5-Math and Qwen2.5-Math-Instruct-1.5B/7B/72B. The core innovation of the Qwen2.5 series lies in in…
Qwen2.5-Coder Technical Report
Binyuan Hui, Jian Yang, Zeyu Cui +21
In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/…
Qwen2 Technical Report
An Yang, Baosong Yang, Binyuan Hui +59
This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models. We release a comprehensive suite of foundational and instruct…
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
Yuxin Tian, Mouxing Yang, Yunfan Li +4
Recent studies applied Parameter Efficient Fine-Tuning techniques (PEFTs) to efficiently narrow the performance gap between pre-training and downstream. There are two important fac…