1 citations · 1 across the 14 of their papers we have counts for
16 papers
Which Long Documents Matter? A Controlled Study of Data Selection for Context Extension
Junlong Jia, Jiang Zhou, Xing Wu +3
Natural long documents are widely used for context extension, but which selection pipelines are effective remains unclear. We conduct a controlled comparison on Qwen2.5-3B at 128K…
AFCS: Atomic-Fact-Conditioned Synthesis for Factual Knowledge Pretraining
Yunhao Wang, Feng Zhang, Tinghao Yu +2
Synthetic data is increasingly used in language-model pretraining. Existing knowledge-corpus synthesis pipelines, including WRAP and Nemotron-CC, usually take a source passage or d…
Environment Evolution for Terminal Agents
Zhiyuan Fan, Tinghao Yu, Yuanjun Cai +9
Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesized from scratch become less…
LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails
Ziyang Chen, Xing Wu, Songlin Hu
Safety guardrails serve as the last line of defense against harmful inputs and outputs of large language models (LLMs), yet they are trained and evaluated almost exclusively on sho…
Deep Research Pretraining via Predictive Navigation
Jiang Zhou, Zhiyuan Fan, Xing Wu +3
Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We intr…
From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals
Yeyong Yu, Wenya Hu, Xing Wu +1
As candidate generation and high-throughput experimentation advance, the primary bottleneck in materials discovery is shifting from property prediction to making reliable evaluatio…
Toward Scalable Terminal Task Synthesis via Skill Graphs
Zhiyuan Fan, Tinghao Yu, Yuanjun Cai +8
Terminal agents have demonstrated strong potential for autonomous command-line execution, yet their training remains constrained by the scarcity of high-quality and diverse executi…
TOOLCAD: Exploring Tool-Using Large Language Models in Text-to-CAD Generation with Reinforcement Learning
Yifei Gong, Xing Wu, Wenda Liu +1
Computer-Aided Design (CAD) is an expert-level task that relies on long-horizon reasoning and coherent modeling actions. Large Language Models (LLMs) have shown remarkable advancem…
WRAP++: Web discoveRy Amplified Pretraining
Jiang Zhou, Yunhao Wang, Xing Wu +2
Synthetic data rephrasing has emerged as a powerful technique for enhancing knowledge acquisition during large language model (LLM) pretraining. However, existing approaches operat…
PolicyLong: Towards On-Policy Context Extension
Junlong Jia, Jiang Zhou, Ziyang Chen +5
Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependencies via information-theoretic ve…
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
Ziyang Chen, Xing Wu, Junlong Jia +4
The rapid expansion of context length in large language models (LLMs) has outpaced existing evaluation benchmarks. Current long-context benchmarks often trade off scalability and r…
LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
Junlong Jia, Xing Wu, Chaochen Gao +8
High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-b…
dots.llm1 Technical Report
Bi Huo, Bin Tu, Cheng Qin +24
Mixture of Experts (MoE) models have emerged as a promising paradigm for scaling language models efficiently by activating only a subset of parameters for each input token. In this…
Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge
Pengwei Tang, Xiaolin Hu, Yong Liu +4
Low-Rank Adaptation (LoRA) is the leading parameter-efficient fine-tuning method for Large Language Models (LLMs), but it still suffers from catastrophic forgetting. Recent work ha…
SedarEval: Automated Evaluation using Self-Adaptive Rubrics
Zhiyuan Fan, Weinong Wang, Xing Wu +1
The evaluation paradigm of LLM-as-judge gains popularity due to its significant reduction in human labor and time costs. This approach utilizes one or more large language models (L…
RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?
Haotian Xu, Xing Wu, Weinong Wang +11
Can scaling transform reasoning? In this work, we explore the untapped potential of scaling Long Chain-of-Thought (Long-CoT) data to 1000k samples, pioneering the development of a…