5 papers · 1 filter
Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework
Dong Wang, Yang Li, Ansong Ni +12
Synthetic data has become increasingly important for training large language models, especially when real data is scarce, expensive, or privacy-sensitive. Many such generation task…
NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks
Yang Li, Youssef Emad, Karthik Padthe +8
Recent work has shown that distilling reasoning traces from a larger teacher model via supervised finetuning outperforms reinforcement learning with the smaller student model alone…
NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions
Weizhe Yuan, Jane Yu, Song Jiang +8
Scaling reasoning capabilities beyond traditional domains such as math and coding is hindered by the lack of diverse and high-quality questions. To overcome this limitation, we int…
Improving Factuality with Explicit Working Memory
Mingda Chen, Yang Li, Karthik Padthe +5
Large language models can generate factually inaccurate content, a problem known as hallucination. Recent works have built upon retrieved-augmented generation to improve factuality…
Text Quality-Based Pruning for Efficient Training of Language Models
Vasu Sharma, Karthik Padthe, Newsha Ardalani +8
In recent times training Language Models (LMs) have relied on computationally heavy training over massive datasets which makes this training process extremely laborious. In this pa…