193 citations · 202 across the 5 of their papers we have counts for
3 papers · 1 filter
Process Reward Agents for Steering Knowledge-Intensive Reasoning
Jiwoong Sohn, Tomasz Sternal, Kenneth Styppa +2
Reasoning in knowledge-intensive domains remains challenging as intermediate steps are often not locally verifiable: unlike math or code, evaluating step correctness may require sy…
When Data Is Scarce: Scaling Sparse Language Models with Repeated Training
Boqian Wu, Qiao Xiao, Patrik Okanovic +6
Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not. In this work, we study sparse training in data-constrained r…
Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling
Qiao Xiao, Boqian Wu, Patrik Okanovic +6
Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model…