107 citations · 254 across the 13 of their papers we have counts for
4 papers · 1 filter
OpenThoughts: Data Recipes for Reasoning Models
Etash Guha, Ryan Marten, Sedrick Keh +47
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoni…
Improving Training Stability for Multitask Ranking Models in Recommender Systems
Jiaxi Tang, Yoel Drori, Daryl Chang +6
Recommender systems play an important role in many content platforms. While most recommendation research is dedicated to designing better models to improve user experience, we foun…
Nonlinear Initialization Methods for Low-Rank Neural Networks
Kiran Vodrahalli, Rakesh Shivanna, Maheswaran Sathiamoorthy +2
We propose a novel low-rank initialization framework for training low-rank deep neural networks -- networks where the weight parameters are re-parameterized by products of two low-…
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery +5
The Mixture-of-Experts (MoE) architecture is showing promising results in improving parameter sharing in multi-task learning (MTL) and in scaling high-capacity neural networks. Sta…