From the 1 of 10 linked papers with an AI index.
10 papers
PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models
Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik +4
The paper presents PLUREL, a lightweight framework for generating synthetic multi-table relational databases, enabling the study of scaling laws in relational foundation models and…
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5
Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge disti…
Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
Tianyu Pang, Vignesh Kothapalli, Shenyang Deng +3
We study optimal learning-rate selection in two-layer and three-layer linear neural networks trained to learn linear target functions. In particular, we derive the exact closed-for…
To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
Vignesh Kothapalli, Ata Fatahibaarzi, Hamed Firooz +1
Chain-of-thought (CoT) prompting combined with few-shot in-context learning (ICL) has unlocked significant reasoning capabilities in large language models (LLMs). However, ICL with…
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song +17
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Al…
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation
Hamed Firooz, Maziar Sanjabi, Adrian Englhardt +20
Ranking and recommendation systems are the foundation for numerous online experiences, ranging from search results to personalized content delivery. These systems have evolved into…