22 citations · 47 across the 13 of their papers we have counts for
4 papers · 1 filter
Memorilla: Latent Semantic Memory for LLMs
Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli +4
Memory enables large language models (LLMs) to use information beyond their parameters, such as past user conversations, long documents, or an agent's earlier actions. For a frozen…
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal +5
Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models…
Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
Tianyu Pang, Vignesh Kothapalli, Shenyang Deng +3
We study optimal learning-rate selection in two-layer and three-layer linear neural networks trained to learn linear target functions. In particular, we derive the exact closed-for…
PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models
Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik +4
Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train…