22 citations · 47 across the 13 of their papers we have counts for
16 papers
Memorilla: Latent Semantic Memory for LLMs
Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli +4
Memory enables large language models (LLMs) to use information beyond their parameters, such as past user conversations, long documents, or an agent's earlier actions. For a frozen…
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal +5
Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models…
Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
Tianyu Pang, Vignesh Kothapalli, Shenyang Deng +3
We study optimal learning-rate selection in two-layer and three-layer linear neural networks trained to learn linear target functions. In particular, we derive the exact closed-for…
PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models
Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik +4
Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train…
To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
Vignesh Kothapalli, Ata Fatahibaarzi, Hamed Firooz +1
Chain-of-thought (CoT) prompting combined with few-shot in-context learning (ICL) has unlocked significant reasoning capabilities in large language models (LLMs). However, ICL with…
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
Wei-Rui Chen, Vignesh Kothapalli, Ata Fatahibaarzi +5
Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data. However, knowledge disti…
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations
Vignesh Kothapalli, Hamed Firooz, Maziar Sanjabi
We introduce CoT-ICL Lab, a framework and methodology to generate synthetic tokenized datasets and systematically study chain-of-thought (CoT) in-context learning (ICL) in language…
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song +17
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Al…
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation
Hamed Firooz, Maziar Sanjabi, Adrian Englhardt +20
Ranking and recommendation systems are the foundation for numerous online experiences, ranging from search results to personalized content delivery. These systems have evolved into…
Liger Kernel: Efficient Triton Kernels for LLM Training
Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli +7
Training Large Language Models (LLMs) efficiently at scale presents a formidable challenge, driven by their ever-increasing computational demands and the need for enhanced performa…
From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
Vignesh Kothapalli, Tianyu Pang, Shenyang Deng +2
Training strategies for modern deep neural networks (NNs) tend to induce a heavy-tailed (HT) empirical spectral density (ESD) in the layer weights. While previous efforts have show…
Can Kernel Methods Explain How the Data Affects Neural Collapse?
Vignesh Kothapalli, Tom Tirer
A vast amount of literature has recently focused on the "Neural Collapse" (NC) phenomenon, which emerges when training neural network (NN) classifiers beyond the zero training erro…
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks
Vignesh Kothapalli, Tom Tirer, Joan Bruna
Graph neural networks (GNNs) have become increasingly popular for classification tasks on graph-structured data. Yet, the interplay between graph topology and feature evolution in…
Randomized Schur Complement Views for Graph Contrastive Learning
Vignesh Kothapalli
We introduce a randomized topological augmentor based on Schur complements for Graph Contrastive Learning (GCL). Given a graph laplacian matrix, the technique generates unbiased ap…
Neural Collapse: A Review on Modelling Principles and Generalization
Vignesh Kothapalli
Deep classifier neural networks enter the terminal phase of training (TPT) when training error reaches zero and tend to exhibit intriguing Neural Collapse (NC) properties. Neural c…
Binary Document Image Super Resolution for Improved Readability and OCR Performance
Ram Krishna Pandey, K Vignesh, A G Ramakrishnan +1
There is a need for information retrieval from large collections of low-resolution (LR) binary document images, which can be found in digital libraries across the world, where the…