36 citations · 37 across the 9 of their papers we have counts for
9 papers
A Defense of the Quadratic Model
Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +3
Due to the complexity of neural network loss landscapes, optimization theory is forced to rely on idealized models, and there is generally a tradeoff between how theoretically trac…
Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
Alexandru Meterez, Pranav Ajit Nair, Depen Morwani +2
Large language models are increasingly trained in continual or open-ended settings, where the total training horizon is not known in advance. Despite this, most existing pretrainin…
Matryoshka Quantization
Pranav Nair, Puranjay Datta, Jeff Dean +2
Quantizing model weights is critical for reducing the communication and inference costs of large models. However, quantizing models -- especially to low precisions like int4 or int…
CDQuant: Greedy Coordinate Descent for Accurate LLM Quantization
Pranav Ajit Nair, Arun Sai Suggala
Large language models (LLMs) have recently demonstrated remarkable performance across diverse language tasks. But their deployment is often constrained by their substantial computa…
Tandem Transformers for Inference Efficient LLMs
Aishwarya P S, Pranav Ajit Nair, Yashas Samaga +4
The autoregressive nature of conventional large language models (LLMs) inherently limits inference speed, as tokens are generated sequentially. While speculative and parallel decod…
Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization
Pranav Ajit Nair, Sukomal Pal, Pradeepika Verma
Domain generalization is hitherto an underexplored area applied in abstractive summarization. Moreover, most existing works on domain generalization have sophisticated training alg…
The Role of Output Vocabulary in T2T LMs for SPARQL Semantic Parsing
Debayan Banerjee, Pranav Ajit Nair, Ricardo Usbeck +1
In this work, we analyse the role of output vocabulary for text-to-text (T2T) models on the task of SPARQL semantic parsing. We perform experiments within the the context of knowle…
GETT-QA: Graph Embedding based T2T Transformer for Knowledge Graph Question Answering
Debayan Banerjee, Pranav Ajit Nair, Ricardo Usbeck +1
In this work, we present an end-to-end Knowledge Graph Question Answering (KGQA) system named GETT-QA. GETT-QA uses T5, a popular text-to-text pre-trained language model. The model…
Modern Baselines for SPARQL Semantic Parsing
Debayan Banerjee, Pranav Ajit Nair, Jivat Neet Kaur +2
In this work, we focus on the task of generating SPARQL queries from natural language questions, which can then be executed on Knowledge Graphs (KGs). We assume that gold entity an…