13 citations · 43 across the 26 of their papers we have counts for
31 papers
DimPO: Dimensionality Reduction for Attention using Preference Optimization
Vojtěch Lanz, Yufei Cui, Prasanna Parthasarathi
A linear projection can reduce the dimension of query and key vectors without updating the pretrained model, but it remains unclear which training objective best preserves model be…
Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Models
Saba Ahmadi, Prasanna Parthasarathi, Yufei Cui
Diffusion language models are a promising alternative to autoregressive models, yet post-training methods for them largely adapt reward-maximizing objectives. We identify a central…
InfMem: Learning System-2 Memory Control for Long-Context Agent
Xinyu Wang, Mingze Li, Peng Lu +6
Reasoning over ultra-long documents requires synthesizing sparse evidence scattered across distant segments under strict memory constraints. While streaming agents enable scalable…
Thinking Long, but Short: Stable Sequential Test-Time Scaling for Large Reasoning Models
Michael R. Metel, Yufei Cui, Boxing Chen +1
Sequential test-time scaling is a promising training-free method to improve large reasoning model accuracy, but as currently implemented, significant limitations have been observed…
GRPO-: Credit Assignment improves LLM Reasoning
Prasanna Parthasarathi, Mathieu Reymond, Boxing Chen +2
Large language models (LLMs) are increasingly deployed for tasks requiring complex reasoning, prompting significant interest in improving their reasoning abilities through post-tra…
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet, Yufei Cui, Boxing Chen +2
Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT). In standard ReFT framewor…
Resona: Improving Context Copying in Linear Recurrence Models with Retrieval
Xinyu Wang, Linrui Ma, Jerry Huang +5
Recent shifts in the space of large language model (LLM) research have shown an increasing focus on novel architectures to compete with prototypical Transformer-based models that h…
Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination
Jerry Huang, Prasanna Parthasarathi, Mehdi Rezagholizadeh +2
The growth in prominence of large language models (LLMs) in everyday life can be largely attributed to their generative abilities, yet some of this is also owed to the risks and co…
Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Models
Jerry Huang, Prasanna Parthasarathi, Mehdi Rezagholizadeh +1
Despite their widespread adoption, large language models (LLMs) remain prohibitive to use under resource constraints, with their ever growing sizes only increasing the barrier for…
EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems
Mohammad Dehghan, Mohammad Ali Alomrani, Sunyam Bagga +12
The emerging citation-based QA systems are gaining more attention especially in generative AI search applications. The importance of extracted knowledge provided to these systems i…
CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems
Abbas Ghaddar, David Alfonso-Hermelo, Philippe Langlais +3
In this work, we dive deep into one of the popular knowledge-grounded dialogue benchmarks that focus on faithfulness, FaithDial. We show that a significant portion of the FaithDial…
Towards Practical Tool Usage for Continually Learning LLMs
Jerry Huang, Prasanna Parthasarathi, Mehdi Rezagholizadeh +1
Large language models (LLMs) show an innate skill for solving language based tasks. But insights have suggested an inability to adjust for information or task-solving skills becomi…
Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Games
Arjun Vaithilingam Sudhakar, Prasanna Parthasarathi, Janarthanan Rajendran +1
Large Language Models (LLMs) have demonstrated superior performance in language understanding benchmarks. CALM, a popular approach, leverages linguistic priors of LLMs -- GPT-2 --…
Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models
Amirhossein Kazemnejad, Mehdi Rezagholizadeh, Prasanna Parthasarathi +1
While pre-trained language models (PLMs) have shown evidence of acquiring vast amounts of knowledge, it remains unclear how much of this parametric knowledge is actually usable in…
Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness
Abdelrahman Zayed, Prasanna Parthasarathi, Goncalo Mordido +3
Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes, which raises equity concerns. Prediction models may discover, us…
Local Structure Matters Most in Most Languages
Louis Clouâtre, Prasanna Parthasarathi, Amal Zouaq +1
Many recent perturbation studies have found unintuitive results on what does and does not matter when performing Natural Language Understanding (NLU) tasks in English. Coding prope…
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes
Louis Clouâtre, Prasanna Parthasarathi, Amal Zouaq +1
Providing better language tools for low-resource and endangered languages is imperative for equitable growth. Recent progress with massively multilingual pretrained models has prov…
Local Structure Matters Most: Perturbation Study in NLU
Louis Clouatre, Prasanna Parthasarathi, Amal Zouaq +1
Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the orde…
Memory Augmented Optimizers for Deep Learning
Paul-Aymeric McRae, Prasanna Parthasarathi, Mahmoud Assran +1
Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradients for efficient parameter updates. Th…
Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ?
Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar
Predicting the next utterance in dialogue is contingent on encoding of users' input text to generate appropriate and relevant response in data-driven approaches. Although the seman…
A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss
Prasanna Parthasarathi, Mohamed Abdelsalam, Joelle Pineau +1
Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training objectives like negative log-likelihood (…
Sometimes We Want Translationese
Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau +1
Rapid progress in Neural Machine Translation (NMT) systems over the last few years has been driven primarily towards improving translation quality, and as a secondary focus, improv…
UnNatural Language Inference
Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau +1
Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to…
On Task-Level Dialogue Composition of Generative Transformer Model
Prasanna Parthasarathi, Arvind Neelakantan, Sharan Narang
Task-oriented dialogue systems help users accomplish tasks such as booking a movie ticket and ordering food via conversation. Generative models parameterized by a deep neural netwo…