221 citations · 283 across the 19 of their papers we have counts for
23 papers
Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism
Prakhar Gupta, Terry Jingchen Zhang, Zhijing Jin
Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B." Sycophancy and related biases show th…
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?
Prakhar Gupta, Terry Jingchen Zhang, Florent Draye +2
Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few-shot example, or a fake prior assistant…
Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
Prakhar Gupta, Garv Shah, Satyam Goyal +1
Adapting a pretrained language model to a new task often hurts the general capabilities it already had, a problem known as catastrophic forgetting. Sparse Memory Finetuning (SMF) t…
Self-Mined Hardness for Safety Fine-Tuning
Prakhar Gupta, Garv Shah, Donghua Zhang
Safety fine-tuning of language models typically requires a curated adversarial dataset. We take a different approach: score each candidate prompt's difficulty by how often the targ…
Consistency Training Along the Transformer Stack
Sukrati Gautam, Neil Shah, Arav Dhoot +7
Consistency training encourages models to behave similarly across different contexts, and has shown promise for reducing misalignment. We broaden the scope of consistency training…
Consistency Training while Mitigating Obfuscation via Rate Matching
Sohaib Imran, Prakhar Gupta, Jannes Elstner +1
Large language models are often influenced by extraneous input features, such as cues revealing a user's preferred answer. Consistency training reduces this influence by training m…
Improving Sparse Memory Finetuning
Satyam Goyal, Anirudh Kanchi, Garv Shah +1
Large Language Models (LLMs) are typically static after training, yet real-world applications require continual adaptation to new knowledge without degrading existing capabilities.…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
Kundan Krishna, Sanjana Ramprasad, Prakhar Gupta +3
LLMs can generate factually incorrect statements even when provided access to reference documents. Such errors can be dangerous in high-stakes applications (e.g., document-grounded…
USB: A Unified Summarization Benchmark Across Tasks and Domains
Kundan Krishna, Prakhar Gupta, Sanjana Ramprasad +3
While the NLP community has produced numerous summarization benchmarks, none provide the rich annotations required to simultaneously address many important problems related to cont…
Self-Refine: Iterative Refinement with Self-Feedback
Aman Madaan, Niket Tandon, Prakhar Gupta +13
Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an…
Using In-Context Learning to Improve Dialogue Safety
Nicholas Meade, Spandana Gella, Devamanyu Hazarika +5
While large neural-based conversational models have become increasingly proficient dialogue agents, recent work has highlighted safety issues with these systems. For example, these…
Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation
Jessica Huynh, Cathy Jiao, Prakhar Gupta +4
Language models have steadily increased in size over the past few years. They achieve a high level of performance on various natural language processing (NLP) tasks such as questio…
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines
Prakhar Gupta, Yang Liu, Di Jin +6
Dialogue models are able to generate coherent and fluent responses, but they can still be challenging to control and may produce non-engaging, unsafe results. This unpredictability…
Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation
Prakhar Gupta, Harsh Jhamtani, Jeffrey P. Bigham
Target-guided response generation enables dialogue systems to smoothly transition a conversation from a dialogue context toward a target sentence. Such control is useful for design…
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh +3
Instruction tuning is an emergent paradigm in NLP wherein natural language instructions are leveraged with language models to induce zero-shot performance on unseen tasks. Instruct…
DialFact: A Benchmark for Fact-Checking in Dialogue
Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu +1
Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation. We introduce the task of fact-checking in dialogue, which is a relatively unexplored…
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation
Prakhar Gupta, Yulia Tsvetkov, Jeffrey P. Bigham
Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. These tasks are formulated as a binary classification of responses given…
Controlling Dialogue Generation with Semantic Exemplars
Prakhar Gupta, Jeffrey P. Bigham, Yulia Tsvetkov +1
Dialogue systems pretrained with large language models generate locally coherent responses, but lack the fine-grained control over responses necessary to achieve specific goals. A…
Using Image Captions and Multitask Learning for Recommending Query Reformulations
Gaurav Verma, Vishwa Vinay, Sahil Bansal +3
Interactive search sessions often contain multiple queries, where the user submits a reformulated version of the previous query in response to the original results. We aim to enhan…
WriterForcing: Generating more interesting story endings
Prakhar Gupta, Vinayshekhar Bannihatti Kumar, Mukul Bhutani +1
We study the problem of generating interesting endings for stories. Neural generative models have shown promising results for various text generation problems. Sequence to Sequence…
Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References
Prakhar Gupta, Shikib Mehri, Tiancheng Zhao +3
The aim of this paper is to mitigate the shortcomings of automatic evaluation of open-domain dialog systems through multi-reference evaluation. Existing metrics have been shown to…
Saliency Prediction for Mobile User Interfaces
Prakhar Gupta, Shubh Gupta, Ajaykrishnan Jayagopal +2
We introduce models for saliency prediction for mobile user interfaces. A mobile interface may include elements like buttons, text, etc. in addition to natural images which enable…