2 citations · 2 across the 10 of their papers we have counts for
11 papers
Language Models that Play Chess and Explain Their Moves
Adithya Bhaskar, Jeffrey Cheng, Danqi Chen
Modern chess engines are silent experts: they play at a superhuman level, but do not offer explanations for their play. On the other hand, language models (LMs) can generate plausi…
Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
Yinghui He, Simran Kaur, Adithya Bhaskar +7
Current post-training methods in verifiable settings fall into two categories. Reinforcement learning (RLVR) relies on binary rewards, which are broadly applicable and powerful, bu…
Extracting Rule-based Descriptions of Attention Features in Transformers
Dan Friedman, Adithya Bhaskar, Alexander Wettig +1
Mechanistic interpretability strives to explain model behavior in terms of bottom-up primitives. The leading paradigm is to express hidden states as a sparse linear combination of…
Language Models that Think, Chat Better
Adithya Bhaskar, Xi Ye, Danqi Chen
Reinforcement learning with verifiable rewards (RLVR) trains language models to use long chain-of-thought reasoning (CoT) in domains like mathematics and code with rule-based verif…
Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?
Adithya Bhaskar, Alexander Wettig, Tianyu Gao +2
Language models handle increasingly long contexts for tasks such as book summarization, but this leads to growing memory costs for the key-value (KV) cache. Many prior works have p…
Continual Memorization of Factoids in Language Models
Howard Chen, Jiayi Geng, Adithya Bhaskar +2
As new knowledge rapidly accumulates, language models (LMs) with pretrained knowledge quickly become obsolete. A common approach to updating LMs is fine-tuning them directly on new…
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
Noam Razin, Sadhika Malladi, Adithya Bhaskar +3
Direct Preference Optimization (DPO) and its variants are increasingly used for aligning language models with human preferences. Although these methods are designed to teach a mode…
Finding Transformer Circuits with Edge Pruning
Adithya Bhaskar, Alexander Wettig, Dan Friedman +1
The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Rece…
The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
Adithya Bhaskar, Dan Friedman, Danqi Chen
Prior work has found that pretrained language models (LMs) fine-tuned with different random seeds can achieve similar in-domain performance but generalize differently on tests of s…
Improving Language Understanding from Screenshots
Tianyu Gao, Zirui Wang, Adithya Bhaskar +1
An emerging family of language models (LMs), capable of processing both text and images within a single visual view, has the promise to unlock complex tasks such as chart understan…
Benchmarking and Improving Text-to-SQL Generation under Ambiguity
Adithya Bhaskar, Tushar Tomar, Ashutosh Sathe +1
Research in Text-to-SQL conversion has been largely benchmarked against datasets where each text query corresponds to one correct SQL. However, natural language queries over real-l…