2 citations · 2 across the 10 of their papers we have counts for
10 papers · 1 filter
Language Models that Play Chess and Explain Their Moves
Adithya Bhaskar, Jeffrey Cheng, Danqi Chen
Modern chess engines are silent experts: they play at a superhuman level, but do not offer explanations for their play. On the other hand, language models (LMs) can generate plausi…
Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
Yinghui He, Simran Kaur, Adithya Bhaskar +7
Current post-training methods in verifiable settings fall into two categories. Reinforcement learning (RLVR) relies on binary rewards, which are broadly applicable and powerful, bu…
Extracting Rule-based Descriptions of Attention Features in Transformers
Dan Friedman, Adithya Bhaskar, Alexander Wettig +1
Mechanistic interpretability strives to explain model behavior in terms of bottom-up primitives. The leading paradigm is to express hidden states as a sparse linear combination of…
Language Models that Think, Chat Better
Adithya Bhaskar, Xi Ye, Danqi Chen
Reinforcement learning with verifiable rewards (RLVR) trains language models to use long chain-of-thought reasoning (CoT) in domains like mathematics and code with rule-based verif…
Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?
Adithya Bhaskar, Alexander Wettig, Tianyu Gao +2
Language models handle increasingly long contexts for tasks such as book summarization, but this leads to growing memory costs for the key-value (KV) cache. Many prior works have p…
Continual Memorization of Factoids in Language Models
Howard Chen, Jiayi Geng, Adithya Bhaskar +2
As new knowledge rapidly accumulates, language models (LMs) with pretrained knowledge quickly become obsolete. A common approach to updating LMs is fine-tuning them directly on new…
Finding Transformer Circuits with Edge Pruning
Adithya Bhaskar, Alexander Wettig, Dan Friedman +1
The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Rece…
The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
Adithya Bhaskar, Dan Friedman, Danqi Chen
Prior work has found that pretrained language models (LMs) fine-tuned with different random seeds can achieve similar in-domain performance but generalize differently on tests of s…
Improving Language Understanding from Screenshots
Tianyu Gao, Zirui Wang, Adithya Bhaskar +1
An emerging family of language models (LMs), capable of processing both text and images within a single visual view, has the promise to unlock complex tasks such as chart understan…
Benchmarking and Improving Text-to-SQL Generation under Ambiguity
Adithya Bhaskar, Tushar Tomar, Ashutosh Sathe +1
Research in Text-to-SQL conversion has been largely benchmarked against datasets where each text query corresponds to one correct SQL. However, natural language queries over real-l…