4 citations · 9 across the 5 of their papers we have counts for
5 papers
Updating Parametric Knowledge with Context Distillation Retains Post-Training Capabilities
Shankar Padmanabhan, Mustafa Omer Gul, Tanya Goyal
Post-training endows pretrained LLMs with a variety of desirable skills, including instruction-following, reasoning, and others. However, these post-trained LLMs only encode knowle…
Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
Giovanni Monea, Yair Feldman, Shankar Padmanabhan +2
The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memo…
Propagating Knowledge Updates to LMs Through Distillation
Shankar Padmanabhan, Yasumasa Onoe, Michael J. Q. Zhang +2
Modern language models have the capacity to store and use immense amounts of knowledge about real-world entities, but it remains unclear how to update such knowledge stored in mode…
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan +2
Pre-trained language models (LMs) are used for knowledge intensive tasks like question answering, but their knowledge gets continuously outdated as the world changes. Prior work ha…
Self-Programming Artificial Intelligence Using Code-Generating Language Models
Alex Sheng, Shankar Padmanabhan
Recent progress in large-scale language models has enabled breakthroughs in previously intractable computer programming tasks. Prior work in meta-learning and neural architecture s…