4 papers
Hyperloop Transformers
Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon Kim
LLM architecture research generally aims to maximize model quality subject to fixed compute/latency budgets. However, many applications of interest such as edge and on-device deplo…
On the Duality between Gradient Transformations and Adapters
Lucas Torroba-Hennigen, Hunter Lang, Han Guo +1
We study memory-efficient optimization of neural networks (in particular language models) with linear gradient transformations, where the gradients are linearly mapped to a lower d…
Towards Verifiable Text Generation with Symbolic References
Lucas Torroba Hennigen, Shannon Shen, Aniruddha Nrusimha +3
LLMs are vulnerable to hallucinations, and thus their outputs generally require laborious human verification for high-stakes applications. To this end, we propose symbolically grou…
Generalizing Backpropagation for Gradient-Based Interpretability
Kevin Du, Lucas Torroba Hennigen, Niklas Stoehr +2
Many popular feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model's output with respect to its inputs. While these methods c…