12 citations · 16 across the 8 of their papers we have counts for
4 papers · 1 filter
Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates
Nikhil Prakash, Donghao Ren, Dominik Moritz +1
Prior studies investigating the internal workings of LLMs have uncovered sparse subnetworks, often referred to as circuits, that are responsible for performing specific tasks. Addi…
Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language
Angie Boggust, Donghao Ren, Yannick Assogba +3
Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature descriptions can be vague…
Evaluating Long Range Dependency Handling in Code Generation LLMs
Yannick Assogba, Donghao Ren
As language models support larger and larger context sizes, evaluating their ability to make effective use of that context becomes increasingly important. We analyze the ability of…
One Wide Feedforward is All You Need
Telmo Pessoa Pires, António V. Lopes, Yannick Assogba +1
The Transformer architecture has two main non-embedding components: Attention and the Feed Forward Network (FFN). Attention captures interdependencies between words regardless of t…