12 citations · 14 across the 10 of their papers we have counts for
10 papers
Data-driven Circuit Discovery for Interpretability of Language Models
Daking Rai, Mor Geva, Ziyu Yao
Circuit study aims to explain how language models (LMs) implement a task by localizing and interpreting a circuit: a computational subgraph responsible for a specific model's behav…
All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens
Siddarth Mamidanna, Daking Rai, Ziyu Yao +1
Large language models (LLMs) demonstrate proficiency across numerous computational tasks, yet their inner workings remain unclear. In theory, the combination of causal self-attenti…
Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones
Daking Rai, Samuel Miller, Kevin Moran +1
Despite remarkable advances in coding capabilities, language models (LMs) still struggle with simple syntactic tasks such as generating balanced parentheses. In this study, we inve…
A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models
Dong Shu, Xuansheng Wu, Haiyan Zhao +4
Large Language Models (LLMs) have transformed natural language processing, yet their internal mechanisms remain largely opaque. Recently, mechanistic interpretability has attracted…
Mechanistic Understanding of Language Models in Syntactic Code Completion
Samuel Miller, Daking Rai, Ziyu Yao
Recently, language models (LMs) have shown impressive proficiency in code generation tasks, especially when fine-tuned on code-specific datasets, commonly known as Code LMs. Howeve…
Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing
Daking Rai, Rydia R. Weiland, Kayla Margaret Gabriella Herrera +2
Explaining the decisions of AI has become vital for fostering appropriate user trust in these systems. This paper investigates explanations for a structured prediction task called…
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
Daking Rai, Yilun Zhou, Shi Feng +2
Mechanistic interpretability (MI) is an emerging sub-field of interpretability that seeks to understand a neural network model by reverse-engineering its internal computations. Rec…
An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
Daking Rai, Ziyu Yao
Large language models (LLMs) have shown strong arithmetic reasoning capabilities when prompted with Chain-of-Thought (CoT) prompts. However, we have only a limited understanding of…
Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques
Daking Rai, Bailin Wang, Yilun Zhou +1
Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs). I…
Explaining Large Language Model-Based Neural Semantic Parsers (Student Abstract)
Daking Rai, Yilun Zhou, Bailin Wang +1
While large language models (LLMs) have demonstrated strong capability in structured prediction tasks such as semantic parsing, few amounts of research have explored the underlying…