4 papers
Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs
Sohan Venkatesh
Large language models fail at counting how many times a word repeats in a list, even though they perform well on far harder reasoning tasks. These failures are commonly attributed…
Architecture, Not Scale: Circuit Localization in Large Language Models
Sohan Venkatesh
Mechanistic interpretability assumes that circuit analysis becomes harder as models scale. We challenge this assumption by showing that the attention architecture matters more than…
Negative Before Positive: Asymmetric Valence Processing in Large Language Models
Sohan Venkatesh
Mechanistic interpretability has revealed how concepts are encoded in large language models (LLMs), but emotional content remains poorly understood at the mechanistic level. We stu…
Algorithmic Blindness in Large Language Models: A Calibration Study of Performance Prediction
Sohan Venkatesh, Ashish Mahendran Kurapath, Tejas Melkote
Large language models (LLMs) demonstrate remarkable breadth of knowledge, yet their ability to reason about computational processes remains poorly understood. Closing this gap matt…