Showing cs.CLShow all
3 papers · 1 filter
cs.CL2026
How Local Mixing Encodes Relative Position in Global NoPE Attention
Cutter Dawes, Nick Alonso, Tom Figliolia +1
The attention operation is naively position invariant. However, positional information is fundamental to natural language, and therefore a variety of explicit position encodings ha…
cs.CL2026
H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models
Cutter Dawes, Aryan Sharma, Angelos Ioannis Lagos +1
Representing and navigating hierarchy is a fundamental primitive of reasoning. Large language models have demonstrated proficiency in a wide variety of tasks requiring hierarchical…
cs.CL2026
Dissociating Decodability and Causal Use in Bracket-Sequence Transformers
Aryan Sharma, Cutter Dawes, Shivam Raval
When trained on tasks requiring an understanding of hierarchical structure, transformers have been found to represent this hierarchy in distinct ways: in the geometry of the residu…