1 citations · 1 across the 27 of their papers we have counts for
10 papers · 1 filter
Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles
Célian Vasson, Daniil Ognev, Kentaro Inui +1
Variable roles describe recurring behaviors such as accumulation, traversal, and retaining the best candidate encountered. We use Sajaniemi's independently defined taxonomy to stud…
Rethinking Contextualization by Reinterpreting Attention Head Channels
Hakaze Cho, Haolin Yang, Zhun Sun +3
Contextualization, the core operation of language modeling, transmits information across words to build sentence-specific word representations. Prior works mainly study contextuali…
SharedSAE: One Feature Dictionary Across Language Models
Daniil Ognev, Célian Vasson, Lijie Hu +2
Sparse autoencoders (SAEs) are widely used to interpret language model activations, but SAE training and latent labelling are typically repeated for every model. Here, we show that…
Relation Geometry in Semantic Space of Language Models
Zhihan Cao, Hiroaki Yamada, Simone Teufel +4
When it comes to generating vector representations of words, current language models are achieving high-quality results. However, what is not known is the extent to which knowledge…
Einstein World Models
Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky +2
Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alo…
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
Mutsumi Sasaki, Go kamoda, Ryosuke Takahashi +4
Quantities with measurement units, such as 110 cm and 1.2 m, require language models (LMs) to combine a numeral with a symbolic unit scale. Here, we study how LMs compare such quan…
Measuring AI Reasoning: A Guide for Researchers
Munachiso Samuel Nwadike, Zangir Iklassov, Kareem Ali +2
In this paper, we offer a guide for researchers on evaluating reasoning in language models, building the case that reasoning should be assessed through evidence of adaptive, multi-…
Cell-Based Representation of Relational Binding in Language Models
Qin Dai, Benjamin Heinzerling, Kentaro Inui
Understanding a discourse requires tracking entities and the relations that hold between them. While Large Language Models (LLMs) perform well on relational reasoning, the mechanis…
Linear Representations of Hierarchical Concepts in Language Models
Masaki Sakata, Benjamin Heinzerling, Takumi Ito +2
We investigate how and to what extent hierarchical relations (e.g., Japan Eastern Asia Asia) are encoded in the internal representations of language models. Bui…
Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models
Haruto Yoshida, Keito Kudo, Yoichi Aoki +4
Large vision-language models (LVLMs) demonstrate strong performance on diagram understanding benchmarks, yet they still struggle with understanding relationships between elements,…