1 citations · 1 across the 27 of their papers we have counts for
8 papers · 1 filter
Representational Analysis of Binding in Language Models
Qin Dai, Benjamin Heinzerling, Kentaro Inui
Entity tracking is essential for complex reasoning. To perform in-context entity tracking, language models (LMs) must bind an entity to its attribute (e.g., bind a container to its…
Repetition Neurons: How Do Language Models Produce Repetitions?
Tatsuya Hiraoka, Kentaro Inui
This paper introduces repetition neurons, regarded as skill neurons responsible for the repetition problem in text generation tasks. These neurons are progressively activated more…
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh +2
This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when answering questions invo…
The Curse of Popularity: Popular Entities have Catastrophic Side Effects when Deleting Knowledge from Language Models
Ryosuke Takahashi, Go Kamoda, Benjamin Heinzerling +2
Language models (LMs) encode world knowledge in their internal parameters through training. However, LMs may learn personal and confidential information from the training data, lea…
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi +4
Multi-step reasoning instruction, such as chain-of-thought prompting, is widely adopted to explore better language models (LMs) performance. We report on the systematic strategy th…
ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation
Ana Brassard, Benjamin Heinzerling, Keito Kudo +2
Evaluating the quality of free-text explanations is a multifaceted, subjective, and labor-intensive task. Large language models (LLMs) present an appealing alternative due to their…
Monotonic Representation of Numeric Properties in Language Models
Benjamin Heinzerling, Kentaro Inui
Language models (LMs) can express factual knowledge involving numeric properties such as Karl Popper was born in 1902. However, how this information is encoded in the model's inter…
J-UniMorph: Japanese Morphological Annotation through the Universal Feature Schema
Kosuke Matsuzaki, Masaya Taniguchi, Kentaro Inui +1
We introduce a Japanese Morphology dataset, J-UniMorph, developed based on the UniMorph feature schema. This dataset addresses the unique and rich verb forms characteristic of the…