Showing 2024Show all
2 papers · 1 filter
cs.CL2024
Repetition Neurons: How Do Language Models Produce Repetitions?
Tatsuya Hiraoka, Kentaro Inui
This paper introduces repetition neurons, regarded as skill neurons responsible for the repetition problem in text generation tasks. These neurons are progressively activated more…
cs.CL2024
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh +2
This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when answering questions invo…