From the 1 of 10 linked papers with an AI index.
10 papers
Relation Geometry in Semantic Space of Language Models
Zhihan Cao, Hiroaki Yamada, Simone Teufel +4
The paper investigates how different semantic relations are reflected in the geometric structure of word embedding spaces produced by various language models, examining region clus…
Sycophancy Hides Linearly in the Attention Heads
Rifo Genadi, Munachiso Nwadike, Nurdaulet Mukhituly +3
We find that correct-to-incorrect sycophancy signals are most linearly separable within multi-head attention activations. Motivated by the linear representation hypothesis, we trai…
Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning
Nhi Hoai Doan, Tatsuya Hiraoka, Kentaro Inui
This paper investigates the relationship between large language models' (LLMs) ability to recognize repetitive input patterns and their performance on in-context learning (ICL). In…
Augmenting Dialog with Think-Aloud Utterances for Modeling Individual Personality Traits by LLM
Seiya Ishikura, Hiroaki Yamada, Tatsuya Hiraoka +1
This study proposes augmenting dialog data with think-aloud utterances (TAUs) for modeling individual personalities in text chat by LLM. TAU is a verbalization of a speaker's thoug…
Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters
Tatsuya Hiraoka, Kentaro Inui
Large language models (LLMs) can spell out tokens character by character with high accuracy, yet they struggle with more complex character-level tasks, such as identifying composit…
Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
Shintaro Ozaki, Tatsuya Hiraoka, Hiroto Otake +8
Large Language Models (LLMs) are known to process information using a proficient internal language consistently, referred to as latent language, which may differ from the input or…