1 citations · 1 across the 27 of their papers we have counts for
10 papers · 1 filter
Can Language Models Handle a Non-Gregorian Calendar? The Case of the Japanese wareki
Mutsumi Sasaki, Go Kamoda, Ryosuke Takahashi +4
Temporal reasoning and knowledge are essential capabilities for language models (LMs). While much prior work has analyzed and improved temporal reasoning in LMs, most studies have…
On Entity Identification in Language Models
Masaki Sakata, Benjamin Heinzerling, Sho Yokoi +2
We analyze the extent to which internal representations of language models (LMs) identify and distinguish mentions of named entities, focusing on the many-to-many correspondence be…
TopK Language Models
Ryosuke Takahashi, Tatsuro Inaba, Kentaro Inui +1
Sparse autoencoders (SAEs) have become an important tool for analyzing and interpreting the activation space of transformer-based language models (LMs). However, SAEs suffer severa…
Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
Shintaro Ozaki, Tatsuya Hiraoka, Hiroto Otake +8
Large Language Models (LLMs) are known to process information using a proficient internal language consistently, referred to as latent language, which may differ from the input or…
Understanding Fact Recall in Language Models: Why Two-Stage Training Encourages Memorization but Mixed Training Teaches Knowledge
Ying Zhang, Benjamin Heinzerling, Dongyuan Li +1
While fine-tuning is the standard for injecting factual knowledge into large language models (LLMs), the mechanisms enabling reliable fact recall via unseen queries remain poorly u…
SPIRIT: Patching Speech Language Models against Jailbreak Attacks
Amirbek Djanibekov, Nurdaulet Mukhituly, Kentaro Inui +2
Speech Language Models (SLMs) enable natural interactions via spoken instructions, which more effectively capture user intent by detecting nuances in speech. The richer speech sign…
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
Tatsuro Inaba, Go Kamoda, Kentaro Inui +5
This study explores how bilingual language models develop complex internal representations. We employ sparse autoencoders to analyze internal representations of bilingual language…
Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
Go Kamoda, Benjamin Heinzerling, Tatsuro Inaba +3
According to the stages-of-inference hypothesis, early layers of language models map their subword-tokenized input, which does not necessarily correspond to a linguistically meanin…
FinchGPT: a Transformer based language model for birdsong analysis
Kosei Kobayashi, Kosuke Matsuzaki, Masaya Taniguchi +3
The long-range dependencies among the tokens, which originate from hierarchical structures, are a defining hallmark of human language. However, whether similar dependencies exist w…
RECALL: Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
Munachiso Nwadike, Zangir Iklassov, Toluwani Aremu +6
We introduce the concept of the self-referencing causal cycle (abbreviated RECALL) - a mechanism that enables large language models (LLMs) to bypass the limitations of unidirection…