10 citations · 35 across the 25 of their papers we have counts for
9 papers · 2 filters
Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models
Taiga Someya, Ryo Yoshida, Hitomi Yanaka +1
Recent work has demonstrated that neural language models encode syntactic structures in their internal representations, yet the derivations by which these structures are constructe…
Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
Yuto Harada, Yusuke Yamauchi, Yusuke Oda +3
Supervised fine-tuning (SFT) is a critical step in aligning large language models (LLMs) with human instructions and values, yet many aspects of SFT remain poorly understood. We tr…
Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
Shintaro Ozaki, Tatsuya Hiraoka, Hiroto Otake +8
Large Language Models (LLMs) are known to process information using a proficient internal language consistently, referred to as latent language, which may differ from the input or…
Rethinking the Relationship between the Power Law and Hierarchical Structures
Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa +2
Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across…
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
Tatsuro Inaba, Go Kamoda, Kentaro Inui +5
This study explores how bilingual language models develop complex internal representations. We employ sparse autoencoders to analyze internal representations of bilingual language…
Can Language Models Learn Typologically Implausible Languages?
Tianyang Xu, Tatsuki Kuribayashi, Yohei Oseki +2
Grammatical features across human languages show intriguing correlations often attributed to learning biases in humans. However, empirical evidence has been limited to experiments…