2 citations · 2 across the 8 of their papers we have counts for
9 papers · 1 filter
Language Acquisition Device in Large Language Models
Masato Mita, Taiga Someya, Ryo Yoshida +1
Large Language Models (LLMs) remain substantially less data-efficient than humans. Pre-pretraining (PPT) on synthetic languages has been proposed to close this gap, with prior work…
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal
Ryo Yoshida, Shinnosuke Isono, Taiga Someya +2
Surprisal theory hypothesizes that the difficulty of human sentence processing increases linearly with surprisal, the negative log-probability of a word given its context. Computat…
Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models
Taiga Someya, Ryo Yoshida, Hitomi Yanaka +1
Recent work has demonstrated that neural language models encode syntactic structures in their internal representations, yet the derivations by which these structures are constructe…
Rethinking the Relationship between the Power Law and Hierarchical Structures
Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa +2
Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across…
If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?
Ryo Yoshida, Shinnosuke Isono, Kohei Kajikawa +3
Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on vanilla Transformers…
Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition
Masato Mita, Ryo Yoshida, Yohei Oseki
Large language models possess general linguistic abilities but acquire language less efficiently than humans. This study proposes a method for integrating the developmental charact…
Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision
Ryo Yoshida, Taiga Someya, Yohei Oseki
Syntactic Language Models (SLMs) can be trained efficiently to reach relatively high performance; however, they have trouble with inference efficiency due to the explicit generatio…
Emergent Word Order Universals from Cognitively-Motivated Language Models
Tatsuki Kuribayashi, Ryo Ueda, Ryo Yoshida +3
The world's languages exhibit certain so-called typological or implicational universals; for example, Subject-Object-Verb (SOV) languages typically use postpositions. Explaining th…
Composition, Attention, or Both?
Ryo Yoshida, Yohei Oseki
In this paper, we propose a novel architecture called Composition Attention Grammars (CAGs) that recursively compose subtrees into a single vector representation with a composition…