6 papers
Language Acquisition Device in Large Language Models
Masato Mita, Taiga Someya, Ryo Yoshida +1
Large Language Models (LLMs) remain substantially less data-efficient than humans. Pre-pretraining (PPT) on synthetic languages has been proposed to close this gap, with prior work…
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal
Ryo Yoshida, Shinnosuke Isono, Taiga Someya +2
Surprisal theory hypothesizes that the difficulty of human sentence processing increases linearly with surprisal, the negative log-probability of a word given its context. Computat…
Rethinking the Relationship between the Power Law and Hierarchical Structures
Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa +2
Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across…
Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models
Taiga Someya, Ryo Yoshida, Hitomi Yanaka +1
Recent work has demonstrated that neural language models encode syntactic structures in their internal representations, yet the derivations by which these structures are constructe…
If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?
Ryo Yoshida, Shinnosuke Isono, Kohei Kajikawa +3
Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on vanilla Transformers…
Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition
Masato Mita, Ryo Yoshida, Yohei Oseki
Large language models possess general linguistic abilities but acquire language less efficiently than humans. This study proposes a method for integrating the developmental charact…