4 papers
CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models
Miyu Oba, Saku Sugawara
Recent work has examined language models from a linguistic perspective to better understand how they acquire language. Most existing benchmarks focus on judging grammatical accepta…
BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency
Akari Haga, Akiyo Fukatsu, Miyu Oba +2
While current large language models have achieved a remarkable success, their data efficiency remains a challenge to overcome. Recently it has been suggested that child-directed sp…
Can Language Models Induce Grammatical Knowledge from Indirect Evidence?
Miyu Oba, Yohei Oseki, Akiyo Fukatsu +4
What kinds of and how much data is necessary for language models to induce grammatical knowledge to judge sentence acceptability? Recent language models still have much room for im…
BQA: Body Language Question Answering Dataset for Video Large Language Models
Shintaro Ozaki, Kazuki Hayashi, Miyu Oba +3
A large part of human communication relies on nonverbal cues such as facial expressions, eye contact, and body language. Unlike language or sign language, such nonverbal communicat…