6 citations · 41 across the 52 of their papers we have counts for
49 papers · 1 filter
Evaluating Semantic Relation Knowledge through Bidirectional Relation Prediction from Gloss Pairs
Zhihan Cao, Adam Nohejl, Hitomi Yanaka
Semantic relations organise word senses and are fundamental to lexical semantics. In order to evaluate semantic relation knowledge of large language models (LLMs), we propose a cha…
Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking
Xiaotian Wang, Youyuan Lin, Zhan Shen +1
Advanced large language models (LLMs) with long context windows can substantially reduce input truncation in document-level machine translation (DocMT). However, direct Doc2Doc tra…
Cross-Lingual Representation Alignment by Token-Level Optimal Transport in a Language-Agnostic Space
Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala +1
Cross-lingual alignment (CLA) aims to align the representations of large language models (LLMs) across languages, enabling cross-lingual transfer to improve multilingual capabiliti…
Relation Geometry in Semantic Space of Language Models
Zhihan Cao, Hiroaki Yamada, Simone Teufel +4
When it comes to generating vector representations of words, current language models are achieving high-quality results. However, what is not known is the extent to which knowledge…
Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models
Kaito Watanabe, Taisei Yamamoto, Tomoki Doi +1
One of the expected abilities of vision-language models (VLMs) is spatial reasoning ability based on a given text and image. To evaluate the spatial reasoning abilities of VLMs, we…
YOMI-Bench: A Benchmark for Evaluating Kanji Reading and Phonological Understanding of LLMs for Japanese
Ryota Mibayashi, Hiroya Takamura, Hitomi Yanaka
We propose YOMI-Bench, a benchmark for evaluating kanji reading and phonological understanding of large language models (LLMs) for Japanese. In Japanese, a single kanji character o…
Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus
Yuheng Lu, Qingcheng Zeng, Heli Qi +6
Deep research agents are increasingly evaluated on their ability to search for evidence, reason over retrieved sources, and produce grounded answers. Existing browsing benchmarks,…
Revisiting the Systematicity in Negation in the Era of In-Context Learning
Hitomi Yanaka, Taisei Yamamoto
Understanding the meaning of negated sentences remains one of the challenges for language models, even in the era of large language models (LLMs). We analyze systematicity regardin…
Analysis of the Neglect-Zero Effect in Large Language Models
Jin Tanaka, Daiki Matsuoka, Ryoma Kumon +1
We investigate the extent to which the language processing of LLMs resembles human cognitive processes, focusing on a human cognitive bias called the …
Sakura at BEA 2026 Shared Task 1: What Makes Vocabulary Difficult?
Adam Nohejl, Xuanxin Wu, Yusuke Ide +3
We describe two types of models for vocabulary difficulty prediction: a high-accuracy black-box model, which achieved the top shared task result in the open track, and an explainab…
Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models
Ryoma Kumon, Hitomi Yanaka
While language models demonstrate sophisticated syntactic capabilities, the extent to which their internal mechanisms align with cross-constructional principles studied in linguist…
Developing a Guideline for the Labovian-Structural Analysis of Oral Narratives in Japanese
Amane Watahiki, Tomoki Doi, Akari Kikuchi +5
Narrative analysis is a cornerstone of qualitative research. One leading approach is the Labovian model, but its application is labor-intensive, requiring a holistic, recursive int…
NeuronMoE: Neuron-Guided Mixture-of-Experts for Efficient Multilingual LLM Extension
Rongzhi Li, Hitomi Yanaka
Extending large language models to low-resource languages is essential for global accessibility, but training separate models per language is prohibitively expensive. Mixture-of-Ex…
Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries
Yusuke Ide, Adam Nohejl, Joshua Tanner +3
Dictionary definitions are an essential resource for learning word senses, but manually creating them is costly. We thus study dictionary definition generation (DDG), i.e., the gen…
Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models
Tomoki Doi, Masaru Isonuma, Hitomi Yanaka
Large language models have the potential to generate explanations for their own predictions in a variety of styles based on user instructions. Recent research has examined whether…
Neuron-Level Analysis of Cultural Understanding in Large Language Models
Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala +1
As large language models (LLMs) are increasingly deployed worldwide, ensuring their fair and comprehensive cultural understanding is important. However, LLMs exhibit cultural bias…
Enhancing Rating Prediction with Off-the-Shelf LLMs Using In-Context User Reviews
Koki Ryu, Hitomi Yanaka
Personalizing the outputs of large language models (LLMs) to align with individual user preferences is an active research area. However, previous studies have mainly focused on cla…
Bias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Dataset
Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala +1
Large language models (LLMs) exhibit social biases, prompting the development of various debiasing methods. However, debiasing methods may degrade the capabilities of LLMs. Previou…
Implementing a Logical Inference System for Japanese Comparatives
Yosuke Mikami, Daiki Matsuoka, Hitomi Yanaka
Natural Language Inference (NLI) involving comparatives is challenging because it requires understanding quantities and comparative relations expressed by sentences. While some app…
Can Large Language Models Robustly Perform Natural Language Inference for Japanese Comparatives?
Yosuke Mikami, Daiki Matsuoka, Hitomi Yanaka
Large Language Models (LLMs) perform remarkably well in Natural Language Inference (NLI). However, NLI involving numerical and logical expressions remains challenging. Comparatives…
LLMs Struggle with NLI for Perfect Aspect: A Cross-Linguistic Study in Chinese and Japanese
Jie Lu, Du Jin, Hitomi Yanaka
Unlike English, which uses distinct forms (e.g., had, has, will have) to mark the perfect aspect across tenses, Chinese and Japanese lack separate grammatical forms for tense withi…
Intersectional Bias in Japanese Large Language Models from a Contextualized Perspective
Hitomi Yanaka, Xinqi He, Jie Lu +6
An increasing number of studies have examined the social bias of rapidly developed large language models (LLMs). Although most of these studies have focused on bias occurring in a…
Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports
Amane Watahiki, Tomoki Doi, Taiga Shinozaki +4
One of the main objectives in developing large vision-language models (LVLMs) is to engineer systems that can assist humans with multimodal tasks, including interpreting descriptio…
Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models
Taiga Someya, Ryo Yoshida, Hitomi Yanaka +1
Recent work has demonstrated that neural language models encode syntactic structures in their internal representations, yet the derivations by which these structures are constructe…