2 citations · 10 across the 59 of their papers we have counts for
17 papers · 1 filter
Agreement-Constrained Probabilistic Minimum Bayes Risk Decoding
Koki Natsumi, Hiroyuki Deguchi, Yusuke Sakai +2
Minimum Bayes risk (MBR) decoding generates high-quality translations by maximizing the expected utility of output candidates, but it evaluates all pairwise scores over the candida…
Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach
Haruki Sakajo, Hiroshi Takato, Hiroshi Tsutsui +3
Large-scale Vision Language Models (LVLMs) exhibit advanced capabilities in tasks that require visual information, including object detection. These capabilities have promising app…
J-ORA: A Framework and Multimodal Dataset for Japanese Object Identification, Reference, Action Prediction in Robot Perception
Jesse Atuhurra, Hidetaka Kamigaito, Taro Watanabe +1
We introduce J-ORA, a novel multimodal dataset that bridges the gap in robot perception by providing detailed object attribute annotations within Japanese human-robot dialogue scen…
Reliability Crisis of Reference-free Metrics for Grammatical Error Correction
Takumi Goto, Yusuke Sakai, Taro Watanabe
Reference-free evaluation metrics for grammatical error correction (GEC) have achieved high correlation with human judgments. However, these metrics are not designed to evaluate ad…
Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
Wataru Hashimoto, Hidetaka Kamigaito, Taro Watanabe
Decoding strategies manipulate the probability distribution underlying the output of a language model and can therefore affect both generation quality and its uncertainty. In this…
SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
Ashmari Pramodya, Nirasha Nelki, Heshan Shalinda +6
Large Language Models (LLMs) demonstrate impressive general knowledge and reasoning abilities, yet their evaluation has predominantly focused on global or anglocentric subjects, of…
Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability
Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
In generative commonsense reasoning tasks such as CommonGen, generative large language models (LLMs) compose sentences that include all given concepts. However, when focusing on in…
Diversity of Transformer Layers: One Aspect of Parameter Scaling Laws
Hidetaka Kamigaito, Ying Zhang, Jingun Kwon +3
Transformers deliver outstanding performance across a wide range of tasks and are now a dominant backbone architecture for large language models (LLMs). Their task-solving performa…
IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator
Yusuke Sakai, Takumi Goto, Taro Watanabe
We propose IMPARA-GED, a novel reference-free automatic grammatical error correction (GEC) evaluation method with grammatical error detection (GED) capabilities. We focus on the qu…
Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries
Haruki Sakajo, Yusuke Ide, Justin Vasselli +4
Cross-lingual vocabulary transfer plays a promising role in adapting pre-trained language models to new languages, including low-resource languages. Existing approaches that utiliz…
gec-metrics: A Unified Library for Grammatical Error Correction Evaluation
Takumi Goto, Yusuke Sakai, Taro Watanabe
We introduce gec-metrics, a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. Our library enables fair system comp…
Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficiencies
Kazuki Hayashi, Shintaro Ozaki, Yusuke Sakai +2
Large-scale Vision-Language Models (LVLMs) are being deployed in real-world settings that require visual inference. As capabilities improve, applications in navigation, education,…
IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation
Kazuki Hayashi, Hidetaka Kamigaito, Shinya Kouda +1
Retrieval-Augmented Generation (RAG) has emerged as a way to complement the in-context knowledge of Large Language Models (LLMs) by integrating external documents. However, real-wo…
TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation
Shintaro Ozaki, Tomoyuki Jinno, Kazuki Hayashi +6
When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible. However, the number of entities is almost c…
Long-Tail Crisis in Nearest Neighbor Language Models
Yuto Nishida, Makoto Morishita, Hiroyuki Deguchi +2
The -nearest-neighbor language model (NN-LM), one of the retrieval-augmented language models, improves the perplexity for given text by directly accessing a large datastore b…
Rethinking Evaluation Metrics for Grammatical Error Correction: Why Use a Different Evaluation Process than Human?
Takumi Goto, Yusuke Sakai, Taro Watanabe
One of the goals of automatic evaluation metrics in grammatical error correction (GEC) is to rank GEC systems such that it matches human preferences. However, current automatic eva…
Tonguescape: Exploring Language Models Understanding of Vowel Articulation
Haruki Sakajo, Yusuke Sakai, Hidetaka Kamigaito +1
Vowels are primarily characterized by tongue position. Humans have discovered these features of vowel articulation through their own experience and explicit objective observation s…