2 citations · 7 across the 41 of their papers we have counts for
42 papers
Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding
Riza Setiawan Soetedjo, Yusuke Sakai, Hidetaka Kamigaito +2
Minimum Bayes Risk (MBR) decoding enables high-quality text generation by selecting the hypothesis that maximizes a utility metric over sampled pseudo-references. However, it is hi…
ExpArt-KG: Artwork Image Description Generation through Iterative Exploration of Knowledge Graphs
Yuta Kato, Shintaro Ozaki, Kazuki Hayashi +4
Large Vision-Language Models (LVLMs) achieve strong performance on image-grounded text generation and visual question answering. However, it remains difficult for them to comprehen…
Attention-Guided Layer Selection for Contrastive Decoding in Large Language Models
Yusuke Sakai, Natthawut Kertkeidkachorn, Kiyoaki Shirai
Contrastive decoding methods such as DoLa improve the factuality of Large Language Models (LLMs) by contrasting the output distributions of mature and premature layers. However, Do…
Noisy-Channel Minimum Bayes Risk Decoding
Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
Minimum Bayes Risk (MBR) decoding yields more robust and higher-quality text generation than maximum a posteriori (MAP) decoding by selecting hypotheses that maximize expected util…
Multilinguality of Large Language Models From a Structural Perspective
Haruki Sakajo, Yusuke Sakai, Hidetaka Kamigaito +1
Large language models (LLMs) have excelled in processing multiple languages through pre- and post-training on multilingual data, even though English dominates the training data. Pr…
Enhancing Factuality through Consensus and Consistency in Summarization Using Minimum Bayes Risk Decoding
Riza Setiawan Soetedjo, Yusuke Sakai, Hidetaka Kamigaito +3
Improving the quality of model-generated summaries, especially factuality, the accuracy of a summary with respect to its source content, remains a challenge. While reranking could…
CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity
Xuefeng Wei, Zhixuan Wang, Xuan Zhou +5
We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks beyond short-form recognition and QA. CARTBENCH comprises four…
Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction
Takumi Goto, Yusuke Sakai, Taro Watanabe
Grammatical error correction using large language models often suffers from the over-correction issue. To mitigate this, we propose a training-free inference method that performs e…
One Single Hub Text Breaks CLIP: Identifying Vulnerabilities in Cross-Modal Encoders via Hubness
Hiroyuki Deguchi, Katsuki Chousa, Yusuke Sakai
The hubness problem, in which hub embeddings are close to many unrelated examples, occurs often in high-dimensional embedding spaces and may pose a practical threat for purposes su…
HalluCiteChecker: A Lightweight Toolkit for Hallucinated Citation Detection and Verification in the Era of AI Scientists
Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
We introduce HalluCiteChecker, a toolkit for detecting and verifying hallucinated citations in scientific papers. While AI assistant technologies have transformed the academic writ…
The Wisdom of Many Queries: Complexity-Diversity Principle for Dense Retriever Training
Xincan Feng, Noriki Nishida, Yusuke Sakai +1
Synthetic query generation has become essential for training dense retrievers, yet prior methods generate one query per document, focusing solely on query quality. We are the first…
StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
Haruki Sakajo, Frederikus Hudi, Yusuke Sakai +2
Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest thei…
Grammatical Error Correction Evaluation by Optimally Transporting Edit Representation
Takumi Goto, Yusuke Sakai, Taro Watanabe
Automatic evaluation in grammatical error correction (GEC) is crucial for selecting the best-performing systems. Currently, reference-based metrics are a popular choice, which basi…
HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL Conferences
Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
Recently, we have often observed hallucinated citations or references that do not correspond to any existing work in papers under review, preprints, or published papers. Such hallu…
Hacking Neural Evaluation Metrics with Single Hub Text
Hiroyuki Deguchi, Katsuki Chousa, Yusuke Sakai
Strongly human-correlated evaluation metrics serve as an essential compass for the development and improvement of generation models and must be highly reliable and robust. Recent e…
Agreement-Constrained Probabilistic Minimum Bayes Risk Decoding
Koki Natsumi, Hiroyuki Deguchi, Yusuke Sakai +2
Minimum Bayes risk (MBR) decoding generates high-quality translations by maximizing the expected utility of output candidates, but it evaluates all pairwise scores over the candida…
Reliability Crisis of Reference-free Metrics for Grammatical Error Correction
Takumi Goto, Yusuke Sakai, Taro Watanabe
Reference-free evaluation metrics for grammatical error correction (GEC) have achieved high correlation with human judgments. However, these metrics are not designed to evaluate ad…
Multilingual Dialogue Generation and Localization with Dialogue Act Scripting
Justin Vasselli, Eunike Andriani Kardinata, Yusuke Sakai +1
Non-English dialogue datasets are scarce, and models are often trained or evaluated on translations of English-language dialogues, an approach which can introduce artifacts that re…
SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
Ashmari Pramodya, Nirasha Nelki, Heshan Shalinda +6
Large Language Models (LLMs) demonstrate impressive general knowledge and reasoning abilities, yet their evaluation has predominantly focused on global or anglocentric subjects, of…
Languages Still Left Behind: Toward a Better Multilingual Machine Translation Benchmark
Chihiro Taguchi, Seng Mai, Keita Kurabe +4
Multilingual machine translation (MT) benchmarks play a central role in evaluating the capabilities of modern MT systems. Among them, the FLORES+ benchmark is widely used, offering…
Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability
Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
In generative commonsense reasoning tasks such as CommonGen, generative large language models (LLMs) compose sentences that include all given concepts. However, when focusing on in…
IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator
Yusuke Sakai, Takumi Goto, Taro Watanabe
We propose IMPARA-GED, a novel reference-free automatic grammatical error correction (GEC) evaluation method with grammatical error detection (GED) capabilities. We focus on the qu…
Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries
Haruki Sakajo, Yusuke Ide, Justin Vasselli +4
Cross-lingual vocabulary transfer plays a promising role in adapting pre-trained language models to new languages, including low-resource languages. Existing approaches that utiliz…
gec-metrics: A Unified Library for Grammatical Error Correction Evaluation
Takumi Goto, Yusuke Sakai, Taro Watanabe
We introduce gec-metrics, a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. Our library enables fair system comp…