35 citations · 151 across the 44 of their papers we have counts for
46 papers · 1 filter
Likelihood-based Mitigation of Evaluation Bias in Large Language Models
Masanari Oi, Masahiro Kaneko, Ryuto Koike +2
Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics. However, the likelihood, a measure of LLM's plausibility for a sent…
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki +1
There exist both scalable tasks, like reading comprehension and fact-checking, where model performance improves with model size, and unscalable tasks, like arithmetic reasoning and…
How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection
Ryuto Koike, Masahiro Kaneko, Naoaki Okazaki
To combat the misuse of Large Language Models (LLMs), many recent studies have presented LLM-generated-text detectors with promising performance. When users instruct LLMs to genera…
SAIE Framework: Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks
Mengsay Loem, Masahiro Kaneko, Naoaki Okazaki
Large Language Models (LLMs) can justify or critique their predictions through discussions with other models or humans, thereby enriching their intrinsic understanding of instances…
Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction
Masahiro Kaneko, Naoaki Okazaki
In Grammatical Error Correction (GEC), it is crucial to ensure the user's comprehension of a reason for correction. Existing studies present tokens, examples, and hints as to the b…
The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Pre-trained language models trained on large-scale data have learned serious levels of social biases. Consequently, various methods have been proposed to debias pre-trained models.…
Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels
Panatchakorn Anantaprayoon, Masahiro Kaneko, Naoaki Okazaki
Discriminatory gender biases have been found in Pre-trained Language Models (PLMs) for multiple languages. In Natural Language Inference (NLI), existing bias evaluation methods hav…
OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples
Ryuto Koike, Masahiro Kaneko, Naoaki Okazaki
Large Language Models (LLMs) have achieved human-level fluency in text generation, making it difficult to distinguish between human-written and LLM-generated texts. This poses a gr…
Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods
Mengsay Loem, Masahiro Kaneko, Sho Takase +1
Large-scale pre-trained language models such as GPT-3 have shown remarkable performance across various natural language processing tasks. However, applying prompt-based methods wit…
Reducing Sequence Length by Predicting Edit Operations with Large Language Models
Masahiro Kaneko, Naoaki Okazaki
Large Language Models (LLMs) have demonstrated remarkable performance in various tasks and gained significant attention. LLMs are also used for local sequence transduction tasks, i…
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach
Masahiro Kaneko, Graham Neubig, Naoaki Okazaki
Humans work together to solve common problems by having discussions, explaining, and agreeing or disagreeing with each other. Similarly, if a system can have discussions with human…
Semantic Specialization for Knowledge-based Word Sense Disambiguation
Sakae Mizuki, Naoaki Okazaki
A promising approach for knowledge-based Word Sense Disambiguation (WSD) is to select the sense whose contextualized embeddings computed for its definition sentence are closest to…
DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction
Youmi Ma, An Wang, Naoaki Okazaki
Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the…
Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Numerous types of social biases have been identified in pre-trained language models (PLMs), and various intrinsic bias evaluation measures have been proposed for quantifying those…
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps
Hiroki Iida, Naoaki Okazaki
IR models using a pretrained language model significantly outperform lexical approaches like BM25. In particular, SPLADE, which encodes texts to sparse vectors, is an effective mod…
Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists…
Nearest Neighbor Non-autoregressive Text Generation
Ayana Niwa, Sho Takase, Naoaki Okazaki
Non-autoregressive (NAR) models can generate sentences with less computation than autoregressive models but sacrifice generation quality. Previous studies addressed this issue thro…
Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention
Mengsay Loem, Sho Takase, Masahiro Kaneko +1
Impressive performance of Transformer has been attributed to self-attention, where dependencies between entire input in a sequence are considered at every position. In this work, w…
Gender Bias in Masked Language Models for Multiple Languages
Masahiro Kaneko, Aizhan Imankulova, Danushka Bollegala +1
Masked Language Models (MLMs) pre-trained by predicting masked tokens on large corpora have been used successfully in natural language processing tasks for a variety of languages.…
PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation
Ao Liu, Haoyu Dong, Naoaki Okazaki +2
Logical table-to-text generation is a task that involves generating logically faithful sentences from tables, which requires models to derive logical level facts from table records…
Gender Bias in Meta-Embeddings
Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki
Different methods have been proposed to develop meta-embeddings from a given set of source embeddings. However, the source embeddings can contain unfair gender-related biases, and…
Semi-Supervised Formality Style Transfer with Consistency Training
Ao Liu, An Wang, Naoaki Okazaki
Formality style transfer (FST) is a task that involves paraphrasing an informal sentence into a formal one without altering its meaning. To address the data-scarcity problem of exi…
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation
Sho Takase, Tatsuya Hiraoka, Naoaki Okazaki
Subword regularizations use multiple subword segmentations during training to improve the robustness of neural machine translation models. In previous subword regularizations, we u…
Interpretability for Language Learners Using Example-Based Grammatical Error Correction
Masahiro Kaneko, Sho Takase, Ayana Niwa +1
Grammatical Error Correction (GEC) should not focus only on high accuracy of corrections but also on interpretability for language learning. However, existing neural-based GEC mode…