6 citations · 18 across the 13 of their papers we have counts for
13 papers
Is Micro Domain-Adaptive Pre-Training Effective for Real-World Operations? Multi-Step Evaluation Reveals Potential and Bottlenecks
Masaya Tsunokake, Yuta Koreeda, Terufumi Morishita +3
When applying LLMs to real-world enterprise operations, LLMs need to handle proprietary knowledge in small domains of specific operations (). A previous stu…
Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates
Atsuki Yamaguchi, Terufumi Morishita, Aline Villavicencio +1
Expanding the linguistic diversity of instruct large language models (LLMs) is crucial for global accessibility but is often hindered by the reliance on costly specialized target l…
A Nested Watermark for Large Language Models
Koichi Nagatsuka, Terufumi Morishita, Yasuhiro Sogawa
The rapid advancement of large language models (LLMs) has raised concerns regarding their potential misuse, particularly in generating fake news and misinformation. To address thes…
Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus
Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi +1
Large language models (LLMs) are capable of solving a wide range of tasks, yet they have struggled with reasoning. To address this, we propose $\textbf{Additional Logic Training (A…
Adapting Chat Language Models Using Only Target Unlabeled Language Data
Atsuki Yamaguchi, Terufumi Morishita, Aline Villavicencio +1
Vocabulary expansion (VE) is the de-facto approach to language adaptation of large language models (LLMs) by adding new tokens and continuing pre-training on target data. While thi…
appjsonify: An Academic Paper PDF-to-JSON Conversion Toolkit
Atsuki Yamaguchi, Terufumi Morishita
We present appjsonify, a Python-based PDF-to-JSON conversion toolkit for academic papers. It parses a PDF file using several visual-based document layout analysis models and rule-b…
LARCH: Large Language Model-based Automatic Readme Creation with Heuristics
Yuta Koreeda, Terufumi Morishita, Osamu Imaichi +1
Writing a readme is a crucial aspect of software development as it plays a vital role in managing and reusing program code. Though it is a pain point for many developers, automatic…
Learning Deductive Reasoning from Synthetic Corpus based on Formal Logic
Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi +1
We study a synthetic corpus based approach for language models (LMs) to acquire logical deductive reasoning ability. The previous studies generated deduction examples using specifi…
How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese
Takuro Fujii, Koki Shibata, Atsuki Yamaguchi +2
This paper investigates the effect of tokenizers on the downstream performance of pretrained language models (PLMs) in scriptio continua languages where no explicit spaces exist be…
How does the task complexity of masked pretraining objectives affect downstream performance?
Atsuki Yamaguchi, Hiroaki Ozaki, Terufumi Morishita +2
Masked language modeling (MLM) is a widely used self-supervised pretraining objective, where a model needs to predict an original token that is replaced with a mask given contexts.…
Controlling keywords and their positions in text generation
Yuichi Sasazawa, Terufumi Morishita, Hiroaki Ozaki +2
One of the challenges in text generation is to control text generation as intended by the user. Previous studies proposed specifying the keywords that should be included in the gen…
Rethinking Fano's Inequality in Ensemble Learning
Terufumi Morishita, Gaku Morio, Shota Horiguchi +2
We propose a fundamental theory on ensemble learning that answers the central question: what factors make an ensemble system good or bad? Previous studies used a variant of Fano's…
Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing
Yuta Koreeda, Gaku Morio, Terufumi Morishita +2
This paper describes the proposed system of the Hitachi team for the Cross-Framework Meaning Representation Parsing (MRP 2019) shared task. In this shared task, the participating s…