2 citations · 8 across the 12 of their papers we have counts for
12 papers
Beyond Perplexity: UTF-8 Validity in Byte-aware Language Models
Sangwhan Moon, Daisuke Oba, Youmi Ma +2
Byte-level tokenization enables language models to handle any Unicode input, but models can generate invalid UTF-8 sequences when encountering rare or unseen characters. We investi…
Neuron Level Analysis of Large Language Model in Legal Domain Reasoning
Eri Onami, Youmi Ma, Shuhei Kurita +1
We presented a neuron-level analysis of legal-domain reasoning in LLMs, comparing it with other applied domain tasks across seven open-weight models. Using neuron attribution score…
Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
Tatsuya Ichinose, Youmi Ma, Masanari Oi +2
Using responses generated by high-performing large language models (LLMs) for instruction tuning has become a widely adopted approach. However, the existing literature overlooks a…
From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models
Youmi Ma, Naoaki Okazaki
Advances in mechanistic interpretability have identified special attention heads, known as retrieval heads, that are responsible for retrieving information from the context. Howeve…
QuantumBench: A Benchmark for Quantum Problem Solving
Shunya Minami, Tatsuya Ishigaki, Ikko Hamamura +6
Large language models are now integrated into many scientific workflows, accelerating data analysis, hypothesis generation, and design space exploration. In parallel with this grow…
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Kazuki Fujii, Yukito Tajima, Sakae Mizuki +14
The performance of large language models (LLMs) in program synthesis and mathematical reasoning is fundamentally limited by the quality of their pre-training corpora. We introduce…
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models
Youmi Ma, Sakae Mizuki, Kazuki Fujii +12
Instruction tuning is crucial for enabling Large Language Models (LLMs) to solve real-world tasks. Prior work has shown the effectiveness of instruction-tuning data synthesized sol…
Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMs
Koshiro Saito, Sakae Mizuki, Masanari Ohi +11
Why do we build local large language models (LLMs)? What should a local LLM learn from the target language? Which abilities can be transferred from other languages? Do language-spe…
Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer
Youmi Ma, An Wang, Naoaki Okazaki
Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document. While studies have been conducted on English DocRE, limited attenti…
Sampling-based Pseudo-Likelihood for Membership Inference Attacks
Masahiro Kaneko, Youmi Ma, Yuki Wata +1
Large Language Models (LLMs) are trained on large-scale web data, which makes it difficult to grasp the contribution of each text. This poses the risk of leaking inappropriate data…
DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction
Youmi Ma, An Wang, Naoaki Okazaki
Document-level relation extraction (DocRE) is the task of identifying all relations between each entity pair in a document. Evidence, defined as sentences containing clues for the…
Named Entity Recognition and Relation Extraction using Enhanced Table Filling by Contextualized Representations
Youmi Ma, Tatsuya Hiraoka, Naoaki Okazaki
In this study, a novel method for extracting named entities and relations from unstructured text based on the table representation is presented. By using contextualized word embedd…