2 citations · 8 across the 12 of their papers we have counts for
4 papers · 1 filter
Beyond Perplexity: UTF-8 Validity in Byte-aware Language Models
Sangwhan Moon, Daisuke Oba, Youmi Ma +2
Byte-level tokenization enables language models to handle any Unicode input, but models can generate invalid UTF-8 sequences when encountering rare or unseen characters. We investi…
Neuron Level Analysis of Large Language Model in Legal Domain Reasoning
Eri Onami, Youmi Ma, Shuhei Kurita +1
We presented a neuron-level analysis of legal-domain reasoning in LLMs, comparing it with other applied domain tasks across seven open-weight models. Using neuron attribution score…
Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
Tatsuya Ichinose, Youmi Ma, Masanari Oi +2
Using responses generated by high-performing large language models (LLMs) for instruction tuning has become a widely adopted approach. However, the existing literature overlooks a…
From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models
Youmi Ma, Naoaki Okazaki
Advances in mechanistic interpretability have identified special attention heads, known as retrieval heads, that are responsible for retrieving information from the context. Howeve…