3 papers
cs.CL2026
It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention
Raito Kiya, Satoki Ohashi, Kosuke Sato +6
Large Language Models (LLMs) often exhibit "Attention Sink" (AS) and the accompanying "Massive Activations" (MAs) at the initial position of a sequence. These phenomena frequently…
cs.CL2025
Instruction-Following Evaluation of Large Vision-Language Models
Daiki Shiono, Shumpei Miyawaki, Ryota Tanaka +1
Following the initial flourishing of large language models (LLMs), there has been a surge in proposed large vision-language models (LVLMs) that integrate LLMs with vision capabilit…
cs.CL2024
Detecting Response Generation Not Requiring Factual Judgment
Ryohei Kamei, Daiki Shiono, Reina Akama +1
With the remarkable development of large language models (LLMs), ensuring the factuality of output has become a challenge. However, having all the contents of the response with giv…